9 CSIT |release| follows a common structured naming convention for all
10 performance and system functional tests, introduced in CSIT rls1701.
12 The naming should be intuitive for majority of the tests. Complete
13 description of CSIT test naming convention is provided on
14 `CSIT test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
15 Below few illustrative examples of the naming usage for test suites across CSIT
16 performance, functional and Honeycomb management test areas.
21 The CSIT approach is to use tree naming convention and to encode following
22 testing information into test suite and test case names:
24 #. packet network port configuration
26 * port type, physical or virtual;
28 * NIC model, if applicable;
29 * port-NIC locality, if applicable;
31 #. packet encapsulations;
33 #. VPP packet processing
35 * packet forwarding mode;
36 * packet processing function(s);
38 #. packet forwarding path
40 * if present, network functions (processes, containers, VMs) and their
41 topology within the computer;
43 #. main measured variable, type of test.
45 Proposed convention is to encode ports and NICs on the left (underlay),
46 followed by outer-most frame header, then other stacked headers up to the
47 header processed by vSwitch-VPP, then VPP forwarding function, then encap on
48 vhost interface, number of vhost interfaces, number of VMs. If chained VMs
49 present, they get added on the right. Test topology is expected to be
50 symmetric, in other words packets enter and leave SUT through ports specified
51 on the left of the test name. Here some examples to illustrate the convention
52 followed by the complete legend, and tables mapping the new test filenames to
58 CSIT test suite naming examples (filename.robot) for common tested VPP
61 1. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P**
63 * *PortNICConfig-WireEncapsulation-PacketForwardingFunction-
64 PacketProcessingFunction1-...-PacketProcessingFunctionN-TestType*
65 * *10ge2p1x520-dot1q-l2bdbasemaclrn-ndrdisc.robot* => 2 ports of 10GE on Intel
66 x520 NIC, dot1q tagged Ethernet, L2 bridge-domain baseline switching with
67 MAC learning, NDR throughput discovery.
68 * *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrchk.robot* => 2 ports of 10GE on
69 Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain baseline switching
70 with MAC learning, NDR throughput discovery.
71 * *10ge2p1x520-ethip4-ip4base-ndrdisc.robot* => 2 ports of 10GE on Intel x520
72 NIC, IPv4 baseline routed forwarding, NDR throughput discovery.
73 * *10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot* => 2 ports of 10GE on Intel
74 x520 NIC, IPv6 scaled up routed forwarding, NDR throughput discovery.
75 * *10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot* => 2 ports of 10GE on
76 Intel x520 NIC, IPv4 baseline routed forwarding, ingress Access Control
77 Lists baseline matching on destination, NDR throughput discovery.
78 * *40ge2p1vic1385-ethip4-ip4base-ndrdisc.robot* => 2 ports of 40GE on Cisco
79 vic1385 NIC, IPv4 baseline routed forwarding, NDR throughput discovery.
80 * *eth2p-ethip4-ip4base-func.robot* => 2 ports of Ethernet, IPv4 baseline
81 routed forwarding, functional tests.
83 2. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC,
84 P2V2P, NIC2VMchain2NIC, P2V2V2P**
86 * *PortNICConfig-WireEncapsulation-PacketForwardingFunction-
87 PacketProcessingFunction1-...-PacketProcessingFunctionN-VirtEncapsulation-
88 VirtPortConfig-VMconfig-TestType*
89 * *10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 ports
90 of 10GE on Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain switching
91 to/from two vhost interfaces and one VM, NDR throughput discovery.
92 * *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2
93 ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain
94 switching to/from two vhost interfaces and one VM, NDR throughput discovery.
95 * *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc.robot* => 2
96 ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain
97 switching to/from four vhost interfaces and two VMs, NDR throughput
99 * *eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot* => 2 ports of
100 Ethernet, IPv4 VXLAN Ethernet, L2 bridge-domain switching to/from two vhost
101 interfaces and one VM, functional tests.
103 3. **API CRUD tests - Create (Write), Read (Retrieve), Update (Modify), Delete
104 (Destroy) operations for configuration and operational data**
106 * *ManagementTestKeyword-ManagementOperation-ManagedFunction1-...-
107 ManagedFunctionN-ManagementAPI1-ManagementAPIN-TestType*
108 * *mgmt-cfg-lisp-apivat-func* => configuration of LISP with VAT API calls,
110 * *mgmt-cfg-l2bd-apihc-apivat-func* => configuration of L2 Bridge-Domain with
111 Honeycomb API and VAT API calls, functional tests.
112 * *mgmt-oper-int-apihcnc-func* => reading status and operational data of
113 interface with Honeycomb NetConf API calls, functional tests.
114 * *mgmt-cfg-int-tap-apihcnc-func* => configuration of tap interfaces with
115 Honeycomb NetConf API calls, functional tests.
116 * *mgmt-notif-int-subint-apihcnc-func* => notifications of interface and
117 sub-interface events with Honeycomb NetConf Notifications, functional tests.
119 For complete description of CSIT test naming convention please refer to `CSIT
120 test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.