8 All CSIT test cases are labelled with Robot Framework tags used to allow for
9 easy test case type identification, test case grouping and selection for
10 execution. Following sections list currently used CSIT tags and their
13 ## Testbed Topology Tags
15 **2_NODE_DOUBLE_LINK_TOPO**
17 2 nodes connected in a circular topology with two links interconnecting
20 **2_NODE_SINGLE_LINK_TOPO**
22 2 nodes connected in a circular topology with at least one link
23 interconnecting devices.
25 **3_NODE_DOUBLE_LINK_TOPO**
27 3 nodes connected in a circular topology with two links interconnecting
30 **3_NODE_SINGLE_LINK_TOPO**
32 3 nodes connected in a circular topology with at least one link
33 interconnecting devices.
39 Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch)
40 and csit-vpp-verify jobs (i.e. CSIT patch).
44 Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch).
50 DUTs and TGs are running on bare metal.
54 DUTs and TGs are running in virtual environment.
58 DUTs with VPP and capable of running Virtual Machine.
62 **NIC_Intel-X520-DA2**
78 **NIC_Cisco-VIC-1227**
82 **NIC_Cisco-VIC-1385**
86 **NIC_Amazon-Nitro-50G**
94 2x10,000 entries in single fib table
98 2x100,000 entries in single fib table
102 2x500,000 entries in single fib table
106 2x1,000,000 entries in single fib table
110 Test with 1 L2 bridge domain.
114 Test with 10 L2 bridge domains.
118 Test with 100 L2 bridge domains.
122 Test with 1000 L2 bridge domains.
126 Test with 1 VLAN sub-interface.
130 Test with 10 VLAN sub-interfaces.
134 Test with 100 VLAN sub-interfaces.
138 Test with 1000 VLAN sub-interfaces.
142 Test with 1 VXLAN tunnel.
146 Test with 10 VXLAN tunnels.
150 Test with 100 VXLAN tunnels.
154 Test with 1000 VXLAN tunnels.
158 IPSec in tunnel mode - {t} tunnels.
162 Traffic flow with {u} unique IPs (users) in one direction.
163 {u}=(1,10,100,1000,2000,4000).
167 Traffic stream with 100 unique flows (10 IPs/users x 10 UDP ports) in one
172 Traffic stream with 10 000 unique flows (10 IPs/users x 1000 UDP ports) in
177 Traffic stream with 100 000 unique flows (100 IPs/users x 1000 UDP ports) in
182 Stateless or stateful traffic stream with {h} client source IP4 addresses,
183 usually with 63 flow differing in source port number. Could be UDP or TCP.
184 If NAT is used, the clients are inside. Outside IP range can differ.
185 {h}=(1024,4096,16384,65536,262144).
189 Test with {t} GENEVE IPv4 tunnel. {t}=(1,4,16,64,256,1024)
191 ## Test Category Tags
195 All vpp_device functional test cases.
199 All performance test cases.
201 ## VPP Device Type Tags
205 All test cases that uses Scapy for packet generation and validation.
207 ## Performance Type Tags
211 Single test finding both No Drop Rate and Partial Drop Rate simultaneously.
212 The search is done by optimized algorithm which performs
213 multiple trial runs at different durations and transmit rates.
214 The results come from the final trials, which have duration of 30 seconds.
218 Performance tests where TG sends the traffic at maximum rate (line rate)
219 and reports total sent/received packets over trial duration.
220 The result is an average of 10 trials of 1 second duration.
224 Performance tests using PLRsearch to find the critical load.
228 Performance tests aimed to measure lost packets (time) when performing
229 reconfiguration while full throughput offered load is applied.
231 ## Ethernet Frame Size Tags
233 These are describing the traffic offered by Traffic Generator,
234 "primary" traffic in case of asymmetric load.
235 For traffic between DUTs, or for "secondary" traffic, see ${overhead} value.
239 {b} Bytes frames used for test.
243 IMIX frame sequence (28x 64B, 16x 570B, 4x 1518B) used for test.
249 Baseline test cases, no encapsulation, no feature(s) configured in tests.
250 No scaling whatsoever, beyond minimum needed for RSS.
254 IPv4 baseline test cases, no encapsulation, no feature(s) configured in
255 tests. Minimal number of routes. Other quantities may be scaled.
259 IPv6 baseline test cases, no encapsulation, no feature(s) configured in
264 L2XC baseline test cases, no encapsulation, no feature(s) configured in
269 L2BD baseline test cases, no encapsulation, no feature(s) configured in
274 L2PATCH baseline test cases, no encapsulation, no feature(s) configured in
279 Scale test cases. Other tags specify which quantities are scaled.
280 Also applies if scaling is set on TG only (e.g. DUT works as IP4BASE).
284 Test cases where encapsulation is used. Use also encapsulation tag(s).
288 At least one feature is configured in test cases. Use also feature tag(s).
292 Tests which use any kind of UDP traffic (STL or ASTF profile).
296 Tests which use any kind of TCP traffic (STL or ASTF profile).
300 Tests which test trex traffic without any software DUTs in the traffic path.
304 Tests which use unidirectional UDP traffic (STL profile only).
308 Tests which use bidirectional UDP traffic (STL profile only).
312 Tests which measure connections per second on minimal UDP pseudoconnections.
313 This implies ASTF traffic profile is used.
314 This tag selects specific output processing in PAL.
318 Tests which measure connections per second on empty TCP connections.
319 This implies ASTF traffic profile is used.
320 This tag selects specific output processing in PAL.
324 Tests which measure requests per second on empty TCP connections.
325 This implies ASTF traffic profile is used.
326 This tag selects specific output processing in PAL.
330 Tests which measure packets per second on lightweight UDP transactions.
331 This implies ASTF traffic profile is used.
332 This tag selects specific output processing in PAL.
336 Tests which measure packets per second on lightweight TCP transactions.
337 This implies ASTF traffic profile is used.
338 This tag selects specific output processing in PAL.
342 Tests which use traffic formed of valid HTTP requests (and responses).
346 LDP NGINX is un-modified NGINX with VPP via LD_PRELOAD.
350 Performance tests that measure throughput of multiple VNF and CNF
351 service topologies at different service densities.
353 ## NF Service Density Tags
357 NF service density tests with VNF or CNF service chain topology(ies).
361 NF service density tests with CNF service pipeline topology(ies).
365 NF service density tests with DPDK l3fwd IPv4 routing as NF workload.
369 NF service density tests with VPP IPv4 routing as NF workload.
373 Service density matrix locator {r}R{c}C, {r}Row denoting number of
374 service instances, {c}Column denoting number of NFs per service
375 instance. {r}=(1,2,4,6,8,10), {c}=(1,2,4,6,8,10).
379 Service density {n}VM{t}T, {n}Number of NF Qemu VMs, {t}Number of threads
384 Service density {n}DCR{t}T, {n}Number of NF Docker containers, {t}Number of
389 {n}Number of chains (or pipelines) added (and/or removed)
392 ## Forwarding Mode Tags
396 VPP L2 bridge-domain, L2 MAC static.
400 VPP L2 bridge-domain, L2 MAC learning.
404 VPP L2 point-to-point cross-connect.
408 VPP IPv4 routed forwarding.
412 VPP IPv6 routed forwarding.
414 **LOADBALANCER_MAGLEV**
416 VPP Load balancer maglev mode.
418 **LOADBALANCER_L3DSR**
420 VPP Load balancer l3dsr mode.
422 **LOADBALANCER_NAT4**
424 VPP Load balancer nat4 mode.
428 Mode, where NICs from the same physical server are directly
429 connected with a cable.
453 IPv4 overlay (IPv4 payload).
457 IPv6 overlay (IPv6 payload).
463 All test cases with dot1q.
467 All test cases with dot1ad.
469 ## Encapsulation Tags
473 All test cases with base Ethernet (no encapsulation).
477 All test cases with LISP.
481 All test cases with LISP-GPE.
485 All test cases with LISP_IP4o4.
489 All test cases with LISPGPE_IP4o4.
493 All test cases with LISPGPE_IP6o4.
497 All test cases with LISPGPE_IP4o6.
501 All test cases with LISPGPE_IP6o6.
505 All test cases with Vxlan.
509 All test cases with VXLAN-GPE.
513 All test cases with GRE.
517 All test cases with GTPU.
521 All test cases with GTPU_HWACCEL.
525 All test cases with IPSEC.
529 All test cases with WIREGUARD.
533 All test cases with Segment routing over IPv6 dataplane.
537 All SRv6 test cases with single SID.
541 All SRv6 test cases with two SIDs and with decapsulation.
543 **SRv6_2SID_NODECAP**
545 All SRv6 test cases with two SIDs and without decapsulation.
549 All test cases with GENEVE.
553 All test cases with GENEVE tunnel in L3 mode.
557 All test cases with FLOW.
561 All test cases with FLOW_DIR.
565 All test cases with FLOW_RSS.
569 All test cases with NTUPLE.
573 All test cases with L2TPV3.
579 All test cases which use physical interface(s).
583 All test cases which uses Generic Segmentation Offload.
587 All test cases which uses VHOST.
591 All test cases which uses VHOST DPDK driver with qemu queue size set
596 All test cases which uses VIRTIO native VPP driver.
600 All test cases which uses VIRTIO native VPP driver with qemu queue size set
605 All test cases which uses VM with optimised scheduler policy.
609 All test cases which uses TUN and TAP.
613 All test cases which uses AFPKT.
617 All test cases which uses Netmap.
621 All test cases which uses Memif.
625 All test cases which uses only single Memif connection per DUT. One DUT
626 instance is running in container having one physical interface exposed to
631 All test cases which uses link bonding (BondEthernet interface).
635 All test cases which uses DPDK link bonding.
639 All test cases which uses VPP link bonding.
643 All test cases which uses link bonding with mode XOR.
647 All test cases which uses link bonding with mode LACP.
651 All test cases which uses link bonding with load-balance mode l34.
655 All test cases which use {n} link(s) for link bonding.
659 All test cases which NIC Driver for DUT is set to {d}. Default is VFIO_PCI.
660 {d}=(AVF, RDMA_CORE, VFIO_PCI, AF_XDP).
664 All test cases which NIC Driver for TG is set to {d}. Default is IGB_UIO.
665 {d}=(RDMA_CORE, IGB_UIO).
669 All test cases which RXQ size (RX descriptors) are set to {n}. Default is 0,
670 which means VPP (API) default.
674 All test cases which TXQ size (TX descriptors) are set to {n}. Default is 0,
675 which means VPP (API) default.
689 NAT44 configured and tested.
693 NAT44 configured and tested.
697 ACL plugin configured and tested.
701 ACL plugin configured and tested on input path.
705 ACL plugin configured and tested on output path.
709 ACL plugin configured and tested in stateless mode (permit action).
713 ACL plugin configured and tested in stateful mode (permit+reflect action).
717 ACL plugin configured and tested with 1 not-hitting ACE.
721 ACL plugin configured and tested with 10 not-hitting ACEs.
725 ACL plugin configured and tested with 50 not-hitting ACEs.
729 SRv6 endpoint to SR-unaware appliance via proxy.
733 SRv6 endpoint to SR-unaware appliance via static proxy.
737 SRv6 endpoint to SR-unaware appliance via dynamic proxy.
741 SRv6 endpoint to SR-unaware appliance via masquerading proxy.
755 IPSec in transport mode.
759 IPSec in tunnel mode.
763 IPSec in interface mode.
767 IPSec using AES algorithms.
771 IPSec using AES 128 CBC algorithms.
775 IPSec using AES 128 GCM algorithms.
779 IPSec using AES 256 GCM algorithms.
783 IPSec using HMAC integrity algorithms.
787 IPSec using HMAC SHA 256 integrity algorithms.
791 IPSec using HMAC SHA 512 integrity algorithms.
795 IPSec using crypto sw scheduler engine.
799 IPSec policy mode with spd fast path enabled.
801 ## Client-Workload Tags
805 All test cases which use at least one virtual machine.
809 All test cases which use Linux container and LXC utils.
813 All test cases which use at least one Docker container.
817 All test cases which use Docker as container manager.
821 All test cases with specific APP use.
823 ## Container Orchestration Tags
827 {n} VPP running in {n} Docker container(s) acting as a VSWITCH.
832 {n} VPP running in {n} Docker container(s) acting as a VNF work load.
835 ## Multi-Threading Tags
840 All test cases using single poll mode thread.
845 All test cases using more then one poll mode driver thread.
849 All test cases with packet processing on {n} socket(s). {n}=(1,2).
853 {c} worker thread pinned to {c} dedicated physical core; or if
854 HyperThreading is enabled, {c}*2 worker threads each pinned to a separate
855 logical core within 1 dedicated physical core. Main thread pinned to core 1.
861 {t} worker threads pinned to {c} dedicated physical cores. Main thread
862 pinned to core 1. By default CSIT is configuring same amount of receive
863 queues per interface as worker threads. {t}=(1,2,4,8), {t}=(1,2,4).