8 All CSIT test cases are labelled with Robot Framework tags used to allow for
9 easy test case type identification, test case grouping and selection for
10 execution. Following sections list currently used CSIT tags and their
13 ## Testbed Topology Tags
15 **2_NODE_DOUBLE_LINK_TOPO**
17 2 nodes connected in a circular topology with two links interconnecting
20 **2_NODE_SINGLE_LINK_TOPO**
22 2 nodes connected in a circular topology with at least one link
23 interconnecting devices.
25 **3_NODE_DOUBLE_LINK_TOPO**
27 3 nodes connected in a circular topology with two links interconnecting
30 **3_NODE_SINGLE_LINK_TOPO**
32 3 nodes connected in a circular topology with at least one link
33 interconnecting devices.
39 Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP
40 patch) and csit-vpp-verify jobs (i.e. CSIT patch).
44 Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP
51 DUTs and TGs are running on bare metal.
55 DUTs and TGs are running in virtual environment.
59 DUTs with VPP and capable of running Virtual Machine.
63 **NIC_Intel-X520-DA2**
79 **NIC_Cisco-VIC-1227**
83 **NIC_Cisco-VIC-1385**
87 **NIC_Amazon-Nitro-50G**
95 2x10,000 entries in single fib table
99 2x100,000 entries in single fib table
103 2x500,000 entries in single fib table
107 2x1,000,000 entries in single fib table
111 Test with 1 L2 bridge domain.
115 Test with 10 L2 bridge domains.
119 Test with 100 L2 bridge domains.
123 Test with 1000 L2 bridge domains.
127 Test with 1 VLAN sub-interface.
131 Test with 10 VLAN sub-interfaces.
135 Test with 100 VLAN sub-interfaces.
139 Test with 1000 VLAN sub-interfaces.
143 Test with 1 VXLAN tunnel.
147 Test with 10 VXLAN tunnels.
151 Test with 100 VXLAN tunnels.
155 Test with 1000 VXLAN tunnels.
159 IPSec in tunnel mode - {t} tunnels.
163 Traffic flow with {u} unique IPs (users) in one direction.
164 {u}=(1,10,100,1000,2000,4000).
168 Traffic stream with 100 unique flows (10 IPs/users x 10 UDP ports) in
173 Traffic stream with 10 000 unique flows (10 IPs/users x 1000 UDP ports)
178 Traffic stream with 100 000 unique flows (100 IPs/users x 1000 UDP
179 ports) in one direction.
183 Stateless or stateful traffic stream with {h} client source IP4
184 addresses, usually with 63 flow differing in source port number.
185 Could be UDP or TCP. If NAT is used, the clients are inside.
186 Outside IP range can differ.
187 {h}=(1024,4096,16384,65536,262144).
191 Test with {t} GENEVE IPv4 tunnel.
192 {t}=(1,4,16,64,256,1024)
194 ## Test Category Tags
198 All vpp_device functional test cases.
202 All performance test cases.
204 ## VPP Device Type Tags
208 All test cases that uses Scapy for packet generation and validation.
210 ## Performance Type Tags
214 Single test finding both No Drop Rate and Partial Drop Rate
215 simultaneously. The search is done by optimized algorithm which
216 performs multiple trial runs at different durations and transmit
217 rates. The results come from the final trials, which have duration
222 Performance tests where TG sends the traffic at maximum rate (line rate)
223 and reports total sent/received packets over trial duration.
224 The result is an average of 10 trials of 1 second duration.
228 Performance tests using PLRsearch to find the critical load.
232 Performance tests aimed to measure lost packets (time) when performing
233 reconfiguration while full throughput offered load is applied.
235 ## Ethernet Frame Size Tags
237 These are describing the traffic offered by Traffic Generator,
238 "primary" traffic in case of asymmetric load.
239 For traffic between DUTs, or for "secondary" traffic, see ${overhead} value.
243 {b} Bytes frames used for test.
247 IMIX frame sequence (28x 64B, 16x 570B, 4x 1518B) used for test.
253 Baseline test cases, no encapsulation, no feature(s) configured in tests.
254 No scaling whatsoever, beyond minimum needed for RSS.
258 IPv4 baseline test cases, no encapsulation, no feature(s) configured in
259 tests. Minimal number of routes. Other quantities may be scaled.
263 IPv6 baseline test cases, no encapsulation, no feature(s) configured in
268 L2XC baseline test cases, no encapsulation, no feature(s) configured in
273 L2BD baseline test cases, no encapsulation, no feature(s) configured in
278 L2PATCH baseline test cases, no encapsulation, no feature(s) configured
283 Scale test cases. Other tags specify which quantities are scaled.
284 Also applies if scaling is set on TG only (e.g. DUT works as IP4BASE).
288 Test cases where encapsulation is used. Use also encapsulation tag(s).
292 At least one feature is configured in test cases. Use also feature
297 Tests which use any kind of UDP traffic (STL or ASTF profile).
301 Tests which use any kind of TCP traffic (STL or ASTF profile).
305 Tests which test trex traffic without any software DUTs in the
310 Tests which use unidirectional UDP traffic (STL profile only).
314 Tests which use bidirectional UDP traffic (STL profile only).
318 Tests which measure connections per second on minimal UDP
319 pseudoconnections. This implies ASTF traffic profile is used.
323 Tests which measure connections per second on empty TCP connections.
324 This implies ASTF traffic profile is used.
328 Tests which measure requests per second on empty TCP connections.
329 This implies ASTF traffic profile is used.
333 Tests which measure packets per second on lightweight UDP transactions.
334 This implies ASTF traffic profile is used.
338 Tests which measure packets per second on lightweight TCP transactions.
339 This implies ASTF traffic profile is used.
343 Tests which use traffic formed of valid HTTP requests (and responses).
347 LDP NGINX is un-modified NGINX with VPP via LD_PRELOAD.
351 Performance tests that measure throughput of multiple VNF and CNF
352 service topologies at different service densities.
354 ## NF Service Density Tags
358 NF service density tests with VNF or CNF service chain topology(ies).
362 NF service density tests with CNF service pipeline topology(ies).
366 NF service density tests with DPDK l3fwd IPv4 routing as NF workload.
370 NF service density tests with VPP IPv4 routing as NF workload.
374 Service density matrix locator {r}R{c}C, {r}Row denoting number of
375 service instances, {c}Column denoting number of NFs per service
377 {r}=(1,2,4,6,8,10), {c}=(1,2,4,6,8,10).
381 Service density {n}VM{t}T, {n}Number of NF Qemu VMs, {t}Number of
386 Service density {n}DCR{t}T, {n}Number of NF Docker containers,
387 {t}Number of threads per NF.
391 {n}Number of chains (or pipelines) added (and/or removed)
394 ## Forwarding Mode Tags
398 VPP L2 bridge-domain, L2 MAC static.
402 VPP L2 bridge-domain, L2 MAC learning.
406 VPP L2 point-to-point cross-connect.
410 VPP IPv4 routed forwarding.
414 VPP IPv6 routed forwarding.
416 **LOADBALANCER_MAGLEV**
418 VPP Load balancer maglev mode.
420 **LOADBALANCER_L3DSR**
422 VPP Load balancer l3dsr mode.
424 **LOADBALANCER_NAT4**
426 VPP Load balancer nat4 mode.
430 Mode, where NICs from the same physical server are directly
431 connected with a cable.
455 IPv4 overlay (IPv4 payload).
459 IPv6 overlay (IPv6 payload).
465 All test cases with dot1q.
469 All test cases with dot1ad.
471 ## Encapsulation Tags
475 All test cases with base Ethernet (no encapsulation).
479 All test cases with LISP.
483 All test cases with LISP-GPE.
487 All test cases with LISP_IP4o4.
491 All test cases with LISPGPE_IP4o4.
495 All test cases with LISPGPE_IP6o4.
499 All test cases with LISPGPE_IP4o6.
503 All test cases with LISPGPE_IP6o6.
507 All test cases with Vxlan.
511 All test cases with VXLAN-GPE.
515 All test cases with GRE.
519 All test cases with GTPU.
523 All test cases with GTPU_HWACCEL.
527 All test cases with IPSEC.
531 All test cases with WIREGUARD.
535 All test cases with Segment routing over IPv6 dataplane.
539 All SRv6 test cases with single SID.
543 All SRv6 test cases with two SIDs and with decapsulation.
545 **SRv6_2SID_NODECAP**
547 All SRv6 test cases with two SIDs and without decapsulation.
551 All test cases with GENEVE.
555 All test cases with GENEVE tunnel in L3 mode.
559 All test cases with FLOW.
563 All test cases with FLOW_DIR.
567 All test cases with FLOW_RSS.
571 All test cases with NTUPLE.
575 All test cases with L2TPV3.
581 All test cases which use physical interface(s).
585 All test cases which uses Generic Segmentation Offload.
589 All test cases which uses VHOST.
593 All test cases which uses VHOST DPDK driver with qemu queue size set
598 All test cases which uses VIRTIO native VPP driver.
602 All test cases which uses VIRTIO native VPP driver with qemu queue
607 All test cases which uses VM with optimised scheduler policy.
611 All test cases which uses TUN and TAP.
615 All test cases which uses AFPKT.
619 All test cases which uses Netmap.
623 All test cases which uses Memif.
627 All test cases which uses only single Memif connection per DUT. One DUT
628 instance is running in container having one physical interface exposed
633 All test cases which uses link bonding (BondEthernet interface).
637 All test cases which uses DPDK link bonding.
641 All test cases which uses VPP link bonding.
645 All test cases which uses link bonding with mode XOR.
649 All test cases which uses link bonding with mode LACP.
653 All test cases which uses link bonding with load-balance mode l34.
657 All test cases which use {n} link(s) for link bonding.
661 All test cases which NIC Driver for DUT is set to {d}.
663 {d}=(AVF, RDMA_CORE, VFIO_PCI, AF_XDP).
667 All test cases which NIC Driver for TG is set to {d}.
669 {d}=(RDMA_CORE, IGB_UIO).
673 All test cases which RXQ size (RX descriptors) are set to {n}.
674 Default is 0, which means VPP (API) default.
678 All test cases which TXQ size (TX descriptors) are set to {n}.
679 Default is 0, which means VPP (API) default.
693 NAT44 configured and tested.
697 NAT44 configured and tested.
701 ACL plugin configured and tested.
705 ACL plugin configured and tested on input path.
709 ACL plugin configured and tested on output path.
713 ACL plugin configured and tested in stateless mode
718 ACL plugin configured and tested in stateful mode
719 (permit+reflect action).
723 ACL plugin configured and tested with 1 not-hitting ACE.
727 ACL plugin configured and tested with 10 not-hitting ACEs.
731 ACL plugin configured and tested with 50 not-hitting ACEs.
735 SRv6 endpoint to SR-unaware appliance via proxy.
739 SRv6 endpoint to SR-unaware appliance via static proxy.
743 SRv6 endpoint to SR-unaware appliance via dynamic proxy.
747 SRv6 endpoint to SR-unaware appliance via masquerading proxy.
761 IPSec in transport mode.
765 IPSec in tunnel mode.
769 IPSec in interface mode.
773 IPSec using AES algorithms.
777 IPSec using AES 128 CBC algorithms.
781 IPSec using AES 128 GCM algorithms.
785 IPSec using AES 256 GCM algorithms.
789 IPSec using HMAC integrity algorithms.
793 IPSec using HMAC SHA 256 integrity algorithms.
797 IPSec using HMAC SHA 512 integrity algorithms.
801 IPSec using crypto sw scheduler engine.
805 IPSec policy mode with spd fast path enabled.
807 ## Client-Workload Tags
811 All test cases which use at least one virtual machine.
815 All test cases which use Linux container and LXC utils.
819 All test cases which use at least one Docker container.
823 All test cases which use Docker as container manager.
827 All test cases with specific APP use.
829 ## Container Orchestration Tags
833 {n} VPP running in {n} Docker container(s) acting as a VSWITCH.
838 {n} VPP running in {n} Docker container(s) acting as a VNF work load.
841 ## Multi-Threading Tags
846 All test cases using single poll mode thread.
851 All test cases using more then one poll mode driver thread.
855 All test cases with packet processing on {n} socket(s). {n}=(1,2).
859 {c} worker thread pinned to {c} dedicated physical core; or if
860 HyperThreading is enabled, {c}*2 worker threads each pinned to
861 a separate logical core within 1 dedicated physical core. Main
862 thread pinned to core 1.
868 {t} worker threads pinned to {c} dedicated physical cores. Main thread
869 pinned to core 1. By default CSIT is configuring same amount of receive
870 queues per interface as worker threads.