1 CSIT RF Tags Descriptions
2 =========================
4 All CSIT test cases are labelled with Robot Framework tags used to allow for
5 easy test case type identification, test case grouping and selection for
6 execution. Following sections list currently used CSIT TAGs and their
7 documentation based on the content of
8 `tag documentation rst file`_.
13 .. topic:: 2_NODE_DOUBLE_LINK_TOPO
15 2 nodes connected in a circular topology with two links interconnecting
18 .. topic:: 2_NODE_SINGLE_LINK_TOPO
20 2 nodes connected in a circular topology with at least one link
21 interconnecting devices.
23 .. topic:: 3_NODE_DOUBLE_LINK_TOPO
25 3 nodes connected in a circular topology with two links interconnecting
28 .. topic:: 3_NODE_SINGLE_LINK_TOPO
30 3 nodes connected in a circular topology with at least one link
31 interconnecting devices.
38 Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch)
39 and csit-vpp-verify jobs (i.e. CSIT patch).
41 .. topic:: SKIP_VPP_PATCH
43 Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch).
50 DUTs and TGs are running on bare metal.
54 DUTs and TGs are running in virtual environment.
58 DUTs with VPP and capable of running Virtual Machine.
63 .. topic:: NIC_Intel-X520-DA2
67 .. topic:: NIC_Intel-XL710
71 .. topic:: NIC_Intel-X710
75 .. topic:: NIC_Intel-XXV710
79 .. topic:: NIC_Cisco-VIC-1227
83 .. topic:: NIC_Cisco-VIC-1385
87 .. topic:: NIC_Amazon-Nitro-50G
96 2x10,000 entries in single fib table
100 2x100,000 entries in single fib table
104 2x500,000 entries in single fib table
108 2x1,000,000 entries in single fib table
112 Test with 1 L2 bridge domain.
116 Test with 10 L2 bridge domains.
120 Test with 100 L2 bridge domains.
124 Test with 1000 L2 bridge domains.
128 Test with 1 VLAN sub-interface.
132 Test with 10 VLAN sub-interfaces.
136 Test with 100 VLAN sub-interfaces.
140 Test with 1000 VLAN sub-interfaces.
144 Test with 1 VXLAN tunnel.
148 Test with 10 VXLAN tunnels.
152 Test with 100 VXLAN tunnels.
156 Test with 1000 VXLAN tunnels.
160 IPSec in tunnel mode - {t} tunnels.
162 .. topic:: SRC_USER_{u}
164 Traffic flow with {u} unique IPs (users) in one direction.
165 {u}=(1,10,100,1000,2000,4000).
169 Traffic stream with 100 unique flows (10 IPs/users x 10 UDP ports) in one
174 Traffic stream with 10 000 unique flows (10 IPs/users x 1000 UDP ports) in
177 .. topic:: 100k_FLOWS
179 Traffic stream with 100 000 unique flows (100 IPs/users x 1000 UDP ports) in
184 Stateless or stateful traffic stream with {h} client source IP4 addresses,
185 usually with 63 flow differing in source port number. Could be UDP or TCP.
186 If NAT is used, the clients are inside. Outside IP range can differ.
187 {h}=(1024,4096,16384,65536,262144).
189 .. topic:: GENEVE4_{t}TUN
191 Test with {t} GENEVE IPv4 tunnel. {t}=(1,4,16,64,256,1024)
196 .. topic:: DEVICETEST
198 All vpp_device functional test cases.
202 All performance test cases.
209 All test cases that uses Scapy for packet generation and validation.
211 Performance Type Tags
212 ---------------------
216 Single test finding both No Drop Rate and Partial Drop Rate simultaneously.
217 The search is done by optimized algorithm which performs
218 multiple trial runs at different durations and transmit rates.
219 The results come from the final trials, which have duration of 30 seconds.
223 Performance tests where TG sends the traffic at maximum rate (line rate)
224 and reports total sent/received packets over trial duration.
225 The result is an average of 10 trials of 1 second duration.
229 Performance tests using PLRsearch to find the critical load.
233 Performance tests aimed to measure lost packets (time) when performing
234 reconfiguration while full throughput offered load is applied.
236 Ethernet Frame Size Tags
237 ------------------------
239 These are describing the traffic offered by Traffic Generator,
240 "primary" traffic in case of asymmetric load.
241 For traffic between DUTs, or for "secondary" traffic, see ${overhead} value.
245 {b} Bytes frames used for test.
249 IMIX frame sequence (28x 64B, 16x 570B, 4x 1518B) used for test.
256 Baseline test cases, no encapsulation, no feature(s) configured in tests.
257 No scaling whatsoever, beyond minimum needed for RSS.
261 IPv4 baseline test cases, no encapsulation, no feature(s) configured in
262 tests. Minimal number of routes. Other quantities may be scaled.
266 IPv6 baseline test cases, no encapsulation, no feature(s) configured in
271 L2XC baseline test cases, no encapsulation, no feature(s) configured in
276 L2BD baseline test cases, no encapsulation, no feature(s) configured in
281 L2PATCH baseline test cases, no encapsulation, no feature(s) configured in
286 Scale test cases. Other tags specify which quantities are scaled.
287 Also applies if scaling is set on TG only (e.g. DUT works as IP4BASE).
291 Test cases where encapsulation is used. Use also encapsulation tag(s).
295 At least one feature is configured in test cases. Use also feature tag(s).
299 Tests which use any kind of UDP traffic (STL or ASTF profile).
303 Tests which use any kind of TCP traffic (STL or ASTF profile).
307 Tests which test trex traffic without any software DUTs in the traffic path.
310 TODO: Should we define tags STL and ASTF?
314 Tests which use unidirectional UDP traffic (STL profile only).
318 Tests which use bidirectional UDP traffic (STL profile only).
322 Tests which measure connections per second on minimal UDP pseudoconnections.
323 This implies ASTF traffic profile is used.
324 This tag selects specific output processing in PAL.
328 Tests which measure connections per second on empty TCP connections.
329 This implies ASTF traffic profile is used.
330 This tag selects specific output processing in PAL.
334 Tests which measure requests per second on empty TCP connections.
335 This implies ASTF traffic profile is used.
336 This tag selects specific output processing in PAL.
340 Tests which measure packets per second on lightweight UDP transactions.
341 This implies ASTF traffic profile is used.
342 This tag selects specific output processing in PAL.
346 Tests which measure packets per second on lightweight TCP transactions.
347 This implies ASTF traffic profile is used.
348 This tag selects specific output processing in PAL.
352 Tests which use traffic formed of valid HTTP requests (and responses).
355 TODO: Add HTTP tag to the current hoststack tests.
356 TODO: Document other tags already used by hoststack tests.
360 LDP NGINX is un-modified NGINX with VPP via LD_PRELOAD.
362 .. topic:: NF_DENSITY
364 Performance tests that measure throughput of multiple VNF and CNF
365 service topologies at different service densities.
367 NF Service Density Tags
368 -----------------------
372 NF service density tests with VNF or CNF service chain topology(ies).
376 NF service density tests with CNF service pipeline topology(ies).
378 .. topic:: NF_L3FWDIP4
380 NF service density tests with DPDK l3fwd IPv4 routing as NF workload.
384 NF service density tests with VPP IPv4 routing as NF workload.
388 Service density matrix locator {r}R{c}C, {r}Row denoting number of
389 service instances, {c}Column denoting number of NFs per service
390 instance. {r}=(1,2,4,6,8,10), {c}=(1,2,4,6,8,10).
394 Service density {n}VM{t}T, {n}Number of NF Qemu VMs, {t}Number of threads
399 Service density {n}DCR{t}T, {n}Number of NF Docker containers, {t}Number of
402 .. topic:: {n}_ADDED_CHAINS
404 {n}Number of chains (or pipelines) added (and/or removed)
410 .. topic:: L2BDMACSTAT
412 VPP L2 bridge-domain, L2 MAC static.
414 .. topic:: L2BDMACLRN
416 VPP L2 bridge-domain, L2 MAC learning.
420 VPP L2 point-to-point cross-connect.
424 VPP IPv4 routed forwarding.
428 VPP IPv6 routed forwarding.
430 .. topic:: LOADBALANCER_MAGLEV
432 VPP Load balancer maglev mode.
434 .. topic:: LOADBALANCER_L3DSR
436 VPP Load balancer l3dsr mode.
438 .. topic:: LOADBALANCER_NAT4
440 VPP Load balancer nat4 mode.
444 Mode, where NICs from the same physical server are directly
445 connected with a cable.
458 .. topic:: MPLSUNRLAY
471 IPv4 overlay (IPv4 payload).
475 IPv6 overlay (IPv6 payload).
482 All test cases with dot1q.
486 All test cases with dot1ad.
493 All test cases with base Ethernet (no encapsulation).
497 All test cases with LISP.
501 All test cases with LISP-GPE.
503 .. topic:: LISP_IP4o4
505 All test cases with LISP_IP4o4.
507 .. topic:: LISPGPE_IP4o4
509 All test cases with LISPGPE_IP4o4.
511 .. topic:: LISPGPE_IP6o4
513 All test cases with LISPGPE_IP6o4.
515 .. topic:: LISPGPE_IP4o6
517 All test cases with LISPGPE_IP4o6.
519 .. topic:: LISPGPE_IP6o6
521 All test cases with LISPGPE_IP6o6.
525 All test cases with Vxlan.
529 All test cases with VXLAN-GPE.
533 All test cases with GRE.
537 All test cases with GTPU.
539 .. topic:: GTPU_HWACCEL
541 All test cases with GTPU_HWACCEL.
545 All test cases with IPSEC.
549 All test cases with WIREGUARD.
553 All test cases with Segment routing over IPv6 dataplane.
557 All SRv6 test cases with single SID.
559 .. topic:: SRv6_2SID_DECAP
561 All SRv6 test cases with two SIDs and with decapsulation.
563 .. topic:: SRv6_2SID_NODECAP
565 All SRv6 test cases with two SIDs and without decapsulation.
569 All test cases with GENEVE.
571 .. topic:: GENEVE_L3MODE
573 All test cases with GENEVE tunnel in L3 mode.
577 All test cases with FLOW.
581 All test cases with FLOW_DIR.
585 All test cases with FLOW_RSS.
589 All test cases with NTUPLE.
593 All test cases with L2TPV3.
600 All test cases which use physical interface(s).
604 All test cases which uses Generic Segmentation Offload.
608 All test cases which uses VHOST.
610 .. topic:: VHOST_1024
612 All test cases which uses VHOST DPDK driver with qemu queue size set
617 All test cases which uses VIRTIO native VPP driver.
619 .. topic:: VIRTIO_1024
621 All test cases which uses VIRTIO native VPP driver with qemu queue size set
626 All test cases which uses VM with optimised scheduler policy.
630 All test cases which uses TUN and TAP.
634 All test cases which uses AFPKT.
638 All test cases which uses Netmap.
642 All test cases which uses Memif.
644 .. topic:: SINGLE_MEMIF
646 All test cases which uses only single Memif connection per DUT. One DUT
647 instance is running in container having one physical interface exposed to
652 All test cases which uses link bonding (BondEthernet interface).
654 .. topic:: LBOND_DPDK
656 All test cases which uses DPDK link bonding.
660 All test cases which uses VPP link bonding.
662 .. topic:: LBOND_MODE_XOR
664 All test cases which uses link bonding with mode XOR.
666 .. topic:: LBOND_MODE_LACP
668 All test cases which uses link bonding with mode LACP.
670 .. topic:: LBOND_LB_L34
672 All test cases which uses link bonding with load-balance mode l34.
674 .. topic:: LBOND_{n}L
676 All test cases which use {n} link(s) for link bonding.
680 All test cases which NIC Driver for DUT is set to {d}. Default is VFIO_PCI.
681 {d}=(AVF, RDMA_CORE, VFIO_PCI, AF_XDP).
683 .. topic:: TG_DRV_{d}
685 All test cases which NIC Driver for TG is set to {d}. Default is IGB_UIO.
686 {d}=(RDMA_CORE, IGB_UIO).
688 .. topic:: RXQ_SIZE_{n}
690 All test cases which RXQ size (RX descriptors) are set to {n}. Default is 0,
691 which means VPP (API) default.
693 .. topic:: TXQ_SIZE_{n}
695 All test cases which TXQ size (TX descriptors) are set to {n}. Default is 0,
696 which means VPP (API) default.
705 .. topic:: ADLALWLIST
711 NAT44 configured and tested.
715 NAT44 configured and tested.
719 ACL plugin configured and tested.
723 ACL plugin configured and tested on input path.
727 ACL plugin configured and tested on output path.
729 .. topic:: ACL_STATELESS
731 ACL plugin configured and tested in stateless mode (permit action).
733 .. topic:: ACL_STATEFUL
735 ACL plugin configured and tested in stateful mode (permit+reflect action).
739 ACL plugin configured and tested with 1 not-hitting ACE.
743 ACL plugin configured and tested with 10 not-hitting ACEs.
747 ACL plugin configured and tested with 50 not-hitting ACEs.
749 .. topic:: SRv6_PROXY
751 SRv6 endpoint to SR-unaware appliance via proxy.
753 .. topic:: SRv6_PROXY_STAT
755 SRv6 endpoint to SR-unaware appliance via static proxy.
757 .. topic:: SRv6_PROXY_DYN
759 SRv6 endpoint to SR-unaware appliance via dynamic proxy.
761 .. topic:: SRv6_PROXY_MASQ
763 SRv6 endpoint to SR-unaware appliance via masquerading proxy.
778 IPSec in transport mode.
782 IPSec in tunnel mode.
786 IPSec in interface mode.
790 IPSec using AES algorithms.
792 .. topic:: AES_128_CBC
794 IPSec using AES 128 CBC algorithms.
796 .. topic:: AES_128_GCM
798 IPSec using AES 128 GCM algorithms.
800 .. topic:: AES_256_GCM
802 IPSec using AES 256 GCM algorithms.
806 IPSec using HMAC integrity algorithms.
808 .. topic:: HMAC_SHA_256
810 IPSec using HMAC SHA 256 integrity algorithms.
812 .. topic:: HMAC_SHA_512
814 IPSec using HMAC SHA 512 integrity algorithms.
818 IPSec using crypto sw scheduler engine.
822 IPSec policy mode with spd fast path enabled.
829 All test cases which use at least one virtual machine.
833 All test cases which use Linux container and LXC utils.
837 All test cases which use at least one Docker container.
841 All test cases which use Docker as container manager.
845 All test cases with specific APP use.
847 Container Orchestration Tags
848 ----------------------------
850 .. topic:: {n}VSWITCH
852 {n} VPP running in {n} Docker container(s) acting as a VSWITCH.
857 {n} VPP running in {n} Docker container(s) acting as a VNF work load.
866 All test cases using single poll mode thread.
871 All test cases using more then one poll mode driver thread.
875 All test cases with packet processing on {n} socket(s). {n}=(1,2).
879 {c} worker thread pinned to {c} dedicated physical core; or if
880 HyperThreading is enabled, {c}*2 worker threads each pinned to a separate
881 logical core within 1 dedicated physical core. Main thread pinned to core 1.
887 {t} worker threads pinned to {c} dedicated physical cores. Main thread
888 pinned to core 1. By default CSIT is configuring same amount of receive
889 queues per interface as worker threads. {t}=(1,2,4,8), {t}=(1,2,4).