X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Ftag_documentation.rst;h=33d38cfff9f4102aff7322bc8176632a4e753202;hp=e0be374ee281041dc427d52f0e28705f89bc9be0;hb=d23c99f9c74dc75cc3176ce5d7885562d13aa2f9;hpb=e3ed9257cd7ef50b44eca72ef7d9ae6c7c792647 diff --git a/docs/tag_documentation.rst b/docs/tag_documentation.rst index e0be374ee2..33d38cfff9 100644 --- a/docs/tag_documentation.rst +++ b/docs/tag_documentation.rst @@ -1,31 +1,23 @@ -.. |copy| unicode:: 0xA9 .. copyright sign +CSIT TAGs Descriptions +====================== -| Copyright |copy| 2016 Cisco and/or its affiliates. -| Licensed under the Apache License, Version 2.0 (the "License"); -| you may not use this file except in compliance with the License. -| You may obtain a copy of the License at: -| -| http://www.apache.org/licenses/LICENSE-2.0 -| -| Unless required by applicable law or agreed to in writing, software -| distributed under the License is distributed on an "AS IS" BASIS, -| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -| See the License for the specific language governing permissions and -| limitations under the License. - - -.. contents:: Table of Contents - :depth: 2 +All CSIT test cases are labelled with Robot Framework tags used to allow for +easy test case type identification, test case grouping and selection for +execution. Following sections list currently used CSIT TAGs and their +documentation. +Topology TAGs +------------- -TAGs and their descriptions -=========================== +.. topic:: 2_NODE_DOUBLE_LINK_TOPO -Documentation for tags used to select and identify test cases. + 2 nodes connected in a circular topology with two links interconnecting + the devices. +.. topic:: 2_NODE_SINGLE_LINK_TOPO -Topology TAGs -------------- + 2 nodes connected in a circular topology with at least one link + interconnecting devices. .. topic:: 3_NODE_DOUBLE_LINK_TOPO @@ -45,6 +37,9 @@ Objective TAGs Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch) and csit-vpp-verify jobs (i.e. CSIT patch). +.. topic:: SKIP_VPP_PATCH + + Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch). Environment TAGs ---------------- @@ -61,6 +56,32 @@ Environment TAGs DUTs with VPP and capable of running Virtual Machine. +NIC model tags +-------------- + +.. topic:: NIC_Intel-X520-DA2 + + Intel X520-DA2 NIC. + +.. topic:: NIC_Intel-XL710 + + Intel XL710 NIC. + +.. topic:: NIC_Intel-X710 + + Intel X710 NIC. + +.. topic:: NIC_Intel-XXV710 + + Intel XXV710 NIC. + +.. topic:: NIC_Cisco-VIC-1227 + + VIC-1227 by Cisco. + +.. topic:: NIC_Cisco-VIC-1385 + + VIC-1385 by Cisco. Scaling TAGs ------------ @@ -77,9 +98,95 @@ Scaling TAGs 2x1,000,000 entries in single fib table +.. topic:: L2BD_1 + + Test with 1 L2 bridge domain. + +.. topic:: L2BD_10 + + Test with 10 L2 bridge domains. + +.. topic:: L2BD_100 + + Test with 100 L2 bridge domains. + +.. topic:: L2BD_1K + + Test with 1000 L2 bridge domains. + +.. topic:: VLAN_1 + + Test with 1 VLAN sub-interface. + +.. topic:: VLAN_10 + + Test with 10 VLAN sub-interfaces. + +.. topic:: VLAN_100 + + Test with 100 VLAN sub-interfaces. + +.. topic:: VLAN_1K + + Test with 1000 VLAN sub-interfaces. + +.. topic:: VXLAN_1 + + Test with 1 VXLAN tunnel. + +.. topic:: VXLAN_10 + + Test with 10 VXLAN tunnels. + +.. topic:: VXLAN_100 + + Test with 100 VXLAN tunnels. + +.. topic:: VXLAN_1K + + Test with 1000 VXLAN tunnels. + +.. topic:: TNL_1000 + + IPSec in tunnel mode - 1000 tunnels. + +.. topic:: SRC_USER_10 + + Traffic flow with 10 unique IPs (users) in one direction. + +.. topic:: SRC_USER_100 + + Traffic flow with 100 unique IPs (users) in one direction. + +.. topic:: SRC_USER_1000 + + Traffic flow with 1000 unique IPs (users) in one direction. + +.. topic:: SRC_USER_2000 -Tags marking the kinds of tags ------------------------------- + Traffic flow with 2000 unique IPs (users) in one direction. + +.. topic:: SRC_USER_4000 + + Traffic flow with 4000 unique IPs (users) in one direction. + +.. topic:: 100_FLOWS + + Traffic stream with 100 unique flows (10 IPs/users x 10 UDP ports) in one + direction. + +.. topic:: 10k_FLOWS + + Traffic stream with 10 000 unique flows (10 IPs/users x 1000 UDP ports) in + one direction. + +.. topic:: 100k_FLOWS + + Traffic stream with 100 000 unique flows (100 IPs/users x 1000 UDP ports) in + one direction. + +Tags marking functional vs. performance of tests +------------------------------------------------ .. topic:: FUNCTEST @@ -89,30 +196,58 @@ Tags marking the kinds of tags All performance test cases. - Performance testing tags ------------------------ -.. topic:: PDRDISC +.. topic:: NDRPDR + + Single test finding both No Drop Rate and Partial Drop Rate simultaneously. + The search is done by optimized algorithm which performs + multiple trial runs at different durations and transmit rates. + +.. topic:: MRR + + Performance tests where TG sends the traffic at maximum rate (line rate) + and reports total sent/received packets over performance trial duration. + +Ethernet frame size tags for performance tests +---------------------------------------------- - Partial Drop Rate evaluation of single run result, with non-zero packet - loss tolerance (LT) expressed in percentage of packets transmitted. +.. topic:: 64B -.. topic:: NDRDISC + 64B frames used for test. - Non Drop Rate evaluation of results. Loss acceptance of dropped packets is - set to zero lost packets. +.. topic:: 78B -.. topic:: NDRCHK + 78B frames used for test. - Performance tests where TG verifies DUTs' throughput at ref-NDR (reference - Non Drop Rate) with zero packet loss tolerance. +.. topic:: 114B -.. topic:: NDRPDRDISC + 114B frames used for test. - Find performance of DUT based on RFC2544 with linear / binary / combined - search. (Previous LONG tests.) +.. topic:: IMIX + IMIX frame sequence (28x 64B, 16x 570B, 4x 1518B) used for test. + +.. topic:: 1460B + + 1460B frames used for test. + +.. topic:: 1480B + + 1480B frames used for test. + +.. topic:: 1514B + + 1514B frames used for test. + +.. topic:: 1518B + + 1518B frames used for test. + +.. topic:: 9000B + + 9000B frames used for test. Test type tags -------------- @@ -121,6 +256,31 @@ Test type tags Baseline test cases, no encapsulation, no feature(s) configured in tests. +.. topic:: IP4BASE + + IPv4 baseline test cases, no encapsulation, no feature(s) configured in + tests. + +.. topic:: IP6BASE + + IPv6 baseline test cases, no encapsulation, no feature(s) configured in + tests. + +.. topic:: L2XCBASE + + L2XC baseline test cases, no encapsulation, no feature(s) configured in + tests. + +.. topic:: L2BDBASE + + L2BD baseline test cases, no encapsulation, no feature(s) configured in + tests. + +.. topic:: L2PATCH + + L2PATCH baseline test cases, no encapsulation, no feature(s) configured in + tests. + .. topic:: SCALE Scale test cases. @@ -133,6 +293,29 @@ Test type tags At least one feature is configured in test cases. Use also feature tag(s). +.. topic:: TLDK + + Functional test cases for TLDK. + +.. topic:: DMM + + Functional test cases for DMM. + +.. topic:: TCP + + Tests which use TCP. + +.. topic:: TCP_CPS + + Performance tests which measure connections per second using http requests. + +.. topic:: TCP_RPS + + Performance tests which measure requests per second using http requests. + +.. topic:: HTTP + + Tests which use HTTP. Forwarding mode tags -------------------- @@ -157,9 +340,8 @@ Forwarding mode tags VPP IPv6 routed forwarding. - Underlay tags ----------- +------------- .. topic:: IP4UNRLAY @@ -173,9 +355,8 @@ Underlay tags MPLS underlay. - Overlay tags ----------- +------------ .. topic:: L2OVRLAY @@ -189,13 +370,8 @@ Overlay tags IPv6 overlay (IPv6 payload). - -Encapsulation tags ------------------- - -.. topic:: ETH - - All test cases with base Ethernet (no encapsulation). +Tagging tags +------------ .. topic:: DOT1Q @@ -205,6 +381,13 @@ Encapsulation tags All test cases with dot1ad. +Encapsulation tags +------------------ + +.. topic:: ETH + + All test cases with base Ethernet (no encapsulation). + .. topic:: LISP All test cases with LISP. @@ -225,6 +408,13 @@ Encapsulation tags All test cases with GRE. +.. topic:: IPSEC + + All test cases with IPSEC. + +.. topic:: SRv6 + + All test cases with Segment routing over IPv6 dataplane. Interface tags -------------- @@ -237,6 +427,18 @@ Interface tags All test cases which uses VHOST. +.. topic:: VHOST_256 + + All test cases which uses VHOST with qemu queue size set to 256. + +.. topic:: VHOST_1024 + + All test cases which uses VHOST with qemu queue size set to 1024. + +.. topic:: CFS_OPT + + All test cases which uses VM with optimised scheduler policy. + .. topic:: TUNTAP All test cases which uses TUN and TAP. @@ -249,6 +451,55 @@ Interface tags All test cases which uses Netmap. +.. topic:: MEMIF + + All test cases which uses Memif. + +.. topic:: SINGLE_MEMIF + + All test cases which uses only single Memif connection per DUT. One DUT + instance is running in container having one physical interface exposed to + container. + +.. topic:: LBOND + + All test cases which uses link bonding (BondEthernet interface). + +.. topic:: LBOND_DPDK + + All test cases which uses DPDK link bonding. + +.. topic:: LBOND_VPP + + All test cases which uses VPP link bonding. + +.. topic:: LBOND_MODE_XOR + + All test cases which uses link bonding with mode XOR. + +.. topic:: LBOND_MODE_LACP + + All test cases which uses link bonding with mode LACP. + +.. topic:: LBOND_LB_L34 + + All test cases which uses link bonding with load-balance mode l34. + +.. topic:: LBOND_1L + + All test cases which uses one link for link bonding. + +.. topic:: LBOND_2L + + All test cases which uses two links for link bonding. + +.. topic:: DRV_AVF + + All test cases which uses Intel Adaptive Virtual Function (AVF) device + plugin for VPP. This plugins provides native device support for Intel AVF. + AVF is driver specification for current and future Intel Virtual Function + devices. In essence, today this driver can be used only with Intel + XL710 / X710 / XXV710 adapters. Feature tags ------------ @@ -261,6 +512,80 @@ Feature tags COP whitelist. +.. topic:: NAT44 + + NAT44 configured and tested. + +.. topic:: NAT64 + + NAT44 configured and tested. + +.. topic:: ACL + + ACL plugin configured and tested. + +.. topic:: IACL + + ACL plugin configured and tested on input path. + +.. topic:: OACL + + ACL plugin configured and tested on output path. + +.. topic:: ACL_STATELESS + + ACL plugin configured and tested in stateless mode (permit action). + +.. topic:: ACL_STATEFUL + + ACL plugin configured and tested in stateful mode (permit+reflect action). + +.. topic:: ACL1 + + ACL plugin configured and tested with 1 not-hitting ACE. + +.. topic:: ACL10 + + ACL plugin configured and tested with 10 not-hitting ACEs. + +.. topic:: ACL50 + + ACL plugin configured and tested with 50 not-hitting ACEs. + +.. topic:: SRv6_PROXY + + SRv6 endpoint to SR-unaware appliance via proxy. + +.. topic:: SRv6_PROXY_STAT + + SRv6 endpoint to SR-unaware appliance via static proxy. + +.. topic:: SRv6_PROXY_DYN + + SRv6 endpoint to SR-unaware appliance via dynamic proxy. + +.. topic:: SRv6_PROXY_MASQ + + SRv6 endpoint to SR-unaware appliance via masquerading proxy. + +Encryption tags +--------------- + +.. topic:: IPSECSW + + Crypto in software. + +.. topic:: IPSECHW + + Crypto in hardware. + +.. topic:: IPSECTRAN + + IPSec in transport mode. + +.. topic:: IPSECTUN + + IPSec in tunnel mode. Client-workload tags -------------------- @@ -271,22 +596,60 @@ Client-workload tags .. topic:: LXC - All test cases which use Linux container. + All test cases which use Linux container and LXC utils. + +.. topic:: DOCKER + + All test cases which use Docker as container manager. .. topic:: APP All test cases with specific APP use. +Container orchestration tags +---------------------------- + +.. topic:: K8S + + All test cases which use Kubernetes for orchestration. + +.. topic:: SFC_CONTROLLER + + All test cases which use ligato/sfc_controller for driving configuration + of vpp inside container. + +.. topic:: VPP_AGENT + + All test cases which use Golang implementation of a control/management plane + for VPP + +.. topic:: 1VSWITCH + + VPP running in Docker container acting as VSWITCH. + +.. topic:: 1VNF + + 1 VPP running in Docker container acting as VNF work load. + +.. topic:: 2VNF + + 2 VPP running in 2 Docker containers acting as VNF work load. + +.. topic:: 4VNF + + 4 VPP running in 4 Docker containers acting as VNF work load. Multi-threading tags -------------------- .. topic:: STHREAD - All test cases using single poll mode thread. + *Dynamic tag*. + All test cases using single poll mode thread. .. topic:: MTHREAD + *Dynamic tag*. All test cases using more then one poll mode driver thread. .. topic:: 1NUMA @@ -297,35 +660,78 @@ Multi-threading tags All test cases with packet processing on two sockets. -.. topic:: SMT +.. topic:: 1C + + 1 worker thread pinned to 1 dedicated physical core; or if HyperThreading is + enabled, 2 worker threads each pinned to a separate logical core within 1 + dedicated physical core. Main thread pinned to core 1. - All test cases with symmetric Multi-Threading (HyperThreading) enabled. +.. topic:: 2C -.. topic:: NOSMT + 2 worker threads pinned to 2 dedicated physical cores; or if HyperThreading + is enabled, 4 worker threads each pinned to a separate logical core within 2 + dedicated physical cores. Main thread pinned to core 1. - All test cases with symmetric Multi-Threading (HyperThreading) disabled. +.. topic:: 4C + + 4 worker threads pinned to 4 dedicated physical cores; or if HyperThreading + is enabled, 8 worker threads each pinned to a separate logical core within 4 + dedicated physical cores. Main thread pinned to core 1. .. topic:: 1T1C + *Dynamic tag*. 1 worker thread pinned to 1 dedicated physical core. 1 receive queue per - interface. Main thread pinned to core 0. + interface. Main thread pinned to core 1. .. topic:: 2T2C + *Dynamic tag*. 2 worker threads pinned to 2 dedicated physical cores. 1 receive queue per - interface. Main thread pinned to core 0. + interface. Main thread pinned to core 1. .. topic:: 4T4C + *Dynamic tag*. 4 worker threads pinned to 4 dedicated physical cores. 2 receive queues per - interface. Main thread pinned to core 0. + interface. Main thread pinned to core 1. + +.. topic:: 2T1C + + *Dynamic tag*. + 2 worker threads each pinned to a separate logical core within 1 dedicated + physical core. 1 receive queue per interface. Main thread pinned to core 1. + +.. topic:: 4T2C + + *Dynamic tag*. + 4 worker threads each pinned to a separate logical core within 2 dedicated + physical cores. 2 receive queues per interface. Main thread pinned to core + 1. + +.. topic:: 8T4C + + *Dynamic tag*. + 8 worker threads each pinned to a separate logical core within 4 dedicated + physical cores. 4 receive queues per interface. Main thread pinned to core + 1. + +Honeycomb tags +-------------- + +.. topic:: HC_FUNC + + Honeycomb functional test cases. + +.. topic:: HC_NSH + + Honeycomb NSH test cases. -.. topic:: 6T6C +.. topic:: HC_PERSIST - 6 worker threads pinned to 6 dedicated physical cores. 3 receive queues per - interface. Main thread pinned to core 0. + Honeycomb persistence test cases. -.. topic:: 8T8C +.. topic:: HC_REST_ONLY - 8 worker threads pinned to 8 dedicated physical cores. 4 receive queues per - interface. Main thread pinned to core 0. + (Exclusion tag) Honeycomb test cases that cannot be run in Netconf mode + using ODL client for Restfconf -> Netconf translation.