X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Ftag_documentation.rst;h=c6064c61a30bff5443e319b29a93d40e18a54588;hp=4edcaac526244f02733b32297b7aa1e09656cb98;hb=32188ccdce50accbb80b405c5813543a4131e744;hpb=28bab4e715a123199972bdc5f79f6a508a879fd6 diff --git a/docs/tag_documentation.rst b/docs/tag_documentation.rst index 4edcaac526..c6064c61a3 100644 --- a/docs/tag_documentation.rst +++ b/docs/tag_documentation.rst @@ -1,28 +1,10 @@ -.. |copy| unicode:: 0xA9 .. copyright sign - -| Copyright |copy| 2016 Cisco and/or its affiliates. -| Licensed under the Apache License, Version 2.0 (the "License"); -| you may not use this file except in compliance with the License. -| You may obtain a copy of the License at: -| -| http://www.apache.org/licenses/LICENSE-2.0 -| -| Unless required by applicable law or agreed to in writing, software -| distributed under the License is distributed on an "AS IS" BASIS, -| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -| See the License for the specific language governing permissions and -| limitations under the License. - - -.. contents:: Table of Contents - :depth: 2 - - -TAGs and their descriptions -=========================== - -Documentation for tags used to select and identify test cases. +CSIT TAGs Descriptions +====================== +All CSIT test cases are labelled with Robot Framework tags used to allow for +easy test case type identification, test case grouping and selection for +execution. Following sections list currently used CSIT TAGs and their +documentation. Topology TAGs ------------- @@ -45,6 +27,9 @@ Objective TAGs Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch) and csit-vpp-verify jobs (i.e. CSIT patch). +.. topic:: SKIP_VPP_PATCH + + Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch). Environment TAGs ---------------- @@ -61,7 +46,6 @@ Environment TAGs DUTs with VPP and capable of running Virtual Machine. - NIC model tags -------------- @@ -85,7 +69,6 @@ NIC model tags VIC-1385 by Cisco. - Scaling TAGs ------------ @@ -101,9 +84,47 @@ Scaling TAGs 2x1,000,000 entries in single fib table +.. topic:: TNL_1000 + + IPSec in tunnel mode - 1000 tunnels. + +.. topic:: SRC_USER_10 + + Traffic flow with 10 unique IPs (users) in one direction. + +.. topic:: SRC_USER_100 + + Traffic flow with 100 unique IPs (users) in one direction. + +.. topic:: SRC_USER_1000 + + Traffic flow with 1000 unique IPs (users) in one direction. -Tags marking the kinds of tags ------------------------------- +.. topic:: SRC_USER_2000 + + Traffic flow with 2000 unique IPs (users) in one direction. + +.. topic:: SRC_USER_4000 + + Traffic flow with 4000 unique IPs (users) in one direction. + +.. topic:: 100_FLOWS + + Traffic stream with 100 unique flows (10 IPs/users x 10 UDP ports) in one + direction. + +.. topic:: 10k_FLOWS + + Traffic stream with 10 000 unique flows (10 IPs/users x 1000 UDP ports) in + one direction. + +.. topic:: 100k_FLOWS + + Traffic stream with 100 000 unique flows (100 IPs/users x 1000 UDP ports) in + one direction. + +Tags marking functional vs. performance of tests +------------------------------------------------ .. topic:: FUNCTEST @@ -113,7 +134,6 @@ Tags marking the kinds of tags All performance test cases. - Performance testing tags ------------------------ @@ -134,9 +154,43 @@ Performance testing tags .. topic:: NDRPDRDISC - Find performance of DUT based on RFC2544 with linear / binary / combined + Find performance of DUT based on :rfc:`2544` with linear / binary / combined search. (Previous LONG tests.) +Ethernet frame size tags for performance tests +---------------------------------------------- + +.. topic:: 64B + + 64B frames used for test. + +.. topic:: 78B + + 78B frames used for test. + +.. topic:: IMIX + + IMIX frame sequence (28x 64B, 16x 570B, 4x 1518B) used for test. + +.. topic:: 1460B + + 1460B frames used for test. + +.. topic:: 1480B + + 1480B frames used for test. + +.. topic:: 1514B + + 1514B frames used for test. + +.. topic:: 1518B + + 1518B frames used for test. + +.. topic:: 9000B + + 9000B frames used for test. Test type tags -------------- @@ -145,6 +199,26 @@ Test type tags Baseline test cases, no encapsulation, no feature(s) configured in tests. +.. topic:: IP4BASE + + IPv4 baseline test cases, no encapsulation, no feature(s) configured in + tests. + +.. topic:: IP6BASE + + IPv6 baseline test cases, no encapsulation, no feature(s) configured in + tests. + +.. topic:: L2XCBASE + + L2XC baseline test cases, no encapsulation, no feature(s) configured in + tests. + +.. topic:: L2BDBASE + + L2BD baseline test cases, no encapsulation, no feature(s) configured in + tests. + .. topic:: SCALE Scale test cases. @@ -157,6 +231,9 @@ Test type tags At least one feature is configured in test cases. Use also feature tag(s). +.. topic:: TLDK + + Functional test cases for TLDK. Forwarding mode tags -------------------- @@ -181,9 +258,8 @@ Forwarding mode tags VPP IPv6 routed forwarding. - Underlay tags ----------- +------------- .. topic:: IP4UNRLAY @@ -197,9 +273,8 @@ Underlay tags MPLS underlay. - Overlay tags ----------- +------------ .. topic:: L2OVRLAY @@ -213,13 +288,8 @@ Overlay tags IPv6 overlay (IPv6 payload). - -Encapsulation tags ------------------- - -.. topic:: ETH - - All test cases with base Ethernet (no encapsulation). +Tagging tags +------------ .. topic:: DOT1Q @@ -229,6 +299,13 @@ Encapsulation tags All test cases with dot1ad. +Encapsulation tags +------------------ + +.. topic:: ETH + + All test cases with base Ethernet (no encapsulation). + .. topic:: LISP All test cases with LISP. @@ -253,7 +330,6 @@ Encapsulation tags All test cases with IPSEC. - Interface tags -------------- @@ -265,6 +341,18 @@ Interface tags All test cases which uses VHOST. +.. topic:: VHOST_256 + + All test cases which uses VHOST with qemu queue size set to 256. + +.. topic:: VHOST_1024 + + All test cases which uses VHOST with qemu queue size set to 1024. + +.. topic:: CFS_OPT + + All test cases which uses VM with optimised scheduler policy. + .. topic:: TUNTAP All test cases which uses TUN and TAP. @@ -277,6 +365,9 @@ Interface tags All test cases which uses Netmap. +.. topic:: MEMIF + + All test cases which uses Memif. Feature tags ------------ @@ -289,10 +380,45 @@ Feature tags COP whitelist. -.. topic:: SNAT +.. topic:: NAT44 + + NAT44 configured and tested. + +.. topic:: NAT64 + + NAT44 configured and tested. + +.. topic:: ACL + + ACL plugin configured and tested. + +.. topic:: IACL + + ACL plugin configured and tested on input path. + +.. topic:: OACL + + ACL plugin configured and tested on output path. + +.. topic:: ACL_STATELESS + + ACL plugin configured and tested in stateless mode (permit action). + +.. topic:: ACL_STATEFUL - SNAT configured and tested. + ACL plugin configured and tested in stateful mode (permit+reflect action). +.. topic:: ACL1 + + ACL plugin configured and tested with 1 not-hitting ACE. + +.. topic:: ACL10 + + ACL plugin configured and tested with 10 not-hitting ACEs. + +.. topic:: ACL50 + + ACL plugin configured and tested with 50 not-hitting ACEs. Encryption tags --------------- @@ -313,7 +439,6 @@ Encryption tags IPSec in tunnel mode. - Client-workload tags -------------------- @@ -323,12 +448,44 @@ Client-workload tags .. topic:: LXC - All test cases which use Linux container. + All test cases which use Linux container and LXC utils. + +.. topic:: DOCKER + + All test cases which use Docker as container manager. .. topic:: APP All test cases with specific APP use. +Container orchestration tags +---------------------------- + +.. topic:: K8S + + All test cases which use Kubernetes for orchestration. + +.. topic:: SFC_CONTROLLER + + All test cases which use ligato/sfc_controller for driving configuration + of vpp inside container. + +.. topic:: VPP_AGENT + + All test cases which use Golang implementation of a control/management plane + for VPP + +.. topic:: 1VSWITCH + + VPP running in Docker container acting as VSWITCH. + +.. topic:: 1VNF + + 1 VPP running in Docker container acting as VNF work load. + +.. topic:: 2VNF + + 2 VPP running in 2 Docker containers acting as VNF work load. Multi-threading tags -------------------- @@ -381,3 +538,23 @@ Multi-threading tags 8 worker threads pinned to 8 dedicated physical cores. 4 receive queues per interface. Main thread pinned to core 0. + +Honeycomb tags +-------------- + +.. topic:: HC_FUNC + + Honeycomb functional test cases. + +.. topic:: HC_NSH + + Honeycomb NSH test cases. + +.. topic:: HC_PERSIST + + Honeycomb persistence test cases. + +.. topic:: HC_REST_ONLY + + (Exclusion tag) Honeycomb test cases that cannot be run in Netconf mode + using ODL client for Restfconf -> Netconf translation.