-.. |copy| unicode:: 0xA9 .. copyright sign
-
-| Copyright |copy| 2016 Cisco and/or its affiliates.
-| Licensed under the Apache License, Version 2.0 (the "License");
-| you may not use this file except in compliance with the License.
-| You may obtain a copy of the License at:
-|
-| http://www.apache.org/licenses/LICENSE-2.0
-|
-| Unless required by applicable law or agreed to in writing, software
-| distributed under the License is distributed on an "AS IS" BASIS,
-| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-| See the License for the specific language governing permissions and
-| limitations under the License.
-
-
-.. contents:: Table of Contents
- :depth: 2
-
-
-TAGs and their descriptions
-===========================
-
-Documentation for tags used to select and identify test cases.
+CSIT TAGs Descriptions
+======================
+All CSIT test cases are labelled with Robot Framework tags used to allow for
+easy test case type identification, test case grouping and selection for
+execution. Following sections list currently used CSIT TAGs and their
+documentation.
Topology TAGs
-------------
Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch)
and csit-vpp-verify jobs (i.e. CSIT patch).
+.. topic:: SKIP_VPP_PATCH
+
+ Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch).
Environment TAGs
----------------
DUTs with VPP and capable of running Virtual Machine.
+NIC model tags
+--------------
+
+.. topic:: NIC_Intel-X520-DA2
+
+ Intel X520-DA2 NIC.
+
+.. topic:: NIC_Intel-XL710
+
+ Intel XL710 NIC.
+
+.. topic:: NIC_Intel-X710
+
+ Intel X710 NIC.
+
+.. topic:: NIC_Cisco-VIC-1227
+
+ VIC-1227 by Cisco.
+
+.. topic:: NIC_Cisco-VIC-1385
+
+ VIC-1385 by Cisco.
Scaling TAGs
------------
2x1,000,000 entries in single fib table
+.. topic:: TNL_1000
+
+ IPSec in tunnel mode - 1000 tunnels.
+
+.. topic:: SRC_USER_10
+
+ Traffic flow with 10 unique IPs (users) in one direction.
+
+.. topic:: SRC_USER_100
+
+ Traffic flow with 100 unique IPs (users) in one direction.
+
+.. topic:: SRC_USER_1000
+
+ Traffic flow with 1000 unique IPs (users) in one direction.
+
+.. topic:: SRC_USER_2000
+
+ Traffic flow with 2000 unique IPs (users) in one direction.
+
+.. topic:: SRC_USER_4000
+
+ Traffic flow with 4000 unique IPs (users) in one direction.
+
+.. topic:: 100_FLOWS
+
+ Traffic stream with 100 unique flows (10 IPs/users x 10 UDP ports) in one
+ direction.
-Tags marking the kinds of tags
-------------------------------
+.. topic:: 10k_FLOWS
+
+ Traffic stream with 10 000 unique flows (10 IPs/users x 1000 UDP ports) in
+ one direction.
+
+.. topic:: 100k_FLOWS
+
+ Traffic stream with 100 000 unique flows (100 IPs/users x 1000 UDP ports) in
+ one direction.
+
+Tags marking functional vs. performance of tests
+------------------------------------------------
.. topic:: FUNCTEST
All performance test cases.
-
Performance testing tags
------------------------
Find performance of DUT based on RFC2544 with linear / binary / combined
search. (Previous LONG tests.)
+Ethernet frame size tags for performance tests
+----------------------------------------------
+
+.. topic:: 64B
+
+ 64B frames used for test.
+
+.. topic:: 78B
+
+ 78B frames used for test.
+
+.. topic:: IMIX
+
+ IMIX frame sequence (28x 64B, 16x 570B, 4x 1518B) used for test.
+
+.. topic:: 1460B
+
+ 1460B frames used for test.
+
+.. topic:: 1480B
+
+ 1480B frames used for test.
+
+.. topic:: 1514B
+
+ 1514B frames used for test.
+
+.. topic:: 1518B
+
+ 1518B frames used for test.
+
+.. topic:: 9000B
+
+ 9000B frames used for test.
Test type tags
--------------
Baseline test cases, no encapsulation, no feature(s) configured in tests.
+.. topic:: IP4BASE
+
+ IPv4 baseline test cases, no encapsulation, no feature(s) configured in
+ tests.
+
+.. topic:: IP6BASE
+
+ IPv6 baseline test cases, no encapsulation, no feature(s) configured in
+ tests.
+
+.. topic:: L2XCBASE
+
+ L2XC baseline test cases, no encapsulation, no feature(s) configured in
+ tests.
+
+.. topic:: L2BDBASE
+
+ L2BD baseline test cases, no encapsulation, no feature(s) configured in
+ tests.
+
.. topic:: SCALE
Scale test cases.
At least one feature is configured in test cases. Use also feature tag(s).
+.. topic:: TLDK
+
+ Functional test cases for TLDK.
Forwarding mode tags
--------------------
VPP IPv6 routed forwarding.
-
Underlay tags
-----------
+-------------
.. topic:: IP4UNRLAY
MPLS underlay.
-
Overlay tags
-----------
+------------
.. topic:: L2OVRLAY
IPv6 overlay (IPv6 payload).
-
-Encapsulation tags
-------------------
-
-.. topic:: ETH
-
- All test cases with base Ethernet (no encapsulation).
+Tagging tags
+------------
.. topic:: DOT1Q
All test cases with dot1ad.
+Encapsulation tags
+------------------
+
+.. topic:: ETH
+
+ All test cases with base Ethernet (no encapsulation).
+
.. topic:: LISP
All test cases with LISP.
All test cases with GRE.
+.. topic:: IPSEC
+
+ All test cases with IPSEC.
Interface tags
--------------
All test cases which uses VHOST.
+.. topic:: VHOST_256
+
+ All test cases which uses VHOST with qemu queue size set to 256.
+
+.. topic:: VHOST_1024
+
+ All test cases which uses VHOST with qemu queue size set to 1024.
+
+.. topic:: CFS_OPT
+
+ All test cases which uses VM with optimised scheduler policy.
+
.. topic:: TUNTAP
All test cases which uses TUN and TAP.
All test cases which uses Netmap.
+.. topic:: MEMIF
+
+ All test cases which uses Memif.
Feature tags
------------
COP whitelist.
+.. topic:: SNAT
+
+ SNAT configured and tested.
+
+.. topic:: ACL
+
+ ACL plugin configured and tested.
+
+.. topic:: IACL
+
+ ACL plugin configured and tested on input path.
+
+.. topic:: OACL
+
+ ACL plugin configured and tested on output path.
+
+.. topic:: ACL_STATELESS
+
+ ACL plugin configured and tested in stateless mode (permit action).
+
+.. topic:: ACL_STATEFUL
+
+ ACL plugin configured and tested in stateful mode (permit+reflect action).
+
+.. topic:: ACL1
+
+ ACL plugin configured and tested with 1 not-hitting ACE.
+
+.. topic:: ACL10
+
+ ACL plugin configured and tested with 10 not-hitting ACEs.
+
+.. topic:: ACL50
+
+ ACL plugin configured and tested with 50 not-hitting ACEs.
+
+Encryption tags
+---------------
+
+.. topic:: IPSECSW
+
+ Crypto in software.
+
+.. topic:: IPSECHW
+
+ Crypto in hardware.
+
+.. topic:: IPSECTRAN
+
+ IPSec in transport mode.
+
+.. topic:: IPSECTUN
+
+ IPSec in tunnel mode.
Client-workload tags
--------------------
All test cases with specific APP use.
-
Multi-threading tags
--------------------
8 worker threads pinned to 8 dedicated physical cores. 4 receive queues per
interface. Main thread pinned to core 0.
+
+Honeycomb tags
+--------------
+
+.. topic:: HC_FUNC
+
+ Honeycomb functional test cases.
+
+.. topic:: HC_PERSIST
+
+ Honeycomb persistence test cases.
+
+.. topic:: HC_REST_ONLY
+
+ (Exclusion tag) Honeycomb test cases that cannot be run in Netconf mode
+ using ODL client for Restfconf -> Netconf translation.
Code Review / csit.git / blobdiff