--- /dev/null
+Overview\r
+========\r
+\r
+Tested Virtual Topologies\r
+-------------------------\r
+\r
+CSIT VPP functional tests are executed on virtualized topologies created using\r
+Virtual Internet Routing Lab (VIRL) simulation platform contributed by Cisco.\r
+VIRL runs on physical baremetal servers hosted by LF FD.io project. Majority\r
+of the tests are executed in the three node logical test topology - Traffic\r
+Generator (TG) node and two Systems Under Test (SUT) nodes connected in a\r
+loop. Some tests use two node logical test topology - TG node and SUT1 node.\r
+Both logical test topologies are shown in the figures below.\r
+\r
+::\r
+\r
+ +------------------------+ +------------------------+\r
+ | | | |\r
+ | +------------------+ | | +------------------+ |\r
+ | | <-----------------> | |\r
+ | | | | | | | |\r
+ | | DUT1 <-----------------> DUT2 | |\r
+ | +--^--^------------+ | | +------------^--^--+ |\r
+ | | | | | | | |\r
+ | | | SUT1 | | SUT2 | | |\r
+ +------------------------+ +------------------------+\r
+ | | | |\r
+ | | | |\r
+ | | +-----------+ | |\r
+ | +---------------> <---------------+ |\r
+ | | TG | |\r
+ +------------------> <------------------+\r
+ +-----------+\r
+\r
+ +------------------------+\r
+ | |\r
+ | +------------------+ |\r
+ +---------------> <--------------+\r
+ | | | | | |\r
+ | |------------> DUT1 <-----------+ |\r
+ | | | +--^--^------------+ | | |\r
+ | | | | | |\r
+ | | | SUT1 | | |\r
+ | | +------------------------+ | |\r
+ | | | |\r
+ | | | |\r
+ | | +-----------+ | |\r
+ | +---------------> <---------------+ |\r
+ | | TG | |\r
+ +------------------> <------------------+\r
+ +-----------+\r
+\r
+SUT1 and SUT2 are two VMs (Ubuntu or Centos, depending on the test suite), TG\r
+is a Traffic Generator (TG, another Ubuntu VM). SUTs run VPP SW application in\r
+Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy SW\r
+application as a packet Traffic Generator. Logical connectivity between SUTs\r
+and to TG is provided using virtual NICs using VMs' virtio driver.\r
+\r
+Virtual testbeds are created on-demand whenever a verification job is started\r
+(e.g. triggered by the gerrit patch submission) and destroyed upon completion\r
+of all functional tests. Each node is a Virtual Machine and each connection\r
+that is drawn on the diagram is available for use in any test case. During the\r
+test execution, all nodes are reachable thru the Management network connected\r
+to every node via dedicated virtual NICs and virtual links (not shown above\r
+for clarity).\r
+\r
+For the test cases that require DUT (VPP) to communicate with VM over the\r
+vhost-user interfaces, a nested VM is created on SUT1 and/or SUT2 for the\r
+duration of these particular test cases only. DUT (VPP) test topology with VM\r
+is shown in the figure below including the applicable packet flow thru the VM\r
+(marked in the figure with ``***``).\r
+\r
+::\r
+\r
+ +------------------------+ +------------------------+\r
+ | +----------+ | | +----------+ |\r
+ | | VM | | | | VM | |\r
+ | | ****** | | | | ****** | |\r
+ | +--^----^--+ | | +--^----^--+ |\r
+ | *| |* | | *| |* |\r
+ | +------v----v------+ | | +------v----v------+ |\r
+ | | * * |**|***********|**| * * | |\r
+ | | ***** *******<----------------->******* ***** | |\r
+ | | * DUT1 | | | | DUT2 * | |\r
+ | +--^---------------+ | | +---------------^--+ |\r
+ | *| | | |* |\r
+ | *| SUT1 | | SUT2 |* |\r
+ +------------------------+ +------------------^-----+\r
+ *| |*\r
+ *| |*\r
+ *| +-----------+ |*\r
+ *| | | |*\r
+ *+------------------> TG <------------------+*\r
+ ******************* | |********************\r
+ +-----------+\r
+\r
+Functional Tests Coverage\r
+-------------------------\r
+\r
+Following VPP functional test areas are covered in the CSIT |release| with\r
+results listed in this report:\r
+\r
+- **DHCP - Client and Proxy** - Dynamic Host Control Protocol Client and Proxy\r
+ for IPv4, IPv6.\r
+- **GRE Overlay Tunnels** - Generic Routing Encapsulation for IPv4.\r
+- **L2BD Ethernet Switching** - L2 Bridge-Domain switched-forwarding for\r
+ untagged Ethernet, dot1q and dot1ad tagged.\r
+- **L2XC Ethernet Switching** - L2 Cross-Connect switched-forwarding for\r
+ untagged Ethernet, dot1q and dot1ad tagged.\r
+- **LISP Overlay Tunnels** - Locator/ID Separation Protocol overlay tunnels and\r
+ locator/id mapping control.\r
+- **Softwire Tunnels** - IPv4-in-IPv6 softwire tunnels.\r
+- **Cop Address Security** - address white-list and black-list filtering for\r
+ IPv4, IPv6.\r
+- **IPSec - Tunnels and Transport** - IPSec tunnel and transport modes.\r
+- **IPv6 Routed-Forwarding** - IPv6 routed-forwarding, NS/ND, RA, ICMPv6.\r
+- **uRPF Source Security** - unicast Reverse Path Forwarding security.\r
+- **Tap Interface** - baseline Linux tap interface tests.\r
+- **Telemetry - IPFIX and SPAN** - IPFIX netflow statistics and SPAN port\r
+ mirroring.\r
+- **VRF Routed-Forwarding** - multi-context IPVPN routed-forwarding for IPv4,\r
+ IPv6.\r
+- **iACL Security** - ingress Access Control List security for IPv4, IPv6, MAC.\r
+- **IPv4 Routed-Forwarding** - IPv4 routed-forwarding, RPF, ARP, Proxy ARP,\r
+ ICMPv4.\r
+- **QoS Policer Metering** - ingress packet rate measuring and marking for IPv4,\r
+ IPv6.\r
+- **VLAN Tag Translation** - L2 VLAN tag translation 2to2, 2to1, 1to2, 1to1.\r
+- **VXLAN Overlay Tunnels** - VXLAN tunneling for L2-over-IP, for IPv4, IPv6.\r
+\r
+Functional Tests Naming\r
+-----------------------\r
+\r
+CSIT |release| introduced a common structured naming convention for all\r
+performance and functional tests. This change was driven by substantially\r
+growing number and type of CSIT test cases. Firstly, the original practice did\r
+not always follow any strict naming convention. Secondly test names did not\r
+always clearly capture tested packet encapsulations, and the actual type or\r
+content of the tests. Thirdly HW configurations in terms of NICs, ports and\r
+their locality were not captured either. These were but few reasons that drove\r
+the decision to change and define a new more complete and stricter test naming\r
+convention, and to apply this to all existing and new test cases.\r
+\r
+The new naming should be intuitive for majority of the tests. The complete\r
+description of CSIT test naming convention is provided on `CSIT test naming\r
+page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.\r
+\r
+Here few illustrative examples of the new naming usage for functional test\r
+suites:\r
+\r
+#. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P**\r
+\r
+ - *eth2p-ethip4-ip4base-func.robot* => 2 ports of Ethernet, IPv4 baseline\r
+ routed forwarding, functional tests.\r
+\r
+#. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC,\r
+ P2V2P, NIC2VMchain2NIC, P2V2V2P**\r
+\r
+ - *eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot* => 2 ports of\r
+ Ethernet, IPv4 VXLAN Ethernet, L2 bridge-domain switching to/from two vhost\r
+ interfaces and one VM, functional tests.\r
Code Review / csit.git / blobdiff