-Overview\r
-========\r
-\r
-Tested Virtual Topologies\r
--------------------------\r
-\r
-CSIT VPP functional tests are executed on virtualized topologies created using\r
-Virtual Internet Routing Lab (VIRL) simulation platform contributed by Cisco.\r
-VIRL runs on physical baremetal servers hosted by LF FD.io project. Majority\r
-of the tests are executed in the three node logical test topology - Traffic\r
-Generator (TG) node and two Systems Under Test (SUT) nodes connected in a\r
-loop. Some tests use two node logical test topology - TG node and SUT1 node.\r
-Both logical test topologies are shown in the figures below.\r
-\r
-::\r
-\r
- +------------------------+ +------------------------+\r
- | | | |\r
- | +------------------+ | | +------------------+ |\r
- | | <-----------------> | |\r
- | | | | | | | |\r
- | | DUT1 <-----------------> DUT2 | |\r
- | +--^--^------------+ | | +------------^--^--+ |\r
- | | | | | | | |\r
- | | | SUT1 | | SUT2 | | |\r
- +------------------------+ +------------------------+\r
- | | | |\r
- | | | |\r
- | | +-----------+ | |\r
- | +---------------> <---------------+ |\r
- | | TG | |\r
- +------------------> <------------------+\r
- +-----------+\r
-\r
- +------------------------+\r
- | |\r
- | +------------------+ |\r
- +---------------> <--------------+\r
- | | | | | |\r
- | |------------> DUT1 <-----------+ |\r
- | | | +------------------+ | | |\r
- | | | | | |\r
- | | | SUT1 | | |\r
- | | +------------------------+ | |\r
- | | | |\r
- | | | |\r
- | | +-----------+ | |\r
- | +---------------> <---------------+ |\r
- | | TG | |\r
- +------------------> <------------------+\r
- +-----------+\r
-\r
-SUT1 and SUT2 are two VMs (Ubuntu or Centos, depending on the test suite), TG\r
-is a Traffic Generator (TG, another Ubuntu VM). SUTs run VPP SW application in\r
-Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy SW\r
-application as a packet Traffic Generator. Logical connectivity between SUTs\r
-and to TG is provided using virtual NICs using VMs' virtio driver.\r
-\r
-Virtual testbeds are created on-demand whenever a verification job is started\r
-(e.g. triggered by the gerrit patch submission) and destroyed upon completion\r
-of all functional tests. Each node is a Virtual Machine and each connection\r
-that is drawn on the diagram is available for use in any test case. During the\r
-test execution, all nodes are reachable thru the Management network connected\r
-to every node via dedicated virtual NICs and virtual links (not shown above\r
-for clarity).\r
-\r
-For the test cases that require DUT (VPP) to communicate with VM over the\r
-vhost-user interfaces, a nested VM is created on SUT1 and/or SUT2 for the\r
-duration of these particular test cases only. DUT (VPP) test topology with VM\r
-is shown in the figure below including the applicable packet flow thru the VM\r
-(marked in the figure with ``***``).\r
-\r
-::\r
-\r
- +------------------------+ +------------------------+\r
- | +----------+ | | +----------+ |\r
- | | VM | | | | VM | |\r
- | | ****** | | | | ****** | |\r
- | +--^----^--+ | | +--^----^--+ |\r
- | *| |* | | *| |* |\r
- | +------v----v------+ | | +------v----v------+ |\r
- | | * * |**|***********|**| * * | |\r
- | | ***** *******<----------------->******* ***** | |\r
- | | * DUT1 | | | | DUT2 * | |\r
- | +--^---------------+ | | +---------------^--+ |\r
- | *| | | |* |\r
- | *| SUT1 | | SUT2 |* |\r
- +------------------------+ +------------------^-----+\r
- *| |*\r
- *| |*\r
- *| +-----------+ |*\r
- *| | | |*\r
- *+------------------> TG <------------------+*\r
- ******************* | |********************\r
- +-----------+\r
-\r
-Functional Tests Coverage\r
--------------------------\r
-\r
-Following VPP functional test areas are covered in the CSIT |release| with\r
-results listed in this report:\r
-\r
-- **DHCP - Client and Proxy** - Dynamic Host Control Protocol Client and Proxy\r
- for IPv4, IPv6.\r
-- **GRE Overlay Tunnels** - Generic Routing Encapsulation for IPv4.\r
-- **L2BD Ethernet Switching** - L2 Bridge-Domain switched-forwarding for\r
- untagged Ethernet, dot1q and dot1ad tagged.\r
-- **L2XC Ethernet Switching** - L2 Cross-Connect switched-forwarding for\r
- untagged Ethernet, dot1q and dot1ad tagged.\r
-- **LISP Overlay Tunnels** - Locator/ID Separation Protocol overlay tunnels and\r
- locator/id mapping control.\r
-- **Softwire Tunnels** - IPv4-in-IPv6 softwire tunnels.\r
-- **Cop Address Security** - address white-list and black-list filtering for\r
- IPv4, IPv6.\r
-- **IPSec - Tunnels and Transport** - IPSec tunnel and transport modes.\r
-- **IPv6 Routed-Forwarding** - IPv6 routed-forwarding, NS/ND, RA, ICMPv6.\r
-- **uRPF Source Security** - unicast Reverse Path Forwarding security.\r
-- **Tap Interface** - baseline Linux tap interface tests.\r
-- **Telemetry - IPFIX and SPAN** - IPFIX netflow statistics and SPAN port\r
- mirroring.\r
-- **VRF Routed-Forwarding** - multi-context IPVPN routed-forwarding for IPv4,\r
- IPv6.\r
-- **iACL Security** - ingress Access Control List security for IPv4, IPv6, MAC.\r
-- **IPv4 Routed-Forwarding** - IPv4 routed-forwarding, RPF, ARP, Proxy ARP,\r
- ICMPv4.\r
-- **QoS Policer Metering** - ingress packet rate measuring and marking for IPv4,\r
- IPv6.\r
-- **VLAN Tag Translation** - L2 VLAN tag translation 2to2, 2to1, 1to2, 1to1.\r
-- **VXLAN Overlay Tunnels** - VXLAN tunneling for L2-over-IP, for IPv4, IPv6.\r
-\r
-Functional Tests Naming\r
------------------------\r
-\r
-CSIT |release| follows a common structured naming convention for all\r
-performance and system functional tests, introduced in CSIT |release-1|.\r
-\r
-The naming should be intuitive for majority of the tests. Complete\r
-description of CSIT test naming convention is provided on `CSIT test naming\r
-page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.\r
-\r
-Here few illustrative examples of the new naming usage for functional test\r
-suites:\r
-\r
-#. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P**\r
-\r
- - *eth2p-ethip4-ip4base-func.robot* => 2 ports of Ethernet, IPv4 baseline\r
- routed forwarding, functional tests.\r
-\r
-#. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC,\r
- P2V2P, NIC2VMchain2NIC, P2V2V2P**\r
-\r
- - *eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot* => 2 ports of\r
- Ethernet, IPv4 VXLAN Ethernet, L2 bridge-domain switching to/from two vhost\r
- interfaces and one VM, functional tests.\r
+Overview
+========
+
+Virtual Topologies
+------------------
+
+CSIT VPP functional tests are executed on virtualized topologies created using
+:abbr:`VIRL (Virtual Internet Routing Lab)` simulation platform contributed by
+Cisco. VIRL runs on physical baremetal servers hosted by LF FD.io project.
+Based on the packet path thru server SUTs, two distinct logical topology types
+are used for VPP DUT data plane testing:
+
+#. NIC-to-NIC switching topologies.
+#. VM service switching topologies.
+
+NIC-to-NIC Switching
+~~~~~~~~~~~~~~~~~~~~
+
+The simplest logical topology for software data plane application like
+VPP is NIC-to-NIC switching. Tested topologies for 2-Node and 3-Node
+testbeds are shown in figures below.
+
+.. only:: latex
+
+ .. raw:: latex
+
+ \begin{figure}[H]
+ \centering
+ \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+ \includegraphics[width=0.90\textwidth]{logical-2n-nic2nic}
+ \label{fig:logical-2n-nic2nic}
+ \end{figure}
+
+.. only:: html
+
+ .. figure:: ../vpp_performance_tests/logical-2n-nic2nic.svg
+ :alt: logical-2n-nic2nic
+ :align: center
+
+
+.. only:: latex
+
+ .. raw:: latex
+
+ \begin{figure}[H]
+ \centering
+ \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+ \includegraphics[width=0.90\textwidth]{logical-3n-nic2nic}
+ \label{fig:logical-3n-nic2nic}
+ \end{figure}
+
+.. only:: html
+
+ .. figure:: ../vpp_performance_tests/logical-3n-nic2nic.svg
+ :alt: logical-3n-nic2nic
+ :align: center
+
+SUT1 and SUT2 are two VMs (Ubuntu or Centos, depending on the test suite), TG
+is a Traffic Generator (TG, another Ubuntu VM). SUTs run VPP SW application in
+Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy SW
+application as a packet Traffic Generator. Logical connectivity between SUTs
+and to TG is provided using virtual NICs using VMs' virtio driver.
+
+Virtual testbeds are created on-demand whenever a verification job is started
+(e.g. triggered by the gerrit patch submission) and destroyed upon completion
+of all functional tests. Each node is a Virtual Machine and each connection
+that is drawn on the diagram is available for use in any test case. During the
+test execution, all nodes are reachable thru the Management network connected
+to every node via dedicated virtual NICs and virtual links (not shown above
+for clarity).
+
+VM Service Switching
+~~~~~~~~~~~~~~~~~~~~
+
+VM service switching topology test cases require VPP DUT to communicate
+with Virtual Machines (VMs) over vhost-user virtual interfaces. A nested VM is
+created on SUT1 and/or SUT2 for the duration of these particular test cases
+only. DUT (VPP) test topology with VM is shown in the figure below.
+
+.. only:: latex
+
+ .. raw:: latex
+
+ \begin{figure}[H]
+ \centering
+ \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+ \includegraphics[width=0.90\textwidth]{logical-2n-vm-vhost}
+ \label{fig:logical-2n-vm-vhost}
+ \end{figure}
+
+.. only:: html
+
+ .. figure:: ../vpp_performance_tests/logical-2n-vm-vhost.svg
+ :alt: logical-2n-vm-vhost
+ :align: center
+
+
+.. only:: latex
+
+ .. raw:: latex
+
+ \begin{figure}[H]
+ \centering
+ \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+ \includegraphics[width=0.90\textwidth]{logical-3n-vm-vhost}
+ \label{fig:logical-3n-vm-vhost}
+ \end{figure}
+
+.. only:: html
+
+ .. figure:: ../vpp_performance_tests/logical-3n-vm-vhost.svg
+ :alt: logical-3n-vm-vhost
+ :align: center
+
+Functional Tests Coverage
+-------------------------
+
+Following VPP functional test areas are covered in the |csit-release| with
+results listed in this report:
+
+- **DHCP - Client and Proxy** - Dynamic Host Control Protocol Client and Proxy
+ for IPv4, IPv6.
+- **GRE Overlay Tunnels** - Generic Routing Encapsulation for IPv4.
+- **L2BD Ethernet Switching** - L2 Bridge-Domain switched-forwarding for
+ untagged Ethernet, dot1q and dot1ad tagged.
+- **L2XC Ethernet Switching** - L2 Cross-Connect switched-forwarding for
+ untagged Ethernet, dot1q and dot1ad tagged.
+- **LISP Overlay Tunnels** - Locator/ID Separation Protocol overlay tunnels and
+ locator/id mapping control.
+- **Softwire Tunnels** - IPv4-in-IPv6 softwire tunnels.
+- **Cop Address Security** - address white-list and black-list filtering for
+ IPv4, IPv6.
+- **IPSec - Tunnels and Transport** - IPSec tunnel and transport modes.
+- **IPv6 Routed-Forwarding** - IPv6 routed-forwarding, NS/ND, RA, ICMPv6.
+- **uRPF Source Security** - unicast Reverse Path Forwarding security.
+- **Tap Interface** - baseline Linux tap interface tests.
+- **Telemetry - IPFIX and SPAN** - IPFIX netflow statistics and SPAN port
+ mirroring.
+- **VRF Routed-Forwarding** - multi-context IPVPN routed-forwarding for IPv4,
+ IPv6.
+- **iACL Security** - ingress Access Control List security for IPv4, IPv6, MAC.
+- **IPv4 Routed-Forwarding** - IPv4 routed-forwarding, RPF, ARP, Proxy ARP,
+ ICMPv4.
+- **QoS Policer Metering** - ingress packet rate measuring and marking for IPv4,
+ IPv6.
+- **VLAN Tag Translation** - L2 VLAN tag translation 2to2, 2to1, 1to2, 1to1.
+- **VXLAN Overlay Tunnels** - VXLAN tunneling for L2-over-IP, for IPv4, IPv6.
+
+Functional Tests Naming
+-----------------------
+
+|csit-release| follows a common structured naming convention for all performance
+and system functional tests, introduced in CSIT-17.01.
+
+The naming should be intuitive for majority of the tests. Complete description
+of CSIT test naming convention is provided on :ref:`csit_test_naming`.
Code Review / csit.git / blobdiff