X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Freport%2Fvpp_functional_tests%2Foverview.rst;h=a4635e7f85ac1b308f3b61e3f97e462fbd6e70cb;hp=a6de3f328267c0afd65eedb6d3362df6116a7bdd;hb=bbfcee8d3cf51ec01d269245970ef41bb072c580;hpb=10649b3ce07bbb3144d196c403d8c1fa65ddf116 diff --git a/docs/report/vpp_functional_tests/overview.rst b/docs/report/vpp_functional_tests/overview.rst index a6de3f3282..a4635e7f85 100644 --- a/docs/report/vpp_functional_tests/overview.rst +++ b/docs/report/vpp_functional_tests/overview.rst @@ -1,154 +1,168 @@ -Overview -======== - -Tested Virtual Topologies -------------------------- - -CSIT VPP functional tests are executed on virtualized topologies created using -Virtual Internet Routing Lab (VIRL) simulation platform contributed by Cisco. -VIRL runs on physical baremetal servers hosted by LF FD.io project. Majority -of the tests are executed in the three node logical test topology - Traffic -Generator (TG) node and two Systems Under Test (SUT) nodes connected in a -loop. Some tests use two node logical test topology - TG node and SUT1 node. -Both logical test topologies are shown in the figures below. - -:: - - +------------------------+ +------------------------+ - | | | | - | +------------------+ | | +------------------+ | - | | <-----------------> | | - | | | | | | | | - | | DUT1 <-----------------> DUT2 | | - | +--^--^------------+ | | +------------^--^--+ | - | | | | | | | | - | | | SUT1 | | SUT2 | | | - +------------------------+ +------------------------+ - | | | | - | | | | - | | +-----------+ | | - | +---------------> <---------------+ | - | | TG | | - +------------------> <------------------+ - +-----------+ - - +------------------------+ - | | - | +------------------+ | - +---------------> <--------------+ - | | | | | | - | |------------> DUT1 <-----------+ | - | | | +------------------+ | | | - | | | | | | - | | | SUT1 | | | - | | +------------------------+ | | - | | | | - | | | | - | | +-----------+ | | - | +---------------> <---------------+ | - | | TG | | - +------------------> <------------------+ - +-----------+ - -SUT1 and SUT2 are two VMs (Ubuntu or Centos, depending on the test suite), TG -is a Traffic Generator (TG, another Ubuntu VM). SUTs run VPP SW application in -Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy SW -application as a packet Traffic Generator. Logical connectivity between SUTs -and to TG is provided using virtual NICs using VMs' virtio driver. - -Virtual testbeds are created on-demand whenever a verification job is started -(e.g. triggered by the gerrit patch submission) and destroyed upon completion -of all functional tests. Each node is a Virtual Machine and each connection -that is drawn on the diagram is available for use in any test case. During the -test execution, all nodes are reachable thru the Management network connected -to every node via dedicated virtual NICs and virtual links (not shown above -for clarity). - -For the test cases that require DUT (VPP) to communicate with VM over the -vhost-user interfaces, a nested VM is created on SUT1 and/or SUT2 for the -duration of these particular test cases only. DUT (VPP) test topology with VM -is shown in the figure below including the applicable packet flow thru the VM -(marked in the figure with ``***``). - -:: - - +------------------------+ +------------------------+ - | +----------+ | | +----------+ | - | | VM | | | | VM | | - | | ****** | | | | ****** | | - | +--^----^--+ | | +--^----^--+ | - | *| |* | | *| |* | - | +------v----v------+ | | +------v----v------+ | - | | * * |**|***********|**| * * | | - | | ***** *******<----------------->******* ***** | | - | | * DUT1 | | | | DUT2 * | | - | +--^---------------+ | | +---------------^--+ | - | *| | | |* | - | *| SUT1 | | SUT2 |* | - +------------------------+ +------------------^-----+ - *| |* - *| |* - *| +-----------+ |* - *| | | |* - *+------------------> TG <------------------+* - ******************* | |******************** - +-----------+ - -Functional Tests Coverage -------------------------- - -Following VPP functional test areas are covered in the CSIT |release| with -results listed in this report: - -- **DHCP - Client and Proxy** - Dynamic Host Control Protocol Client and Proxy - for IPv4, IPv6. -- **GRE Overlay Tunnels** - Generic Routing Encapsulation for IPv4. -- **L2BD Ethernet Switching** - L2 Bridge-Domain switched-forwarding for - untagged Ethernet, dot1q and dot1ad tagged. -- **L2XC Ethernet Switching** - L2 Cross-Connect switched-forwarding for - untagged Ethernet, dot1q and dot1ad tagged. -- **LISP Overlay Tunnels** - Locator/ID Separation Protocol overlay tunnels and - locator/id mapping control. -- **Softwire Tunnels** - IPv4-in-IPv6 softwire tunnels. -- **Cop Address Security** - address white-list and black-list filtering for - IPv4, IPv6. -- **IPSec - Tunnels and Transport** - IPSec tunnel and transport modes. -- **IPv6 Routed-Forwarding** - IPv6 routed-forwarding, NS/ND, RA, ICMPv6. -- **uRPF Source Security** - unicast Reverse Path Forwarding security. -- **Tap Interface** - baseline Linux tap interface tests. -- **Telemetry - IPFIX and SPAN** - IPFIX netflow statistics and SPAN port - mirroring. -- **VRF Routed-Forwarding** - multi-context IPVPN routed-forwarding for IPv4, - IPv6. -- **iACL Security** - ingress Access Control List security for IPv4, IPv6, MAC. -- **IPv4 Routed-Forwarding** - IPv4 routed-forwarding, RPF, ARP, Proxy ARP, - ICMPv4. -- **QoS Policer Metering** - ingress packet rate measuring and marking for IPv4, - IPv6. -- **VLAN Tag Translation** - L2 VLAN tag translation 2to2, 2to1, 1to2, 1to1. -- **VXLAN Overlay Tunnels** - VXLAN tunneling for L2-over-IP, for IPv4, IPv6. - -Functional Tests Naming ------------------------ - -CSIT |release| follows a common structured naming convention for all -performance and system functional tests, introduced in CSIT |release-1|. - -The naming should be intuitive for majority of the tests. Complete -description of CSIT test naming convention is provided on `CSIT test naming -page `_. - -Here few illustrative examples of the new naming usage for functional test -suites: - -#. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P** - - - *eth2p-ethip4-ip4base-func.robot* => 2 ports of Ethernet, IPv4 baseline - routed forwarding, functional tests. - -#. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC, - P2V2P, NIC2VMchain2NIC, P2V2V2P** - - - *eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot* => 2 ports of - Ethernet, IPv4 VXLAN Ethernet, L2 bridge-domain switching to/from two vhost - interfaces and one VM, functional tests. +Overview +======== + +Virtual Topologies +------------------ + +CSIT VPP functional tests are executed in VM-based virtual topologies +created on demand using :abbr:`VIRL (Virtual Internet Routing Lab)` +simulation platform contributed by Cisco. VIRL runs on physical +baremetal servers hosted by LF FD.io project. Based on the packet path +thru SUT VMs, two distinct logical topology types are used for VPP DUT +data plane testing: + +#. vNIC-to-vNIC switching topologies. +#. Nested-VM service switching topologies. + +vNIC-to-vNIC Switching +~~~~~~~~~~~~~~~~~~~~~~ + +The simplest virtual topology for software data plane application like +VPP is vNIC-to-vNIC switching. Tested virtual topologies for 2-Node and +3-Node testbeds are shown in figures below. + +.. only:: latex + + .. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_tmp/src/vpp_functional_tests/}} + \includegraphics[width=0.90\textwidth]{virtual-2n-nic2nic} + \label{fig:virtual-2n-nic2nic} + \end{figure} + +.. only:: html + + .. figure:: ../vpp_functional_tests/virtual-2n-nic2nic.svg + :alt: virtual-2n-nic2nic + :align: center + + +.. only:: latex + + .. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_tmp/src/vpp_functional_tests/}} + \includegraphics[width=0.90\textwidth]{virtual-3n-nic2nic} + \label{fig:virtual-3n-nic2nic} + \end{figure} + +.. only:: html + + .. figure:: ../vpp_functional_tests/virtual-3n-nic2nic.svg + :alt: virtual-3n-nic2nic + :align: center + +SUT1 and SUT2 are two VMs (running Ubuntu or Centos, depending on the test +suite), TG is a Traffic Generator (running Ubuntu VM). SUTs run VPP +SW application in Linux user-mode as a Device Under Test (DUT) within +the VM. TG runs Scapy SW application as a packet Traffic Generator. +Network connectivity between SUTs and to TG is provided using virtual +NICs and VMs' virtio drivers. + +Virtual testbeds are created on-demand whenever a verification job is +started (e.g. triggered by the gerrit patch submission) and destroyed +upon completion of all functional tests. Each node is a Virtual Machine +and each connection that is drawn on the diagram is available for use in +any test case. During the test execution, all nodes are reachable thru +the Management network connected to every node via dedicated virtual +NICs and virtual links (not shown above for clarity). + +Nested-VM Service Switching +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Nested-VM (Virtual Machine) service switching topology test cases +require VPP DUT to communicate with nested-VM(s) over vhost-user virtual +interfaces. Nested-VM(s) is(are) created on SUT1 and/or SUT2 for the +duration of these particular test cases only. Virtual test topology with +nested-VM(s) is shown in the figure below. + +.. only:: latex + + .. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_tmp/src/vpp_functional_tests/}} + \includegraphics[width=0.90\textwidth]{virtual-3n-vm-vhost} + \label{fig:virtual-3n-vm-vhost} + \end{figure} + +.. only:: html + + .. figure:: ../vpp_functional_tests/virtual-3n-vm-vhost.svg + :alt: virtual-3n-vm-vhost + :align: center + +Functional Tests Coverage +------------------------- + +|csit-release| includes following VPP functionality tested in virtual VM +environment: + ++-----------------------+----------------------------------------------+ +| Functionality | Description | ++=======================+==============================================+ +| ACL | Ingress Access Control List security for L2 | +| | Bridge-Domain MAC switching, IPv4 routing, | +| | IPv6 routing. | ++-----------------------+----------------------------------------------+ +| COP | COP address white-list and black-list | +| | filtering for IPv4 and IPv6 routing. | ++-----------------------+----------------------------------------------+ +| DHCP | Dynamic Host Control Protocol Client and | +| | Proxy for IPv4 and IPv6 routing. | ++-----------------------+----------------------------------------------+ +| GRE | Generic Routing Encapsulation Overlay | +| | Tunnels for IPv4. | ++-----------------------+----------------------------------------------+ +| IPSec | IPSec tunnel and transport modes. | ++-----------------------+----------------------------------------------+ +| IPv4 | IPv4 routing, RPF, ARP, Proxy ARP, ICMPv4. | ++-----------------------+----------------------------------------------+ +| IPv6 | IPv6 routing, NS/ND, RA, ICMPv6. | ++-----------------------+----------------------------------------------+ +| L2BD | L2 Bridge-Domain switching for untagged | +| | Ethernet, dot1q and dot1ad tagged. | ++-----------------------+----------------------------------------------+ +| L2XC | L2 Cross-Connect switching for untagged | +| | Ethernet, dot1q and dot1ad tagged. | ++-----------------------+----------------------------------------------+ +| LISP | Locator/ID Separation Protocol overlay | +| | tunnels and locator/id mapping control. | ++-----------------------+----------------------------------------------+ +| QoS Policer Metering | Ingress packet rate metering and marking for | +| | IPv4, IPv6. | ++-----------------------+----------------------------------------------+ +| Softwire Tunnels | IPv4-in-IPv6 softwire tunnels. | ++-----------------------+----------------------------------------------+ +| Tap Interface | Baseline Linux tap interface tests. | ++-----------------------+----------------------------------------------+ +| IPFIX and SPAN | Telemetry IPFIX netflow statistics and SPAN | +| | port mirroring. | ++-----------------------+----------------------------------------------+ +| uRPF Source Security | Unicast Reverse Path Forwarding security for | +| | IPv4 and IPv6 routing. | ++-----------------------+----------------------------------------------+ +| VLAN Tag Translation | L2 VLAN tag translation 2to2, 2to1, 1to2, | +| | 1to1. | ++-----------------------+----------------------------------------------+ +| VRF Routing | Multi-context VRF IPVPN routing for IPv4 and | +| | IPv6. | ++-----------------------+----------------------------------------------+ +| VXLAN | VXLAN overlay tunneling for L2-over-IPv4 and | +| | -over-IPv6. | ++-----------------------+----------------------------------------------+ + +Functional Tests Naming +----------------------- + +|csit-release| follows a common structured naming convention for all +performance and system functional tests, introduced in CSIT-17.01. + +The naming should be intuitive for majority of the tests. Complete +description of CSIT test naming convention is provided on +:ref:`csit_test_naming`.