docs/report/vpp_functional_tests/overview.rst

   1 Overview
   2 ========
   3
   4 Tested Virtual Topologies
   5 -------------------------
   6
   7 CSIT VPP functional tests are executed on virtualized topologies created using
   8 :abbr:`VIRL (Virtual Internet Routing Lab)` simulation platform contributed by
   9 Cisco. VIRL runs on physical baremetal servers hosted by LF FD.io project.
  10 Majority of the tests are executed in the three node logical test topology -
  11 Traffic Generator (TG) node and two Systems Under Test (SUT) nodes connected in
  12 a loop. Some tests use two node logical test topology - TG node and SUT1 node.
  13 Both logical test topologies are shown in the figures below.::
  14
  15     +------------------------+           +------------------------+
  16     |                        |           |                        |
  17     |  +------------------+  |           |  +------------------+  |
  18     |  |                  <----------------->                  |  |
  19     |  |                  |  |           |  |                  |  |
  20     |  |       DUT1       <----------------->       DUT2       |  |
  21     |  +--^--^------------+  |           |  +------------^--^--+  |
  22     |     |  |               |           |               |  |     |
  23     |     |  |         SUT1  |           |  SUT2         |  |     |
  24     +------------------------+           +------------------------+
  25           |  |                                           |  |
  26           |  |                                           |  |
  27           |  |               +-----------+               |  |
  28           |  +--------------->           <---------------+  |
  29           |                  |    TG     |                  |
  30           +------------------>           <------------------+
  31                              +-----------+
  32
  33                        +------------------------+
  34                        |                        |
  35                        |  +------------------+  |
  36           +--------------->                  <--------------+
  37           |            |  |                  |  |           |
  38           |  |------------>       DUT1       <-----------+  |
  39           |  |         |  +------------------+  |        |  |
  40           |  |         |                        |        |  |
  41           |  |         |                  SUT1  |        |  |
  42           |  |         +------------------------+        |  |
  43           |  |                                           |  |
  44           |  |                                           |  |
  45           |  |               +-----------+               |  |
  46           |  +--------------->           <---------------+  |
  47           |                  |    TG     |                  |
  48           +------------------>           <------------------+
  49                              +-----------+
  50
  51 SUT1 and SUT2 are two VMs (Ubuntu or Centos, depending on the test suite), TG
  52 is a Traffic Generator (TG, another Ubuntu VM). SUTs run VPP SW application in
  53 Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy SW
  54 application as a packet Traffic Generator. Logical connectivity between SUTs
  55 and to TG is provided using virtual NICs using VMs' virtio driver.
  56
  57 Virtual testbeds are created on-demand whenever a verification job is started
  58 (e.g. triggered by the gerrit patch submission) and destroyed upon completion
  59 of all functional tests. Each node is a Virtual Machine and each connection
  60 that is drawn on the diagram is available for use in any test case. During the
  61 test execution, all nodes are reachable thru the Management network connected
  62 to every node via dedicated virtual NICs and virtual links (not shown above
  63 for clarity).
  64
  65 For the test cases that require DUT (VPP) to communicate with VM over the
  66 vhost-user interfaces, a nested VM is created on SUT1 and/or SUT2 for the
  67 duration of these particular test cases only. DUT (VPP) test topology with VM
  68 is shown in the figure below including the applicable packet flow thru the VM
  69 (marked in the figure with ``***``).::
  70
  71     +------------------------+           +------------------------+
  72     |      +----------+      |           |      +----------+      |
  73     |      |    VM    |      |           |      |    VM    |      |
  74     |      |  ******  |      |           |      |  ******  |      |
  75     |      +--^----^--+      |           |      +--^----^--+      |
  76     |        *|    |*        |           |        *|    |*        |
  77     |  +------v----v------+  |           |  +------v----v------+  |
  78     |  |      *    *      |**|***********|**|      *    *      |  |
  79     |  |  *****    *******<----------------->*******    *****  |  |
  80     |  |  *    DUT1       |  |           |  |       DUT2    *  |  |
  81     |  +--^---------------+  |           |  +---------------^--+  |
  82     |    *|                  |           |                  |*    |
  83     |    *|            SUT1  |           |  SUT2            |*    |
  84     +------------------------+           +------------------^-----+
  85          *|                                                 |*
  86          *|                                                 |*
  87          *|                  +-----------+                  |*
  88          *|                  |           |                  |*
  89          *+------------------>    TG     <------------------+*
  90          ******************* |           |********************
  91                              +-----------+
  92
  93 Functional Tests Coverage
  94 -------------------------
  95
  96 Following VPP functional test areas are covered in the |csit-release| with
  97 results listed in this report:
  98
  99 - **DHCP - Client and Proxy** - Dynamic Host Control Protocol Client and Proxy
 100   for IPv4, IPv6.
 101 - **GRE Overlay Tunnels** - Generic Routing Encapsulation for IPv4.
 102 - **L2BD Ethernet Switching** - L2 Bridge-Domain switched-forwarding for
 103   untagged Ethernet, dot1q and dot1ad tagged.
 104 - **L2XC Ethernet Switching** - L2 Cross-Connect switched-forwarding for
 105   untagged Ethernet, dot1q and dot1ad tagged.
 106 - **LISP Overlay Tunnels** - Locator/ID Separation Protocol overlay tunnels and
 107   locator/id mapping control.
 108 - **Softwire Tunnels** - IPv4-in-IPv6 softwire tunnels.
 109 - **Cop Address Security** - address white-list and black-list filtering for
 110   IPv4, IPv6.
 111 - **IPSec - Tunnels and Transport** - IPSec tunnel and transport modes.
 112 - **IPv6 Routed-Forwarding** - IPv6 routed-forwarding, NS/ND, RA, ICMPv6.
 113 - **uRPF Source Security** - unicast Reverse Path Forwarding security.
 114 - **Tap Interface** - baseline Linux tap interface tests.
 115 - **Telemetry - IPFIX and SPAN** - IPFIX netflow statistics and SPAN port
 116   mirroring.
 117 - **VRF Routed-Forwarding** - multi-context IPVPN routed-forwarding for IPv4,
 118   IPv6.
 119 - **iACL Security** - ingress Access Control List security for IPv4, IPv6, MAC.
 120 - **IPv4 Routed-Forwarding** - IPv4 routed-forwarding, RPF, ARP, Proxy ARP,
 121   ICMPv4.
 122 - **QoS Policer Metering** - ingress packet rate measuring and marking for IPv4,
 123   IPv6.
 124 - **VLAN Tag Translation** - L2 VLAN tag translation 2to2, 2to1, 1to2, 1to1.
 125 - **VXLAN Overlay Tunnels** - VXLAN tunneling for L2-over-IP, for IPv4, IPv6.
 126
 127 Functional Tests Naming
 128 -----------------------
 129
 130 |csit-release| follows a common structured naming convention for all performance
 131 and system functional tests, introduced in CSIT rls1701.
 132
 133 The naming should be intuitive for majority of the tests. Complete description
 134 of CSIT test naming convention is provided on :ref:`csit_test_naming`..
 135
 136 Here few illustrative examples of the new naming usage for functional test
 137 suites:
 138
 139 #. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P**
 140
 141    - *eth2p-ethip4-ip4base-func.robot* => 2 ports of Ethernet, IPv4 baseline
 142      routed forwarding, functional tests.
 143
 144 #. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC,
 145    P2V2P, NIC2VMchain2NIC, P2V2V2P**
 146
 147    - *eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot* => 2 ports of
 148      Ethernet, IPv4 VXLAN Ethernet, L2 bridge-domain switching to/from two vhost
 149      interfaces and one VM, functional tests.