4 All CSIT test results listed in this report are sourced and auto-generated
5 from :file:`output.xml` :abbr:`RF (Robot Framework)` files resulting from
6 :abbr:`LF (Linux Foundation)` FD.io Jenkins jobs execution against |vpp-release|
7 release artifacts. References are provided to the original :abbr:`LF (Linux
8 Foundation)` FD.io Jenkins job results. Additional references are provided to
9 the :abbr:`RF (Robot Framework)` result files that got archived in FD.io nexus
10 online storage system.
12 FD.io CSIT project currently covers multiple FD.io system and sub-system
13 testing areas and this is reflected in this report, where each testing area
14 is listed separately, as follows:
16 #. **VPP - Performance** - VPP benchmarking tests are executed in physical
17 FD.io testbeds, focusing on VPP network data plane performance at this stage,
18 both for Phy-to-Phy (NIC-to-NIC) and Phy-to-VM-to-Phy (NIC-to-VM-to-NIC)
19 forwarding topologies. Tested across a range of NICs, 10GE and 40GE
20 interfaces, range of multi-thread and multi-core configurations. VPP
21 application runs in host user-mode. TRex is used as a traffic generator.
23 #. **LXC and Docker Containers VPP memif - Performance** - VPP memif
24 virtual interface tests interconnect multiple VPP instances running in
25 containers. VPP vswitch instance runs in bare-metal user-mode
26 handling Intel x520 NIC 10GbE, Intel x710 NIC 10GbE, Intel xl710 NIC 40GbE
27 interfaces and connecting over memif (Slave side) virtual interfaces to more
28 instances of VPP running in LXC or in Docker Containers, both with memif
29 virtual interfaces (Master side). Tested across a range of multi-thread and
30 multi-core configurations. TRex is used as a traffic generator.
32 #. **Container Topologies Orchestrated by K8s - Performance** - CSIT Container
33 topologies connected over the memif virtual interface (shared memory
34 interface). For these tests VPP vswitch instance runs in a Docker Container
35 handling Intel x520 NIC 10GbE, Intel x710 NIC 10GbE interfaces and connecting
36 over memif virtual interfaces to more instances of VPP running in Docker
37 Containers with memif virtual interfaces. All containers are
38 orchestrated by Kubernetes, with `Ligato <https://github.com/ligato>`_ for
39 container networking. TRex is used as a traffic generator.
41 #. **DPDK Performance** - VPP is using DPDK code to control and drive
42 the NICs and physical interfaces. Tests are used as a baseline to
43 profile performance of the DPDK sub-system. DPDK tests are executed in
44 physical FD.io testbeds, focusing on Testpmd/L3FWD data plane performance for
45 Phy-to-Phy (NIC-to-NIC). Tests cover a range of NICs, 10GE and 40GE
46 interfaces, range of multi-thread and multi-core configurations.
47 Testpmd/L3FWD application runs in host user-mode. TRex is used as a traffic
50 #. **VPP Functional** - VPP functional tests are executed in virtual
51 FD.io testbeds focusing on VPP packet processing functionality, including
52 network data plane and in -line control plane. Tests cover vNIC-to-vNIC
53 vNIC-to-VM-to-vNIC forwarding topologies. Scapy is used as a traffic
56 #. **Honeycomb Functional** - Honeycomb functional tests are executed in
57 virtual FD.io testbeds, focusing on Honeycomb management and programming
58 functionality of VPP. Tests cover a range of CRUD operations executed
61 #. **NSH_SFC Functional** - NSH_SFC functional tests are executed in
62 virtual FD.io testbeds focusing on NSH_SFC of VPP. Tests cover a range of
63 CRUD operations executed against VPP.
65 In addition to above, CSIT |release| report does also include VPP unit test
66 results. VPP unit tests are developed within the FD.io VPP project and as they
67 complement CSIT system functional tests, they are provided mainly as a reference
68 and to provide a more complete view of automated testing executed against
71 FD.io CSIT system is developed using two main coding platforms :abbr:`RF (Robot
72 Framework)` and Python2.7. CSIT |release| source code for the executed test
73 suites is available in CSIT branch |release| in the directory
74 :file:`./tests/<name_of_the_test_suite>`. A local copy of CSIT source code
75 can be obtained by cloning CSIT git repository - :command:`git clone
76 https://gerrit.fd.io/r/csit`. The CSIT testing virtual environment can be run
77 on a local computer workstation (laptop, server) using Vagrant by following
78 the instructions in `CSIT tutorials
79 <https://wiki.fd.io/view/CSIT#Tutorials>`_.