4 All CSIT test results listed in this report are sourced and auto-generated
5 from :file:`output.xml` :abbr:`RF (Robot Framework)` files resulting from
6 :abbr:`LF (Linux Foundation)` FD.io Jenkins jobs execution against |vpp-release|
7 release artifacts. References are provided to the original :abbr:`LF (Linux
8 Foundation)` FD.io Jenkins job results. However, as :abbr:`LF (Linux
9 Foundation)` FD.io Jenkins infrastructure does not automatically archive all jobs
10 (history record is provided for the last 30 days or 40 jobs only), additional
11 references are provided to the :abbr:`RF (Robot Framework)` result files that
12 got archived in FD.io nexus online storage system.
14 FD.io CSIT project currently covers multiple FD.io system and sub-system
15 testing areas and this is reflected in this report, where each testing area
16 is listed separately, as follows:
18 #. **VPP Performance Tests** - VPP performance tests are executed in physical
19 FD.io testbeds, focusing on VPP network data plane performance at this stage,
20 both for Phy-to-Phy (NIC-to-NIC) and Phy-to-VM-to-Phy (NIC-to-VM-to-NIC)
21 forwarding topologies. Tested across a range of NICs, 10GE and 40GE
22 interfaces, range of multi-thread and multi-core configurations. VPP
23 application runs in host user-mode. TRex is used as a traffic generator.
25 #. **Container memif connections** - VPP memif virtual interface (shared memory
26 interface) tests to interconnect VPP instances. VPP vswitch instance runs in
27 bare-metal user-mode handling Intel x520 NIC 10GbE interfaces and connecting
28 over memif (Master side) virtual interfaces to more instances of VPP running
29 in LXC or in Docker Containers, both with memif virtual interfaces (Slave
30 side). Tested across a range of NICs, 10GE and 40GE interfaces, range of
31 multi-thread and multi-core configurations. VPP application runs in host
32 user-mode. TRex is used as a traffic generator.
34 #. **Container Orchestrated Performance Tests** - CSIT |release| introduced new
35 tests of Container topologies connected over the memif virtual interface
36 (shared memory interface). For these tests VPP vswitch instance runs in a
37 Docker Container handling Intel x520 NIC 10GbE interfaces and connecting over
38 memif (Master side) virtual interfaces to more instances of VPP running in
39 Docker Containers with memif virtual interfaces (Slave side). Tested across a
40 range of multi-thread and multi-core configurations. VPP application runs in
41 host user-mode. TRex is used as a traffic generator.
43 #. **DPDK Performance Tests** - VPP is using DPDK code to control and drive
44 the NICs and physical interfaces. Testpmd tests are used as a baseline to
45 profile the DPDK sub-system of VPP. DPDK performance tests executed in
46 physical FD.io testbeds, focusing on Testpmd/L3FWD data plane performance for
47 Phy-to-Phy (NIC-to-NIC). Tests cover a range of NICs, 10GE and 40GE
48 interfaces, range of multi-thread and multi-core configurations.
49 Testpmd/L3FWD application runs in host user-mode. TRex is used as a traffic
52 #. **VPP Functional Tests** - VPP functional tests are executed in virtual
53 FD.io testbeds focusing on VPP packet processing functionality, including
54 network data plane and in -line control plane. Tests cover vNIC-to-vNIC
55 vNIC-to-VM-to-vNIC forwarding topologies. Scapy is used as a traffic
58 #. **Honeycomb Functional Tests** - Honeycomb functional tests are executed in
59 virtual FD.io testbeds, focusing on Honeycomb management and programming
60 functionality of VPP. Tests cover a range of CRUD operations executed
63 #. **Honeycomb Performance Tests** - Honeycomb performance tests are executed in
64 physical FD.io testbeds, focusing on the performance of Honeycomb management
65 and programming functionality of VPP. Tests cover a range of CRUD operations
68 #. **NSH_SFC Functional Tests** - NSH_SFC functional tests are executed in
69 virtual FD.io testbeds focusing on NSH_SFC of VPP. Tests cover a range of
70 CRUD operations executed against VPP.
72 In addition to above, CSIT |release| report does also include VPP unit test
73 results. VPP unit tests are developed within the FD.io VPP project and as they
74 complement CSIT system functional tests, they are provided mainly as a reference
75 and to provide a more complete view of automated testing executed against
78 FD.io CSIT system is developed using two main coding platforms :abbr:`RF (Robot
79 Framework)` and Python. CSIT |release| source code for the executed test
80 suites is available in CSIT branch |release| in the directory
81 :file:`./tests/<name_of_the_test_suite>`. A local copy of CSIT source code
82 can be obtained by cloning CSIT git repository - :command:`git clone
83 https://gerrit.fd.io/r/csit`. The CSIT testing virtual environment can be run
84 on a local computer workstation (laptop, server) using Vagrant by following
85 the instructions in `CSIT tutorials
86 <https://wiki.fd.io/view/CSIT#Tutorials>`_.