1 Test Scenarios Overview
2 =======================
4 FD.io |csit-release| report includes multiple test scenarios of VPP
5 centric applications, topologies and use cases. In addition it also
6 covers baseline tests of DPDK sample applications. Tests are executed in
7 physical (performance tests) and virtual environments (functional
10 Brief overview of test scenarios covered in this report:
12 #. **VPP Performance**: VPP performance tests are executed in physical
13 FD.io testbeds, focusing on VPP network data plane performance in
14 NIC-to-NIC switching topologies. Tested across Intel Xeon Haswell
15 and Skylake servers, range of NICs (10GE, 25GE, 40GE) and multi-
16 thread/multi-core configurations. VPP application runs in bare-metal
17 host user-mode handling NICs. TRex is used as a traffic generator.
19 #. **VPP Vhostuser Performance with KVM VMs**: VPP VM service switching
20 performance tests using vhostuser virtual interface for
21 interconnecting multiple Testpmd-in-VM instances. VPP vswitch
22 instance runs in bare-metal user-mode handling NICs and connecting
23 over vhost-user interfaces to VM instances each running DPDK
24 Testpmd with virtio virtual interfaces. Similarly to VPP
25 Performance, tests are run across a range of configurations. TRex
26 is used as a traffic generator.
28 #. **VPP Memif Performance with LXC and Docker Containers**: VPP
29 Container service switching performance tests using memif virtual
30 interface for interconnecting multiple VPP-in-container instances.
31 VPP vswitch instance runs in bare-metal user-mode handling NICs and
32 connecting over memif (Slave side) interfaces to more instances of
33 VPP running in LXC or in Docker Containers, both with memif
34 interfaces (Master side). Similarly to VPP Performance, tests are
35 run across a range of configurations. TRex is used as a traffic
38 #. **K8s Container/Pod Topologies Performance**: VPP container
39 performance tests using memif for interconnecting VPP-in-
40 Container/Pod instances orchestrated by K8s integrated with `Ligato
41 <https://github.com/ligato>`_ for container networking. TRex is
42 used as a traffic generator.
44 #. **DPDK Performance**: VPP uses DPDK to drive the NICs and physical
45 interfaces. DPDK performance tests are used as a baseline to
46 profile performance of the DPDK sub-system. Two DPDK applications
47 are tested: Testpmd and L3fwd. DPDK tests are executed in the same
48 testing environment as VPP tests. DPDK Testpmd and L3fwd
49 applications run in host user-mode. TRex is used as a traffic
52 #. **VPP Functional**: VPP functional tests are executed in virtual
53 FD.io testbeds, focusing on VPP packet processing functionality,
54 including both network data plane and in-line control plane. Tests
55 cover vNIC-to-vNIC vNIC-to-nestedVM-to-vNIC forwarding topologies.
56 Scapy is used as a traffic generator.
58 #. **Honeycomb Functional**: Honeycomb functional tests are executed in
59 virtual FD.io testbeds, focusing on Honeycomb management and
60 programming functionality of VPP. Tests cover a range of CRUD
61 operations executed against VPP.
63 #. **NSH_SFC Functional**: NSH_SFC functional tests are executed in
64 virtual FD.io testbeds focusing on VPP nsh-plugin data plane
65 functionality. Scapy is used as a traffic generator.
67 #. **DMM Functional**: DMM functional tests are executed in virtual
68 FD.io testbeds demonstrating a single server (DUT1) and single
69 client (DUT2) scenario using DMM framework and Linux kernel TCP/IP
72 All CSIT test data included in this report is auto-
73 generated from :abbr:`RF (Robot Framework)` :file:`output.xml` files
74 produced by :abbr:`LF (Linux Foundation)` FD.io Jenkins jobs executed
75 against |vpp-release| artifacts. References are provided to the
76 original FD.io Jenkins job results and all archived source files.
78 FD.io CSIT system is developed using two main coding platforms: :abbr:`RF (Robot
79 Framework)` and Python2.7. |csit-release| source code for the executed test
80 suites is available in CSIT branch |release| in the directory
81 :file:`./tests/<name_of_the_test_suite>`. A local copy of CSIT source code
82 can be obtained by cloning CSIT git repository - :command:`git clone
83 https://gerrit.fd.io/r/csit`.