X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Freport%2Fintroduction%2Ftest_scenarios_overview.rst;h=0d520f029620340263b8e399cc8cb62a72b55a71;hp=aebb0b1900848fdf4f5c73dd996f6d5ac833cfae;hb=bb7e8747d67176367fabadac85bc4e2571a47c93;hpb=0546e1656070b734ec57a4d8f732392bcafee0df diff --git a/docs/report/introduction/test_scenarios_overview.rst b/docs/report/introduction/test_scenarios_overview.rst index aebb0b1900..0d520f0296 100644 --- a/docs/report/introduction/test_scenarios_overview.rst +++ b/docs/report/introduction/test_scenarios_overview.rst @@ -1,5 +1,5 @@ -Test Scenarios Overview -======================= +Test Scenarios +============== FD.io |csit-release| report includes multiple test scenarios of VPP centric applications, topologies and use cases. In addition it also @@ -7,24 +7,22 @@ covers baseline tests of DPDK sample applications. Tests are executed in physical (performance tests) and virtual environments (functional tests). -Following list provides a brief overview of test scenarios covered in -this report: +Brief overview of test scenarios covered in this report: #. **VPP Performance**: VPP performance tests are executed in physical FD.io testbeds, focusing on VPP network data plane performance in NIC-to-NIC switching topologies. Tested across Intel Xeon Haswell - and Skylake servers, range of NICs (10GE, 25GE, 40GE) and multi- - thread/multi-core configurations. VPP application runs in bare-metal + and Skylake servers, ARM, Denverton, range of NICs (10GE, 25GE, 40GE) and + multi-thread/multi-core configurations. VPP application runs in bare-metal host user-mode handling NICs. TRex is used as a traffic generator. #. **VPP Vhostuser Performance with KVM VMs**: VPP VM service switching performance tests using vhostuser virtual interface for - interconnecting multiple Testpmd-in-VM instances. VPP vswitch + interconnecting multiple NF-in-VM instances. VPP vswitch instance runs in bare-metal user-mode handling NICs and connecting - over vhost-user interfaces to VM instances each running DPDK - Testpmd with virtio virtual interfaces. Similarly to VPP - Performance, tests are run across a range of configurations. TRex - is used as a traffic generator. + over vhost-user interfaces to VM instances each running VPP with virtio + virtual interfaces. Similarly to VPP Performance, tests are run across a + range of configurations. TRex is used as a traffic generator. #. **VPP Memif Performance with LXC and Docker Containers**: VPP Container service switching performance tests using memif virtual @@ -36,12 +34,6 @@ this report: run across a range of configurations. TRex is used as a traffic generator. -#. **K8s Container/Pod Topologies Performance**: VPP container - performance tests using memif for interconnecting VPP-in- - Container/Pod instances orchestrated by K8s integrated with `Ligato - `_ for container networking. TRex is - used as a traffic generator. - #. **DPDK Performance**: VPP uses DPDK to drive the NICs and physical interfaces. DPDK performance tests are used as a baseline to profile performance of the DPDK sub-system. Two DPDK applications @@ -56,27 +48,11 @@ this report: cover vNIC-to-vNIC vNIC-to-nestedVM-to-vNIC forwarding topologies. Scapy is used as a traffic generator. -#. **Honeycomb Functional**: Honeycomb functional tests are executed in - virtual FD.io testbeds, focusing on Honeycomb management and - programming functionality of VPP. Tests cover a range of CRUD - operations executed against VPP. - -#. **NSH_SFC Functional**: NSH_SFC functional tests are executed in - virtual FD.io testbeds focusing on VPP nsh-plugin data plane - functionality. Scapy is used as a traffic generator. - -#. **DMM Functional**: DMM functional tests are executed in virtual - FD.io testbeds demonstrating a single server (DUT1) and single - client (DUT2) scenario using DMM framework and Linux kernel TCP/IP - stack. - -All CSIT test results listed in this report are sourced and auto- +All CSIT test data included in this report is auto- generated from :abbr:`RF (Robot Framework)` :file:`output.xml` files -resulting from :abbr:`LF (Linux Foundation)` FD.io Jenkins jobs executed -against |vpp-release| release artifacts. References are provided to the -original FD.io Jenkins job results. Additional references are provided -to the :abbr:`RF (Robot Framework)` result files that got archived in -FD.io Nexus online storage system. +produced by :abbr:`LF (Linux Foundation)` FD.io Jenkins jobs executed +against |vpp-release| artifacts. References are provided to the +original FD.io Jenkins job results and all archived source files. FD.io CSIT system is developed using two main coding platforms: :abbr:`RF (Robot Framework)` and Python2.7. |csit-release| source code for the executed test