QEMU is used for VPP-VM testing enviroment. By default, standard QEMU version
preinstalled from OS repositories is used on VIRL/vpp_device functional testing
-(qemu-2.11.x for Ubuntu 18.04, qemu-2.5.0 for Ubuntu 16.04). For perfomance
-testing QEMU is downloaded from `project homepage <qemu.org>`_ and compiled
-during testing. This allows framework to easily inject QEMU patches in case of
-need. In QEMU version <2.8 we used it for increasing QEMU virtion queue size.
+(qemu-2.11.x for Ubuntu 18.04, qemu-2.5.0 for Ubuntu 16.04).
In CSIT setup DUTs have small VM image `/var/lib/vm/vhost-nested.img`. QEMU
binary can be adjusted in global settings. VM image must have installed at least
qemu-guest-agent, sshd, bridge-utils, VirtIO support and Testpmd/L3fwd
The initial implementation of NFV service density tests in
|csit-release| is using two NF applications:
-- VNF: DPDK L3fwd running in KVM VM, configured with /8 IPv4 prefix
- routing. L3fwd got chosen as a lightweight fast IPv4 VNF application,
- and follows CSIT approach of using DPDK sample applications in VMs for
- performance testing.
-- CNF: VPP running in Docker Container, configured with /24 IPv4 prefix
- routing. VPP got chosen as a fast IPv4 NF application that supports
- required memif interface (L3fwd does not). This is similar to all
- other Container tests in CSIT that use VPP.
+- VNF: VPP of the same version as vswitch running in KVM VM, configured with /8
+ IPv4 prefix routing.
+- CNF: VPP of the same version as vswitch running in Docker Container,
+ configured with /8 IPv4 prefix routing. VPP got chosen as a fast IPv4 NF
+ application that supports required memif interface (L3fwd does not). This is
+ similar to all other Container tests in CSIT that use VPP.
Tests are designed such that in all tested cases VPP vswitch is the most
stressed application, as for each flow vswitch is processing each packet
- Data-plane on single core
+ - (main:core) = (1:1) => 1mt1c - 1 main thread on 1 core.
- (data:core) = (1:1) => 2dt1c - 2 Data-plane Threads on 1 Core.
- - (main:core) = (1:1) => 1mt1c - 1 Main Thread on 1 Core.
- Data-plane on two cores
- - (data:core) = (1:2) => 4dt2c - 4 Data-plane Threads on 2 Cores.
- (main:core) = (1:1) => 1mt1c - 1 Main Thread on 1 Core.
+ - (data:core) = (1:2) => 4dt2c - 4 Data-plane Threads on 2 Cores.
- VNF and CNF
- Data-plane on single core
+ - (main:core) = (2:1) => 2mt1c - 2 Main Threads on 1 Core, 1 Thread
+ per NF, core shared between two NFs.
- (data:core) = (1:1) => 2dt1c - 2 Data-plane Threads on 1 Core per
NF.
+
+ - Data-plane on single logical core (Two NFs per physical core)
+
- (main:core) = (2:1) => 2mt1c - 2 Main Threads on 1 Core, 1 Thread
per NF, core shared between two NFs.
+ - (data:core) = (2:1) => 1dt1c - 1 Data-plane Threads on 1 Core per
+ NF.
+
Maximum tested service densities are limited by a number of physical
cores per NUMA. |csit-release| allocates cores within NUMA0. Support for
#. no-tx-checksum-offload - disables UDP / TCP TX checksum offload in DPDK.
Typically needed for use faster vector PMDs (together with
no-multi-seg).
-#. socket-mem <value>,<value> - memory per numa. (Not required anymore
- due to VPP code changes, will be removed in CSIT-19.04.)
+#. buffers-per-numa <value> - increases number of buffers allocated, needed
+ in scenarios with large number of interfaces and worker threads.
+ Value is per CPU socket. Default is 16384. CSIT is setting statically
+ 107520 buffers per CPU thread (215040 if HTT is enabled). This value is also
+ maximum possible amount limited by number of memory mappings in DPDK
+ libraries for 2MB Hugepages used in CSIT.
Per Test Settings
~~~~~~~~~~~~~~~~~
test configuration.
#. num-rx-queues <value> - depends on a number of VPP threads and NIC
interfaces.
-#. num-rx-desc/num-tx-desc - number of rx/tx descriptors for specific
- NICs, incl. xl710, x710, xxv710.
-#. num-mbufs <value> - increases number of buffers allocated, needed
- only in scenarios with large number of interfaces and worker threads.
- Value is per CPU socket. Default is 16384.
#. no-multi-seg - disables multi-segment buffers in DPDK, improves
packet throughput, but disables Jumbo MTU support. Disabled for all
tests apart from the ones that require Jumbo 9000B frame support.
#. VPP PERFORMANCE TESTS
- - **Service density 2n-skx tests**: Network Function Virtualization (NFV)
- service density tests focus on measuring total per server throughput at
- varied NFV service *packing* densities with vswitch providing host
- dataplane. The goal is to compare and contrast performance of a shared
- vswitch for different network topologies and virtualization technologies,
- and their impact on vswitch performance and efficiency in a range of NFV
- service configurations.
+ - **Service density 2n-skx tests**: Added higher density tests with dtc=0.5
+ (2 NF each with 1 DT per physical core) with VPP as a VNF payload.
- **Experimental Soak Tests**: Added performamce soak tests framework
code for extended time duration tests and throughput discovery
#. TEST FRAMEWORK
- - **Container code optimizations**: Optimized container library allows to
- run containre_memif tests faster.
+ - **Qemu code refactor**: Complete code refactor of the key components
+ QemuUtil.py and QemuManager.py (L1 and L2 KW counterparts). Added
+ implementation of kernel-image-kvm based VM alongisde the previously used
+ NestedVM images. Added ability to run VPP as a payload in VNF.
- **CSIT PAPI Support**: Continue converting existing VAT L1 keywords to
PAPI L1 KWs in CSIT using VPP Python bindings. Required for migrating away
- from VAT.
+ from VAT. Redesign of key components of PAPI Executor and PAPI history.
- **General Code Housekeeping**: Ongoing RF keywords optimizations,
removal of redundant RF keywords.
+ - **Test suite generator**: Extend the test suite generator for ability to
+ generate NIC permutation and search algorithm permutations from base
+ suites.
+
+ - **TOX verification**: Added verifications for test suite generator.
+
#. PRESENTATION AND ANALYTICS LAYER
- **Graphs Layout Improvements**: Improved performance graphs layout
| 1 | `CSIT-570 | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue. |
| | <https://jira.fd.io/browse/CSIT-570>`_ | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds. |
+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 2 | `VPP-1563 | AVF L2patch tests are failing for all packet size and core combination. Reason: null-node blackholed packets in show error. |
-| | <https://jira.fd.io/browse/VPP-1563>`_ | |
-+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 3 | `CSIT-1234 | VPP IPSecHW/IPSecsW scale interface mode, low NDR and PDR 64B throughput in 3n-hsw testbeds, in CSIT-19.01 vs. CSIT-18.10. |
-| | <https://jira.fd.io/browse/CSIT-1234>`_ | |
-+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 4 | `CSIT-1431 | AVF 4 cores tests are sporadically failing. Under investigation. |
-| | <https://jira.fd.io/browse/CSIT-1431>`_ | |
-+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 5 | `CSIT-1465 | 4c VPP VM vhost tests failing on 3n-skx. Under investigation. |
-| | <https://jira.fd.io/browse/CSIT-1465>`_ | |
-+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 6 | `CSIT-1466 | IPSecHW scale tests failing due to VPP reset. Fixed in one of subsequent VPP patches. Confirmed by running tests with VPP |
-| | <https://jira.fd.io/browse/CSIT-1466>`_ | build 19.01.1-8~g50a392f~b56. |
+| 2 | `CSIT-???? | IPSecHW interface AES-GBC tests failing due to traffic not passing. |
+| | <https://jira.fd.io/browse/CSIT-????>`_ | |
+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
.. _CSIT DMM Functional Tests Documentation: https://docs.fd.io/csit/{release}/doc/tests.dmm.func.html
.. _CSIT VPP Device Tests Documentation: https://docs.fd.io/csit/{release}/doc/tests.vpp.device.html
.. _VPP test framework documentation: https://docs.fd.io/vpp/{vpprelease}/vpp_make_test/html/
-.. _FD.io CSIT Testbeds - Xeon Haswell: https://git.fd.io/csit/tree/docs/lab/Testbeds_Xeon_Hsw_VIRL.md?h={release}
-.. _FD.io CSIT testbeds - Xeon Skylake, Arm, Atom: https://git.fd.io/csit/tree/docs/lab/Testbeds_Xeon_Skx_Arm_Atom.md?h={release}
+.. _FD.io CSIT Testbeds - Xeon Haswell: https://git.fd.io/csit/tree/docs/lab/testbeds_ucs_hsw_hw_bios_cfg.md?h={release}
+.. _FD.io CSIT testbeds - Xeon Skylake, Arm, Atom: https://git.fd.io/csit/tree/docs/lab/testbeds_sm_skx_hw_bios_cfg.md?h={release}
.. _Ansible inventory - hosts: https://git.fd.io/csit/tree/resources/tools/testbed-setup/ansible/inventories/lf_inventory/host_vars?h={release}
.. _FD.io test executor dpdk performance job 3n-hsw: https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-verify-{srelease}-3n-hsw
.. _FD.io test executor dpdk performance job 3n-skx: https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-verify-{srelease}-3n-skx
Code Review / csit.git / commitdiff