From: Maciek Konstantynowicz Date: Thu, 2 May 2019 18:20:38 +0000 (+0100) Subject: report 1904: updated methodology (nfv density, startup.conf) and vpp perf rls notes X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=commitdiff_plain;h=a8c3f441fa595311f60bcc634a2720f204ced733 report 1904: updated methodology (nfv density, startup.conf) and vpp perf rls notes Change-Id: If6d24511d5d29dcc25c21437364e0da15af3cca1 Signed-off-by: Maciek Konstantynowicz --- diff --git a/docs/report/introduction/methodology_nfv_service_density.rst b/docs/report/introduction/methodology_nfv_service_density.rst index 23bdac9d7c..b09c1be629 100644 --- a/docs/report/introduction/methodology_nfv_service_density.rst +++ b/docs/report/introduction/methodology_nfv_service_density.rst @@ -19,15 +19,13 @@ different service density setups by varying two parameters: - Number of service instances (e.g. 1,2,4..10). - Number of NFs per service instance (e.g. 1,2,4..10). -The initial implementation of NFV service density tests in -|csit-release| is using two NF applications: +Implementation of NFV service density tests in |csit-release| is using two NF +applications: - VNF: VPP of the same version as vswitch running in KVM VM, configured with /8 IPv4 prefix routing. - CNF: VPP of the same version as vswitch running in Docker Container, - configured with /8 IPv4 prefix routing. VPP got chosen as a fast IPv4 NF - application that supports required memif interface (L3fwd does not). This is - similar to all other Container tests in CSIT that use VPP. + configured with /8 IPv4 prefix routing. Tests are designed such that in all tested cases VPP vswitch is the most stressed application, as for each flow vswitch is processing each packet @@ -103,9 +101,8 @@ physical core mapping ratios: - (main:core) = (2:1) => 2mt1c - 2 Main Threads on 1 Core, 1 Thread per NF, core shared between two NFs. - - (data:core) = (2:1) => 1dt1c - 1 Data-plane Threads on 1 Core per - NF. - + - (data:core) = (2:1) => 2dt1c - 2 Data-plane Threads on 1 Core, 1 + Thread per NF, core shared between two NFs. Maximum tested service densities are limited by a number of physical cores per NUMA. |csit-release| allocates cores within NUMA0. Support for diff --git a/docs/report/introduction/methodology_vpp_startup_settings.rst b/docs/report/introduction/methodology_vpp_startup_settings.rst index 81c5e4e73e..e3e8d29b23 100644 --- a/docs/report/introduction/methodology_vpp_startup_settings.rst +++ b/docs/report/introduction/methodology_vpp_startup_settings.rst @@ -1,29 +1,32 @@ VPP Startup Settings -------------------- -CSIT code manipulates a number of VPP settings in startup.conf for optimized -performance. List of common settings applied to all tests and test -dependent settings follows. +CSIT code manipulates a number of VPP settings in startup.conf for +optimized performance. List of common settings applied to all tests and +test dependent settings follows. -See `VPP startup.conf`_ -for a complete set and description of listed settings. +See `VPP startup.conf`_ for a complete set and description of listed +settings. Common Settings ~~~~~~~~~~~~~~~ -List of vpp startup.conf settings applied to all tests: +List of VPP startup.conf settings applied to all tests: #. heap-size - set separately for ip4, ip6, stats, main depending on scale tested. -#. no-tx-checksum-offload - disables UDP / TCP TX checksum offload in DPDK. - Typically needed for use faster vector PMDs (together with +#. no-tx-checksum-offload - disables UDP / TCP TX checksum offload in + DPDK. Typically needed for use faster vector PMDs (together with no-multi-seg). -#. buffers-per-numa - increases number of buffers allocated, needed - in scenarios with large number of interfaces and worker threads. - Value is per CPU socket. Default is 16384. CSIT is setting statically - 107520 buffers per CPU thread (215040 if HTT is enabled). This value is also - maximum possible amount limited by number of memory mappings in DPDK - libraries for 2MB Hugepages used in CSIT. +#. buffers-per-numa - sets a number of memory buffers allocated + to VPP per CPU socket. VPP default is 16384. Needs to be increased for + scenarios with large number of interfaces and worker threads. To + accommodate for scale tests, CSIT is setting it to the maximum possible + value corresponding to the limit of DPDK memory mappings (currently + 256). For Xeon Skylake platforms configured with 2MB hugepages and VPP + data-size and buffer-size defaults (2048B and 2496B respectively), this + results in value of 215040 (256 * 840 = 215040, 840 * 2496B buffers fit + in 2MB hugepage ). For Xeon Haswell nodes value of 107520 is used. Per Test Settings ~~~~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 70092ad098..a96a4f4287 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -6,32 +6,33 @@ Changes in |csit-release| #. VPP PERFORMANCE TESTS - - **Service density 2n-skx tests**: Added higher density tests with dtc=0.5 - (2 NF each with 1 DT per physical core) with VPP as a VNF payload. + - **Service density 2n-skx tests**: Added higher NF density tests with two + NFs' data-plane threads sharing a physical core. VPP IPv4 routing is now + used as a VNF payload similar to CNF tests. - - **Experimental Soak Tests**: Added performamce soak tests framework + - **Soak Tests**: Optimized performamce soak tests framework code for extended time duration tests and throughput discovery at given PLR and at give total test time e.g. minutes, hours, - days, weeks, months, years. See updated + days, weeks. See updated :ref:`test_methodology` section for more details. #. TEST FRAMEWORK - - **Qemu code refactor**: Complete code refactor of the key components + - **Qemu code refactor**: Complete code refactor of the key components of QemuUtil.py and QemuManager.py (L1 and L2 KW counterparts). Added - implementation of kernel-image-kvm based VM alongisde the previously used + implementation of kernel-image-kvm based VM replacing the previously used NestedVM images. Added ability to run VPP as a payload in VNF. - - **CSIT PAPI Support**: Continue converting existing VAT L1 keywords to - PAPI L1 KWs in CSIT using VPP Python bindings. Required for migrating away - from VAT. Redesign of key components of PAPI Executor and PAPI history. + - **CSIT PAPI Support**: Continued conversion of CSIT VAT L1 keywords to + PAPI L1 KWs in CSIT using VPP Python bindings. Redesign of key components + of PAPI Executor and PAPI history. - **General Code Housekeeping**: Ongoing RF keywords optimizations, removal of redundant RF keywords. - - **Test suite generator**: Extend the test suite generator for ability to - generate NIC permutation and search algorithm permutations from base - suites. + - **Test suite generator**: Added capability to generate suites for + different NIC models as well as throughput search algorithm types. Uses + base tests suites as source. - **TOX verification**: Added verifications for test suite generator. @@ -67,6 +68,6 @@ List of known issues in |csit-release| for VPP performance tests: | 1 | `CSIT-570 | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue. | | | `_ | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds. | +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+ -| 2 | `CSIT-???? | IPSecHW interface AES-GBC tests failing due to traffic not passing. | +| 2 | `CSIT-???? | IPSecHW interface AES-128-CBC tests failing due to traffic not passing. | | | `_ | | +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+