report 1904: updated methodology (nfv density, startup.conf) and vpp perf rls notes

Change-Id: If6d24511d5d29dcc25c21437364e0da15af3cca1
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>

diff --git a/docs/report/introduction/methodology_nfv_service_density.rst b/docs/report/introduction/methodology_nfv_service_density.rst
index 23bdac9..b09c1be 100644 (file)
--- a/docs/report/introduction/methodology_nfv_service_density.rst
+++ b/docs/report/introduction/methodology_nfv_service_density.rst
@@ -19,15 +19,13 @@ different service density setups by varying two parameters:
 - Number of service instances (e.g. 1,2,4..10).
 - Number of NFs per service instance (e.g. 1,2,4..10).
 
 - Number of service instances (e.g. 1,2,4..10).
 - Number of NFs per service instance (e.g. 1,2,4..10).
 

-The initial implementation of NFV service density tests in
-|csit-release| is using two NF applications:
+Implementation of NFV service density tests in |csit-release| is using two NF
+applications:

 
 - VNF: VPP of the same version as vswitch running in KVM VM, configured with /8
   IPv4 prefix routing.
 - CNF: VPP of the same version as vswitch running in Docker Container,
 
 - VNF: VPP of the same version as vswitch running in KVM VM, configured with /8
   IPv4 prefix routing.
 - CNF: VPP of the same version as vswitch running in Docker Container,

-  configured with /8 IPv4 prefix routing. VPP got chosen as a fast IPv4 NF
-  application that supports required memif interface (L3fwd does not). This is
-  similar to all other Container tests in CSIT that use VPP.
+  configured with /8 IPv4 prefix routing.

 
 Tests are designed such that in all tested cases VPP vswitch is the most
 stressed application, as for each flow vswitch is processing each packet
 
 Tests are designed such that in all tested cases VPP vswitch is the most
 stressed application, as for each flow vswitch is processing each packet


@@ -103,9 +101,8 @@ physical core mapping ratios:
 
     - (main:core) = (2:1) => 2mt1c - 2 Main Threads on 1 Core, 1 Thread
       per NF, core shared between two NFs.
 
     - (main:core) = (2:1) => 2mt1c - 2 Main Threads on 1 Core, 1 Thread
       per NF, core shared between two NFs.

-    - (data:core) = (2:1) => 1dt1c - 1 Data-plane Threads on 1 Core per
-      NF.
-
+    - (data:core) = (2:1) => 2dt1c - 2 Data-plane Threads on 1 Core, 1
+      Thread per NF, core shared between two NFs.

 
 Maximum tested service densities are limited by a number of physical
 cores per NUMA. |csit-release| allocates cores within NUMA0. Support for
 
 Maximum tested service densities are limited by a number of physical
 cores per NUMA. |csit-release| allocates cores within NUMA0. Support for

diff --git a/docs/report/introduction/methodology_vpp_startup_settings.rst b/docs/report/introduction/methodology_vpp_startup_settings.rst
index 81c5e4e..e3e8d29 100644 (file)
--- a/docs/report/introduction/methodology_vpp_startup_settings.rst
+++ b/docs/report/introduction/methodology_vpp_startup_settings.rst
@@ -1,29 +1,32 @@
 VPP Startup Settings
 --------------------
 
 VPP Startup Settings
 --------------------
 

-CSIT code manipulates a number of VPP settings in startup.conf for optimized
-performance. List of common settings applied to all tests and test
-dependent settings follows.
+CSIT code manipulates a number of VPP settings in startup.conf for
+optimized performance. List of common settings applied to all tests and
+test dependent settings follows.

 
 

-See `VPP startup.conf`_
-for a complete set and description of listed settings.
+See `VPP startup.conf`_ for a complete set and description of listed
+settings.

 
 Common Settings
 ~~~~~~~~~~~~~~~
 
 
 Common Settings
 ~~~~~~~~~~~~~~~
 

-List of vpp startup.conf settings applied to all tests:
+List of VPP startup.conf settings applied to all tests:

 
 #. heap-size <value> - set separately for ip4, ip6, stats, main
    depending on scale tested.
 
 #. heap-size <value> - set separately for ip4, ip6, stats, main
    depending on scale tested.

-#. no-tx-checksum-offload - disables UDP / TCP TX checksum offload in DPDK.
-   Typically needed for use faster vector PMDs (together with
+#. no-tx-checksum-offload - disables UDP / TCP TX checksum offload in
+   DPDK. Typically needed for use faster vector PMDs (together with

    no-multi-seg).
    no-multi-seg).

-#. buffers-per-numa <value> - increases number of buffers allocated, needed
-   in scenarios with large number of interfaces and worker threads.
-   Value is per CPU socket. Default is 16384. CSIT is setting statically
-   107520 buffers per CPU thread (215040 if HTT is enabled). This value is also
-   maximum possible amount limited by number of memory mappings in DPDK
-   libraries for 2MB Hugepages used in CSIT.
+#. buffers-per-numa <value> - sets a number of memory buffers allocated
+   to VPP per CPU socket. VPP default is 16384. Needs to be increased for
+   scenarios with large number of interfaces and worker threads. To
+   accommodate for scale tests, CSIT is setting it to the maximum possible
+   value corresponding to the limit of DPDK memory mappings (currently
+   256). For Xeon Skylake platforms configured with 2MB hugepages and VPP
+   data-size and buffer-size defaults (2048B and 2496B respectively), this
+   results in value of 215040 (256 * 840 = 215040, 840 * 2496B buffers fit
+   in 2MB hugepage ). For Xeon Haswell nodes value of 107520 is used.

 
 Per Test Settings
 ~~~~~~~~~~~~~~~~~
 
 Per Test Settings
 ~~~~~~~~~~~~~~~~~

diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 70092ad..a96a4f4 100644 (file)
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -6,32 +6,33 @@ Changes in |csit-release|
 
 #. VPP PERFORMANCE TESTS
 
 
 #. VPP PERFORMANCE TESTS
 

-   - **Service density 2n-skx tests**: Added higher density tests with dtc=0.5
-     (2 NF each with 1 DT per physical core) with VPP as a VNF payload.
+   - **Service density 2n-skx tests**: Added higher NF density tests with two
+     NFs' data-plane threads sharing a physical core. VPP IPv4 routing is now 
+     used as a VNF payload similar to CNF tests.

 
 

-   - **Experimental Soak Tests**: Added performamce soak tests framework
+   - **Soak Tests**: Optimized performamce soak tests framework

      code for extended time duration tests and throughput discovery
      at given PLR and at give total test time e.g. minutes, hours,
      code for extended time duration tests and throughput discovery
      at given PLR and at give total test time e.g. minutes, hours,

-     days, weeks, months, years. See updated
+     days, weeks. See updated

      :ref:`test_methodology` section for more details.
 
 #. TEST FRAMEWORK
 
      :ref:`test_methodology` section for more details.
 
 #. TEST FRAMEWORK
 

-   - **Qemu code refactor**: Complete code refactor of the key components
+   - **Qemu code refactor**: Complete code refactor of the key components of

      QemuUtil.py and QemuManager.py (L1 and L2 KW counterparts). Added
      QemuUtil.py and QemuManager.py (L1 and L2 KW counterparts). Added

-     implementation of kernel-image-kvm based VM alongisde the previously used
+     implementation of kernel-image-kvm based VM replacing the previously used

      NestedVM images. Added ability to run VPP as a payload in VNF.
 
      NestedVM images. Added ability to run VPP as a payload in VNF.
 

-   - **CSIT PAPI Support**: Continue converting existing VAT L1 keywords to
-     PAPI L1 KWs in CSIT using VPP Python bindings. Required for migrating away
-     from VAT. Redesign of key components of PAPI Executor and PAPI history.
+   - **CSIT PAPI Support**: Continued conversion of CSIT VAT L1 keywords to 
+     PAPI L1 KWs in CSIT using VPP Python bindings. Redesign of key components
+     of PAPI Executor and PAPI history.

 
    - **General Code Housekeeping**: Ongoing RF keywords optimizations,
      removal of redundant RF keywords.
 
 
    - **General Code Housekeeping**: Ongoing RF keywords optimizations,
      removal of redundant RF keywords.
 

-   - **Test suite generator**: Extend the test suite generator for ability to
-     generate NIC permutation and search algorithm permutations from base
-     suites.
+   - **Test suite generator**: Added capability to generate suites for
+     different NIC models as well as throughput search algorithm types. Uses
+     base tests suites as source.

 
    - **TOX verification**: Added verifications for test suite generator.
 
 
    - **TOX verification**: Added verifications for test suite generator.
 


@@ -67,6 +68,6 @@ List of known issues in |csit-release| for VPP performance tests:
 | 1  | `CSIT-570                               | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue.                         |
 |    | <https://jira.fd.io/browse/CSIT-570>`_  | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds.                           |
 +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
 | 1  | `CSIT-570                               | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue.                         |
 |    | <https://jira.fd.io/browse/CSIT-570>`_  | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds.                           |
 +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+

-| 2  | `CSIT-????                              | IPSecHW interface AES-GBC tests failing due to traffic not passing.                                                             |
+| 2  | `CSIT-????                              | IPSecHW interface AES-128-CBC tests failing due to traffic not passing.                                                             |

 |    | <https://jira.fd.io/browse/CSIT-????>`_ |                                                                                                                                 |
 +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
 |    | <https://jira.fd.io/browse/CSIT-????>`_ |                                                                                                                                 |
 +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+