Code Review / csit.git / blobdiff
? search:
summary | shortlog | log | commit | commitdiff | review | tree
raw | inline | side by side
DOC: 1908rls release notes for VPP Hoststack TCP performance tests.
[csit.git] / docs / report / vpp_performance_tests / csit_release_notes.rst
diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 324eb6d..8534aa1 100644 (file)
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -6,35 +6,37 @@ Changes in |csit-release|
 
 #. VPP PERFORMANCE TESTS
 
 
 #. VPP PERFORMANCE TESTS
 
-   - **Service density 2n-skx tests**: Added higher NF density tests with two
-     NFs' data-plane threads sharing a physical core. VPP IPv4 routing is now 
-     used as a VNF payload similar to CNF tests.
+   - **Service density 2n-skx tests**: Added higher NF density tests with
+     802.1q (vlan) and 802.1ad (vxlan) encapsulation from Traffic Generator.
 
 
-   - **Soak Tests**: Optimized performamce soak tests framework
-     code for extended time duration tests and throughput discovery
-     at given PLR and at give total test time e.g. minutes, hours,
-     days, weeks. See updated
-     :ref:`test_methodology` section for more details.
+   - **GBP tests**: Added GBP routing test cases with 802.1q (vlan) external
+     traffic.
 
 
-#. TEST FRAMEWORK
+   - **AVF IPv4 scale tests**: Increased coverage of AVF IPv4 base and scale
+     test cases.
+
+   - **2n-skx tests**: Increased coverage of selected (COP, iACL, Policer)
+     test cases.
+
+   - **IPsec scale tests**: Added IPsec interface mode scale tests with
+     1, 40, 400, 1000, 5000, 10000, 20000, 40000, 60000 tunnels. Removed DPDK
+     backend dependency.
+
+   - **Hoststack TCP tests**: Added Hoststack TCP performance tests
+     using WRK to the HTTP static server plugin measuring connections
+     per second and requests per second.
 
 
-   - **Qemu code refactor**: Complete code refactor of the key components of
-     QemuUtil.py and QemuManager.py (L1 and L2 KW counterparts). Added
-     implementation of kernel-image-kvm based VM replacing the previously used
-     NestedVM images. Added ability to run VPP as a payload in VNF.
+#. TEST FRAMEWORK
 
 
-   - **CSIT PAPI Support**: Continued conversion of CSIT VAT L1 keywords to 
+   - **CSIT PAPI Support**: Finished conversion of CSIT VAT L1 keywords to
      PAPI L1 KWs in CSIT using VPP Python bindings. Redesign of key components
      PAPI L1 KWs in CSIT using VPP Python bindings. Redesign of key components
-     of PAPI Executor and PAPI history.
+     of PAPI Executor and PAPI history. Currently the only exception is
+     usage of VAT command for scale configuration.
 
    - **General Code Housekeeping**: Ongoing RF keywords optimizations,
 
    - **General Code Housekeeping**: Ongoing RF keywords optimizations,
-     removal of redundant RF keywords.
-
-   - **Test suite generator**: Added capability to generate suites for
-     different NIC models as well as throughput search algorithm types. Uses
-     base tests suites as source.
+     removal of redundant RF keywords and aligning of suite/test
+     setup/teardowns.
 
 
-   - **TOX verification**: Added verifications for test suite generator.
 
 #. PRESENTATION AND ANALYTICS LAYER
 
 
 #. PRESENTATION AND ANALYTICS LAYER
 
@@ -42,16 +44,6 @@ Changes in |csit-release|
      for better readibility and maintenance: test grouping, axis
      labels, descriptions, other informative decoration.
 
      for better readibility and maintenance: test grouping, axis
      labels, descriptions, other informative decoration.
 
-..
-    #. MISCELLANEOUS
-
-       - **2n-dnv Tests (3rd Party)**: Published performance tests for 2n-
-         dnv (2-Node Atom Denverton) from 3rd party testbeds running FD.io
-         |csit-release| automated testing code.
-         Only graphs for Packet Throughput and Speedup Multi-core and not
-         for Packet Latency were published as there are no results for Packet
-         Latency available.
-
 .. raw:: latex
 
     \clearpage
 .. raw:: latex
 
     \clearpage
@@ -63,12 +55,15 @@ Known Issues
 
 List of known issues in |csit-release| for VPP performance tests:
 
 
 List of known issues in |csit-release| for VPP performance tests:
 
-+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| #  | JiraID                                  | Issue Description                                                                                                               |
-+====+=========================================+=================================================================================================================================+
-| 1  | `CSIT-570                               | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue.                         |
-|    | <https://jira.fd.io/browse/CSIT-570>`_  | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds.                           |
-+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 2  | `CSIT-????                              | IPSecHW interface AES-128-CBC tests failing due to traffic not passing.                                                             |
-|    | <https://jira.fd.io/browse/CSIT-????>`_ |                                                                                                                                 |
-+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
++----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
+| #  | JiraID                                  | Issue Description                                                                                        |
++====+=========================================+==========================================================================================================+
+| 1  | `CSIT-570                               | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue.  |
+|    | <https://jira.fd.io/browse/CSIT-570>`_  | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds.    |
++----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
+| 2  | `CSIT-1503                              | [`TRex-519 <https://trex-tgn.cisco.com/youtrack/issue/trex-519>`_] XL710/XXV710 with FW 6.0.1 will have  |
+|    | <https://jira.fd.io/browse/CSIT-1503>`_ | Rx drop rate of 27MPPS.                                                                                  |
++----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
+| 3  | `CSIT-1499                              | AVF tests are sporadically failing on initialization of AVF interface.                                   |
+|    | <https://jira.fd.io/browse/CSIT-1499>`_ |                                                                                                          |
++----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
Integration tests