Code Review / csit.git / blobdiff
? search:
summary | shortlog | log | commit | commitdiff | review | tree
raw | inline | side by side
DOC: 1908rls release notes for VPP Hoststack TCP performance tests.
[csit.git] / docs / report / vpp_performance_tests / csit_release_notes.rst
diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 53b9395..8534aa1 100644 (file)
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -6,35 +6,37 @@ Changes in |csit-release|
 
 #. VPP PERFORMANCE TESTS
 
-   - **Service density 2n-skx tests**: Added higher NF density tests with two
-     NFs' data-plane threads sharing a physical core. VPP IPv4 routing is now
-     used as a VNF payload similar to CNF tests.
+   - **Service density 2n-skx tests**: Added higher NF density tests with
+     802.1q (vlan) and 802.1ad (vxlan) encapsulation from Traffic Generator.
 
-   - **Soak Tests**: Optimized performamce soak tests framework
-     code for extended time duration tests and throughput discovery
-     at given PLR and at give total test time e.g. minutes, hours,
-     days, weeks. See updated
-     :ref:`test_methodology` section for more details.
+   - **GBP tests**: Added GBP routing test cases with 802.1q (vlan) external
+     traffic.
 
-#. TEST FRAMEWORK
+   - **AVF IPv4 scale tests**: Increased coverage of AVF IPv4 base and scale
+     test cases.
+
+   - **2n-skx tests**: Increased coverage of selected (COP, iACL, Policer)
+     test cases.
+
+   - **IPsec scale tests**: Added IPsec interface mode scale tests with
+     1, 40, 400, 1000, 5000, 10000, 20000, 40000, 60000 tunnels. Removed DPDK
+     backend dependency.
 
-   - **Qemu code refactor**: Complete code refactor of the key components of
-     QemuUtil.py and QemuManager.py (L1 and L2 KW counterparts). Added
-     implementation of kernel-image-kvm based VM replacing the previously used
-     NestedVM images. Added ability to run VPP as a payload in VNF.
+   - **Hoststack TCP tests**: Added Hoststack TCP performance tests
+     using WRK to the HTTP static server plugin measuring connections
+     per second and requests per second.
+
+#. TEST FRAMEWORK
 
-   - **CSIT PAPI Support**: Continued conversion of CSIT VAT L1 keywords to
+   - **CSIT PAPI Support**: Finished conversion of CSIT VAT L1 keywords to
      PAPI L1 KWs in CSIT using VPP Python bindings. Redesign of key components
-     of PAPI Executor and PAPI history.
+     of PAPI Executor and PAPI history. Currently the only exception is
+     usage of VAT command for scale configuration.
 
    - **General Code Housekeeping**: Ongoing RF keywords optimizations,
-     removal of redundant RF keywords.
+     removal of redundant RF keywords and aligning of suite/test
+     setup/teardowns.
 
-   - **Test suite generator**: Added capability to generate suites for
-     different NIC models as well as throughput search algorithm types. Uses
-     base tests suites as source.
-
-   - **TOX verification**: Added verifications for test suite generator.
 
 #. PRESENTATION AND ANALYTICS LAYER
 
@@ -42,16 +44,6 @@ Changes in |csit-release|
      for better readibility and maintenance: test grouping, axis
      labels, descriptions, other informative decoration.
 
-..
-    #. MISCELLANEOUS
-
-       - **2n-dnv Tests (3rd Party)**: Published performance tests for 2n-
-         dnv (2-Node Atom Denverton) from 3rd party testbeds running FD.io
-         |csit-release| automated testing code.
-         Only graphs for Packet Throughput and Speedup Multi-core and not
-         for Packet Latency were published as there are no results for Packet
-         Latency available.
-
 .. raw:: latex
 
     \clearpage
@@ -69,15 +61,9 @@ List of known issues in |csit-release| for VPP performance tests:
 | 1  | `CSIT-570                               | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue.  |
 |    | <https://jira.fd.io/browse/CSIT-570>`_  | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds.    |
 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
-| 2  | `CSIT-1501                              | Sporadic crypto backend fails loading `VPP-1670 <https://jira.fd.io/browse/VPP-1670>`_                   |
-|    | <https://jira.fd.io/browse/CSIT-1501>`_ |                                                                                                          |
-+----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
-| 3  | `CSIT-1427                              | Sporadic HW aes-128-cbc-sha1 tunnel-interface tests are failing.                                         |
-|    | <https://jira.fd.io/browse/CSIT-1427>`_ | `VPP-1671 <https://jira.fd.io/browse/VPP-1671>`_                                                         |
-+----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
-| 4  | `CSIT-1498                              | Memif tests are sporadically failing on initialization of memif connection.                              |
-|    | <https://jira.fd.io/browse/CSIT-1498>`_ |                                                                                                          |
+| 2  | `CSIT-1503                              | [`TRex-519 <https://trex-tgn.cisco.com/youtrack/issue/trex-519>`_] XL710/XXV710 with FW 6.0.1 will have  |
+|    | <https://jira.fd.io/browse/CSIT-1503>`_ | Rx drop rate of 27MPPS.                                                                                  |
 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
-| 5  | `CSIT-1499                              | AVF tests are sporadically failing on initialization of AVF interface.                                   |
+| 3  | `CSIT-1499                              | AVF tests are sporadically failing on initialization of AVF interface.                                   |
 |    | <https://jira.fd.io/browse/CSIT-1499>`_ |                                                                                                          |
 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
Integration tests