Code Review / csit.git / blobdiff
? search:
summary | shortlog | log | commit | commitdiff | review | tree
raw | inline | side by side
report 1904: updated methodology (nfv density, startup.conf) and vpp perf rls notes
[csit.git] / docs / report / vpp_performance_tests / csit_release_notes.rst
diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 50dce62..a96a4f4 100644 (file)
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -6,32 +6,36 @@ Changes in |csit-release|
 
 #. VPP PERFORMANCE TESTS
 
 
 #. VPP PERFORMANCE TESTS
 
-   - **Service density 2n-skx tests**: Network Function Virtualization (NFV)
-     service density tests focus on measuring total per server throughput at
-     varied NFV service *packing* densities with vswitch providing host
-     dataplane. The goal is to compare and contrast performance of a shared
-     vswitch for different network topologies and virtualization technologies,
-     and their impact on vswitch performance and efficiency in a range of NFV
-     service configurations.
-
-   - **Experimental Soak Tests**: Added performamce soak tests framework
+   - **Service density 2n-skx tests**: Added higher NF density tests with two
+     NFs' data-plane threads sharing a physical core. VPP IPv4 routing is now 
+     used as a VNF payload similar to CNF tests.
+
+   - **Soak Tests**: Optimized performamce soak tests framework
      code for extended time duration tests and throughput discovery
      at given PLR and at give total test time e.g. minutes, hours,
      code for extended time duration tests and throughput discovery
      at given PLR and at give total test time e.g. minutes, hours,
-     days, weeks, months, years. See updated
+     days, weeks. See updated
      :ref:`test_methodology` section for more details.
 
 #. TEST FRAMEWORK
 
      :ref:`test_methodology` section for more details.
 
 #. TEST FRAMEWORK
 
-   - **Container code optimizations**: Optimized container library allows to
-     run containre_memif tests faster.
+   - **Qemu code refactor**: Complete code refactor of the key components of
+     QemuUtil.py and QemuManager.py (L1 and L2 KW counterparts). Added
+     implementation of kernel-image-kvm based VM replacing the previously used
+     NestedVM images. Added ability to run VPP as a payload in VNF.
 
 
-   - **CSIT PAPI Support**: Continue converting existing VAT L1 keywords to
-     PAPI L1 KWs in CSIT using VPP Python bindings. Required for migrating away
-     from VAT.
+   - **CSIT PAPI Support**: Continued conversion of CSIT VAT L1 keywords to 
+     PAPI L1 KWs in CSIT using VPP Python bindings. Redesign of key components
+     of PAPI Executor and PAPI history.
 
    - **General Code Housekeeping**: Ongoing RF keywords optimizations,
      removal of redundant RF keywords.
 
 
    - **General Code Housekeeping**: Ongoing RF keywords optimizations,
      removal of redundant RF keywords.
 
+   - **Test suite generator**: Added capability to generate suites for
+     different NIC models as well as throughput search algorithm types. Uses
+     base tests suites as source.
+
+   - **TOX verification**: Added verifications for test suite generator.
+
 #. PRESENTATION AND ANALYTICS LAYER
 
    - **Graphs Layout Improvements**: Improved performance graphs layout
 #. PRESENTATION AND ANALYTICS LAYER
 
    - **Graphs Layout Improvements**: Improved performance graphs layout
@@ -40,8 +44,8 @@ Changes in |csit-release|
 
 #. MISCELLANEOUS
 
 
 #. MISCELLANEOUS
 
-   - **3n-dnv Tests (3rd Party)**: Published performance tests for 3n-
-     dnv (3-Node Atom Denverton) from 3rd party testbeds running FD.io
+   - **2n-dnv Tests (3rd Party)**: Published performance tests for 2n-
+     dnv (2-Node Atom Denverton) from 3rd party testbeds running FD.io
      |csit-release| automated testing code.
      Only graphs for Packet Throughput and Speedup Multi-core and not
      for Packet Latency were published as there are no results for Packet
      |csit-release| automated testing code.
      Only graphs for Packet Throughput and Speedup Multi-core and not
      for Packet Latency were published as there are no results for Packet
@@ -64,18 +68,6 @@ List of known issues in |csit-release| for VPP performance tests:
 | 1  | `CSIT-570                               | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue.                         |
 |    | <https://jira.fd.io/browse/CSIT-570>`_  | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds.                           |
 +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
 | 1  | `CSIT-570                               | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue.                         |
 |    | <https://jira.fd.io/browse/CSIT-570>`_  | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds.                           |
 +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 2  | `VPP-1562                               | Link bonding (mode LACP, transmit policy l34) test are failing due to VPP crashing producing core dump.                         |
-|    | <https://jira.fd.io/browse/VPP-1562>`_  |                                                                                                                                 |
-+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 3  | `CSIT-1427                              | Scale HW IPsec are failing due to issue with selecting ipsec backend in VPP.                                                    |
-|    | <https://jira.fd.io/browse/CSIT-1427>`_ |                                                                                                                                 |
-+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 4  | `VPP-1563                               | AVF L2patch tests are failing for all packet size and core combination. Reason: null-node blackholed packets in show error.     |
-|    | <https://jira.fd.io/browse/VPP-1563>`_  |                                                                                                                                 |
-+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 5  | `CSIT-1234                              | VPP IPSecHW/IPSecsW scale interface mode, low NDR and PDR 64B throughput in 3n-hsw testbeds, in CSIT-19.01 vs. CSIT-18.10.      |
-|    | <https://jira.fd.io/browse/CSIT-1234>`_ |                                                                                                                                 |
-+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 6  | `CSIT-1428                              | VM vhost tests with RXQ more than 2 are failing on 3n-skx. Suspecting RXQ configuration issues.                                 |
-|    | <https://jira.fd.io/browse/CSIT-1428>`_ |                                                                                                                                 |
+| 2  | `CSIT-????                              | IPSecHW interface AES-128-CBC tests failing due to traffic not passing.                                                             |
+|    | <https://jira.fd.io/browse/CSIT-????>`_ |                                                                                                                                 |
 +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
 +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
Integration tests