Code Review / csit.git / blobdiff
? search:
summary | shortlog | log | commit | commitdiff | review | tree
raw | inline | side by side
Temporarily remove Ligato tests
[csit.git] / docs / report / vpp_performance_tests / csit_release_notes.rst
diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 0318e7a..8f37fed 100644 (file)
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -4,7 +4,7 @@ Release Notes
 Changes in |csit-release|
 -------------------------
 
-#. **VPP Performance Tests**
+#. VPP PERFORMANCE TESTS
 
    - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
      configurable trial duration and number of consecutive executions.
@@ -44,42 +44,48 @@ Changes in |csit-release|
      Ansible playbooks refactor for quick server (re)installation and
      reference pointers of configuration.
 
-#. **Presentation and Analytics Layer**
+#. PRESENTATION AND ANALYTICS LAYER
 
-   - **Performance trending**: Further improved continuous performance
-     trending with anomaly detection and analysis.
+   - **Performance trending**: Further improvements of continuous
+     performance trending, anomaly detection and analysis.
 
-#. **Test Framework Optimizations**
+#. TEST FRAMEWORK OPTIMIZATIONS
 
    - **General Code Housekeeping**: Ongoing RF keywords optimizations,
      removal of redundant RF keywords.
 
+.. raw:: latex
+
+    \clearpage
+
+.. _vpp_known_issues:
+
 Known Issues
 ------------
 
 List of known issues in |csit-release| for VPP performance tests:
 
-+---+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| # | JiraID                                  | Issue Description                                                                                                               |
-+===+=========================================+=================================================================================================================================+
-| 1 | `CSIT-570                               | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue.                         |
-|   | <https://jira.fd.io/browse/CSIT-570>`_  | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds.                           |
-+---+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 2 | `VPP-1361                               | High failure rate of api call sw_interface_set_flags [admin-up|link-up].                                                        |
-|   | <https://jira.fd.io/browse/VPP-1361>`_  | Failure rate: 30-40% of tests failing due to interfaces not in link-up state after API call sw_interface_set_flags.             |
-+---+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 3 | `CSIT-1234                              | VPP IPSecHW scale interface mode 1core, low NDR and PDR 64B throughput in 3n-hsw testbeds, in CSIT-18.07 vs. CSIT-18.04.        |
-|   | <https://jira.fd.io/browse/CSIT-1234>`_ | ip4ipsecscale1000tnl-ip4base-int 1core CSIT-18.07/18.04 relative change: NDR -31%, PDR -32%, MRR -38%.                          |
-+---+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 4 | `CSIT-1242                              | VPP xl710 ip4base test 1core, low NDR and PDR 64B throughput in 3n-hsw testbeds, in CSIT-18.07 vs. CSIT-18.04.                  |
-|   | <https://jira.fd.io/browse/CSIT-1242>`_ | xl710 ip4base 1core CSIT-18.07/18.04 relative change: NDR -29%, high stdev.                                                     |
-+---+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 5 | `CSIT-1243                              | VPP nat44 base test 2core, low NDR and PDR 64B throughput in 3n-skx testbeds, compared to 3n-hsw testbeds.                      |
-|   | <https://jira.fd.io/browse/CSIT-1243>`_ | ip4base-nat44 2core 3n-skx/3n-hsw relative change: NDR -19%, PDR -22%.                                                          |
-+---+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 6 | `CSIT-1244                              | VPP lispip4 base test 2core, low NDR and PDR 64B throughput in 3n-skx testbeds, compared to 3n-hsw testbeds.                    |
-|   | <https://jira.fd.io/browse/CSIT-1244>`_ | ip4lispip4-ip4base 2core 3n-skx/3n-hsw relative change: NDR -11%, PDR -18%.                                                     |
-+---+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 7 | `CSIT-1245                              | VPP srv6proxy-stat and srv6proxy-masq, much higher NDR and PDR 78B throughput in 3n-hsw testbeds, in CSIT-18.07 vs. CSIT-18.04. |
-|   | <https://jira.fd.io/browse/CSIT-1245>`_ | Due to wrong test suite configuration in dynamic-proxy mode. Artefact of suite code refactoring.                                |
-+---+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
++----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
+| #  | JiraID                                  | Issue Description                                                                                                               |
++====+=========================================+=================================================================================================================================+
+| 1  | `CSIT-570                               | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue.                         |
+|    | <https://jira.fd.io/browse/CSIT-570>`_  | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds.                           |
++----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
+| 2  | `CSIT-1234                              | VPP IPSecHW scale interface mode 1core, low NDR and PDR 64B throughput in 3n-hsw testbeds, in CSIT-18.07 vs. CSIT-18.04.        |
+|    | <https://jira.fd.io/browse/CSIT-1234>`_ | ip4ipsecscale1000tnl-ip4base-int 1core CSIT-18.07/18.04 relative change: NDR -31%, PDR -32%, MRR -38%.                          |
++----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
+| 3  | `CSIT-1242                              | VPP xl710 ip4base test 1core, low NDR and PDR 64B throughput in 3n-hsw testbeds, in CSIT-18.07 vs. CSIT-18.04.                  |
+|    | <https://jira.fd.io/browse/CSIT-1242>`_ | xl710 ip4base 1core CSIT-18.07/18.04 relative change: NDR -29%, high stdev.                                                     |
++----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
+| 4  | `CSIT-1243                              | VPP nat44 base test 2core, low NDR and PDR 64B throughput in 3n-skx testbeds, compared to 3n-hsw testbeds.                      |
+|    | <https://jira.fd.io/browse/CSIT-1243>`_ | ip4base-nat44 2core 3n-skx/3n-hsw relative change: NDR -19%, PDR -22%.                                                          |
++----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
+| 5  | `CSIT-1245                              | VPP srv6proxy-stat and srv6proxy-masq, much higher NDR and PDR 78B throughput in 3n-hsw testbeds, in CSIT-18.07 vs. CSIT-18.04. |
+|    | <https://jira.fd.io/browse/CSIT-1245>`_ | Due to wrong test suite configuration in dynamic-proxy mode. Artefact of suite code refactoring.                                |
++----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
+| 6  | `CSIT-1246                              | Ligato K8S orchestrated tests are failing due to incompatibility of the latest released Ligato vpp-agent with VPP-18.07.        |
+|    | <https://jira.fd.io/browse/CSIT-1246>`_ | Past vpp-agent releases are not compatible either.                                                                              |
++----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
+| 7  | `CSIT-1253                              | VPP lbdpdk link bonding tests failing due to interfaces not coming up.                                                          |
+|    | <https://jira.fd.io/browse/CSIT-1253>`_ | VPP lbdpdk link bonding tests relying on DPDK functionality for bonding fail.                                                   |
++----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
Integration tests