Add ligato test results
[csit.git] / docs / report / vpp_performance_tests / csit_release_notes.rst
index c9e62e8..34d943e 100644 (file)
@@ -99,24 +99,68 @@ NDR Throughput
 \r
 Non-Drop Rate Throughput discovery tests:\r
 \r
-.. csv-table::\r
-    :align: center\r
-    :header: VPP Functionality,Test Name,VPP-16.09 [Mpps],VPP-17.01 [Mpps],VPP-17.04 mean [Mpps],VPP-17.07 mean [Mpps],VPP-17.07 stdev [Mpps],17.04 to 17.07 change\r
-    :file: ../../../docs/report/vpp_performance_tests/performance_improvements/ndr_throughput.csv\r
+.. only:: html\r
+\r
+   .. csv-table::\r
+      :align: center\r
+      :file: performance_improvements/performance_improvements_ndr_top.csv\r
+\r
+.. only:: latex\r
+\r
+   .. raw:: latex\r
+\r
+      \makeatletter\r
+      \csvset{\r
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{5cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
+      }\r
+      \makeatother\r
+\r
+      {\tiny\r
+      \csvautobooklongtable[separator=comma,\r
+        respect all,\r
+        no check column count,\r
+        perfimprovements column width=1cm,\r
+        late after line={\\\hline},\r
+        late after last line={\end{longtable}}\r
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_ndr_top.csv}\r
+      }\r
+\r
 \r
 PDR Throughput\r
 ~~~~~~~~~~~~~~\r
 \r
 Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%:\r
 \r
-.. csv-table::\r
-    :align: center\r
-    :header: VPP Functionality,Test Name,VPP-16.09 [Mpps],VPP-17.01 [Mpps],VPP-17.04 mean [Mpps],VPP-17.07 mean [Mpps],VPP-17.07 stdev [Mpps],17.04 to 17.07 change\r
-    :file: ../../../docs/report/vpp_performance_tests/performance_improvements/pdr_throughput.csv\r
+.. only:: html\r
+\r
+   .. csv-table::\r
+      :align: center\r
+      :file: performance_improvements/performance_improvements_pdr_top.csv\r
+\r
+.. only:: latex\r
+\r
+   .. raw:: latex\r
+\r
+      \makeatletter\r
+      \csvset{\r
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{5cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
+      }\r
+      \makeatother\r
+\r
+      {\tiny\r
+      \csvautobooklongtable[separator=comma,\r
+        respect all,\r
+        no check column count,\r
+        perfimprovements column width=1cm,\r
+        late after line={\\\hline},\r
+        late after last line={\end{longtable}}\r
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_pdr_top.csv}\r
+      }\r
+\r
 \r
 Measured improvements are in line with VPP code optimizations listed in\r
-`VPP-17.07 release notes\r
-<https://docs.fd.io/vpp/17.07/release_notes_1707.html>`_.\r
+`VPP-17.10 release notes\r
+<https://docs.fd.io/vpp/17.10/release_notes_1710.html>`_.\r
 \r
 Other Performance Changes\r
 -------------------------\r
@@ -131,20 +175,63 @@ NDR Throughput
 \r
 Non-Drop Rate Throughput discovery tests:\r
 \r
-.. csv-table::\r
-    :align: center\r
-    :header: VPP Functionality,Test Name,VPP-16.09 [Mpps],VPP-17.01 [Mpps],VPP-17.04 mean [Mpps],VPP-17.07 mean [Mpps],VPP-17.07 stdev [Mpps],17.04 to 17.07 change\r
-    :file: ../../../docs/report/vpp_performance_tests/performance_improvements/ndr_throughput_others.csv\r
+.. only:: html\r
+\r
+   .. csv-table::\r
+      :align: center\r
+      :file: performance_improvements/performance_improvements_ndr_low.csv\r
+\r
+.. only:: latex\r
+\r
+   .. raw:: latex\r
+\r
+      \makeatletter\r
+      \csvset{\r
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{5cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
+      }\r
+      \makeatother\r
+\r
+      {\tiny\r
+      \csvautobooklongtable[separator=comma,\r
+        respect all,\r
+        no check column count,\r
+        perfimprovements column width=1cm,\r
+        late after line={\\\hline},\r
+        late after last line={\end{longtable}}\r
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_ndr_low.csv}\r
+      }\r
+\r
 \r
 PDR Throughput\r
 ~~~~~~~~~~~~~~\r
 \r
 Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%:\r
 \r
-.. csv-table::\r
-    :align: center\r
-    :header: VPP Functionality,Test Name,VPP-16.09 [Mpps],VPP-17.01 [Mpps],VPP-17.04 mean [Mpps],VPP-17.07 mean [Mpps],VPP-17.07 stdev [Mpps],17.04 to 17.07 change\r
-    :file: ../../../docs/report/vpp_performance_tests/performance_improvements/pdr_throughput_others.csv\r
+.. only:: html\r
+\r
+   .. csv-table::\r
+      :align: center\r
+      :file: performance_improvements/performance_improvements_pdr_low.csv\r
+\r
+.. only:: latex\r
+\r
+   .. raw:: latex\r
+\r
+      \makeatletter\r
+      \csvset{\r
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{5cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
+      }\r
+      \makeatother\r
+\r
+      {\tiny\r
+      \csvautobooklongtable[separator=comma,\r
+        respect all,\r
+        no check column count,\r
+        perfimprovements column width=1cm,\r
+        late after line={\\\hline},\r
+        late after last line={\end{longtable}}\r
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_pdr_low.csv}\r
+      }\r
 \r
 \r
 Known Issues\r
@@ -155,27 +242,17 @@ Here is the list of known issues in CSIT |release| for VPP performance tests:
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 | # | Issue                                           | Jira ID    | Description                                                     |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 1 | Security-groups acl-plugin scale tests failure  | CSIT-731   | VPP with 2 worker threads crashes during security-groups        |\r
-|   | with stateful acls if VPP with 2 worker threads | VPP-912    | iaclNsf and oaclNsf tests with 100k flows.                      |\r
-+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 2 | VPP fails memif tests in 4 worker 2 core setup  | CSIT-732   | VPP with 4 worker threads running on 2 physical cores crashes   |\r
-|   |                                                 | VPP-920    | during memif tests. Initial debugging points to DPDK code       |\r
-+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 3 | NDR discovery test failures 1518B frame size    | VPP-663    | VPP reporting errors: dpdk-input Rx ip checksum errors.         |\r
-|   | for ip4scale200k, ip4scale2m scale IPv4 routed- |            | Observed frequency: all test runs.                              |\r
-|   | forwarding tests. ip4scale20k tests are fine.   |            |                                                                 |\r
-+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 4 | Vic1385 and Vic1227 low performance.            | VPP-664    | Low NDR performance.                                            |\r
+| 1 | Vic1385 and Vic1227 low performance.            | VPP-664    | Low NDR performance.                                            |\r
 |   |                                                 |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 5 | Sporadic NDR discovery test failures on x520.   | CSIT-750   | Suspected issue with HW settings (BIOS, FW) in LF               |\r
+| 2 | Sporadic NDR discovery test failures on x520.   | CSIT-750   | Suspected issue with HW settings (BIOS, FW) in LF               |\r
 |   |                                                 |            | infrastructure. Issue can't be replicated outside LF.           |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 6 | VPP in 2t2c setups - large variation            | CSIT-568   | Suspected NIC firmware or DPDK driver issue affecting NDR       |\r
+| 3 | VPP in 2t2c setups - large variation            | CSIT-568   | Suspected NIC firmware or DPDK driver issue affecting NDR       |\r
 |   | of discovered NDR throughput values across      |            | throughput. Applies to XL710 and X710 NICs, x520 NICs are fine. |\r
 |   | multiple test runs with xl710 and x710 NICs.    |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 7 | Lower than expected NDR and PDR throughput with | CSIT-569   | Suspected NIC firmware or DPDK driver issue affecting NDR and   |\r
+| 4 | Lower than expected NDR and PDR throughput with | CSIT-569   | Suspected NIC firmware or DPDK driver issue affecting NDR and   |\r
 |   | xl710 and x710 NICs, compared to x520 NICs.     |            | PDR throughput. Applies to XL710 and X710 NICs.                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 \r