Report: Compare MRR data

[csit.git] / docs / report / vpp_performance_tests / csit_release_notes.rst

diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 17003bc..e7c61e6 100644 (file)
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -101,6 +101,19 @@ Measured improvements are in line with VPP code optimizations listed in
 `VPP-18.01 release notes\r
 <https://docs.fd.io/vpp/18.01/release_notes_1801.html>`_.\r
 \r
+MRR Throughput Changes\r
+~~~~~~~~~~~~~~~~~~~~~~\r
+\r
+MRR changes between releases are available in a CSV and\r
+pretty ASCII formats:\r
+\r
+  - `csv format for 1t1c <../_static/vpp/performance-changes-mrr-1t1c-full.csv>`_,\r
+  - `csv format for 2t2c <../_static/vpp/performance-changes-mrr-2t2c-full.csv>`_,\r
+  - `csv format for 4t4c <../_static/vpp/performance-changes-mrr-4t4c-full.csv>`_,\r
+  - `pretty ASCII format for 1t1c <../_static/vpp/performance-changes-mrr-1t1c-full.txt>`_,\r
+  - `pretty ASCII format for 2t2c <../_static/vpp/performance-changes-mrr-2t2c-full.txt>`_,\r
+  - `pretty ASCII format for 4t4c <../_static/vpp/performance-changes-mrr-4t4c-full.txt>`_.\r
+\r
 Known Issues\r
 ------------\r
 \r
@@ -122,4 +135,20 @@ Here is the list of known issues in CSIT |release| for VPP performance tests:
 | 4 | QAT IPSec scale with 1000 tunnels (interfaces)  | VPP-1121   | VPP crashes during configuration of 1000 IPsec tunnels.         |\r
 |   | in 2t2c config, all tests are failing.          |            | 1t1c tests are not affected                                     |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-\r
+| 5 | rls1801 plugin related performance regression   | CSIT-925   | With all plugins loaded NDR, PDR and MaxRates vary              |\r
+|   |                                                 |            | intermittently from 3% to 5% across multiple test executions.   |\r
+|   |                                                 |            | Requires plugin code bisecting.                                 |\r
++---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
+| 6 | rls1801 generic small performance regression    | CSIT-926   | Generic performance regression of discovered NDR, PDR and       |\r
+|   | ip4base, l2xcbase, l2bdbase                     |            | MaxRates of -3%..-1% vs. rls1710, affects ip4base, l2xcbase,    |\r
+|   |                                                 |            | l2bdbase test suites. Not detected by CSIT performance trending |\r
+|   |                                                 |            | scheme as it was masked out by another issue CSIT-925.          |\r
++---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
+| 7 | rls1801 substantial NDR performance regression  | CSIT-927   | Much lower NDR for vhostvr1024 tests, with mean values          |\r
+|   | for vhost-user vring size of 1024               |            | regression of -17%..-42% vs. rls1710, but also very high        |\r
+|   |                                                 |            | standard deviation of up to 1.46 Mpps => poor repeatibility.    |\r
+|   |                                                 |            | Making mean values not fully representative.                    |\r
++---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
+| 8 | rls1801 substantial NDR/PDR regression for      | CSIT-928   | NDR regression of -7%..-15%, PDR regression of -3%..-15%        |\r
+|   | IPSec tunnel scale with HW QAT crypto-dev       |            | compared to rls1710.                                            |\r
++---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r