CSIT-1197: Add Comparison Across Testbeds to the Report

[csit.git] / docs / report / vpp_performance_tests / csit_release_notes.rst

diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 4fdee97..74e3082 100644 (file)
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -38,21 +38,21 @@ Changes in |csit-release|
 Performance Changes
 -------------------
 
-Relative performance changes in measured packet throughput in CSIT
-|release| are calculated against the results from CSIT |release-1|
+Relative performance changes in measured packet throughput in |csit-release|
+are calculated against the results from |csit-release-1|
 report. Listed mean and standard deviation values are computed based on
 a series of the same tests executed against respective VPP releases to
-verify test results repeatibility, with percentage change calculated for
+verify test results repeatability, with percentage change calculated for
 mean values. Note that the standard deviation is quite high for a small
 number of packet throughput tests, what indicates poor test results
 repeatability and makes the relative change of mean throughput value not
 fully representative for these tests. The root causes behind poor
-results repeatibility vary between the test cases.
+results repeatability vary between the test cases.
 
 NDR Changes
 ~~~~~~~~~~~
 
-NDR small packet throughput changes between releases are available in a
+NDR throughput changes between releases are available in a
 CSV and pretty ASCII formats:
 
   - `csv format for 1t1c <../_static/vpp/performance-changes-1t1c-ndr.csv>`_,
@@ -63,13 +63,15 @@ CSV and pretty ASCII formats:
 .. note::
 
     Test results have been generated by
-    `FD.io test executor vpp performance jobs`_ with Robot Framework result
-    files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_.
+    `FD.io test executor vpp performance job 3n-hsw`_,
+    with Robot Framework result
+    files csit-vpp-perf-|srelease|-\*.zip
+    `archived here <../_static/archive/>`_.
 
 PDR Changes
 ~~~~~~~~~~~
 
-NDR small packet throughput changes between releases are available in a
+PDR throughput changes between releases are available in a
 CSV and pretty ASCII formats:
 
   - `csv format for 1t1c <../_static/vpp/performance-changes-1t1c-pdr.csv>`_,
@@ -80,8 +82,10 @@ CSV and pretty ASCII formats:
 .. note::
 
     Test results have been generated by
-    `FD.io test executor vpp performance jobs`_ with Robot Framework result
-    files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_.
+    `FD.io test executor vpp performance job 3n-hsw`_,
+    with Robot Framework result
+    files csit-vpp-perf-|srelease|-\*.zip
+    `archived here <../_static/archive/>`_.
 
 MRR Changes
 ~~~~~~~~~~~
@@ -106,15 +110,44 @@ CSV and pretty ASCII formats:
 Comparison Across Testbeds
 --------------------------
 
-.. warning::
+Relative performance changes in measured packet throughputon 3-Node Skx testbed
+are calculated against the results measured on 3-Node Hsw testbed.
 
-    TODO: Add:
+NDR Changes
+~~~~~~~~~~~
 
-    Table 1.
-    Test Case     3-Node Hsw    3-Node Skx    Skx vs. Hsw Delta [%]
+NDR changes between testbeds are available in a
+CSV and pretty ASCII formats:
 
-    Table 2.
-    Test Case     3-Node Skx    2-Node Skx    2-Node vs. 3-Node Delta [%]
+  - `csv format for ndr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
+  - `pretty ASCII format for ndr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
+
+.. note::
+
+    Test results have been generated by
+    `FD.io test executor vpp performance job 3n-hsw`_ and
+    `FD.io test executor vpp performance job 3n-skx`_
+    with Robot Framework result
+    files csit-vpp-perf-|srelease|-\*.zip
+    `archived here <../_static/archive/>`_.
+
+PDR Changes
+~~~~~~~~~~~
+
+PDR throughput changes between testbeds are available in a
+CSV and pretty ASCII formats:
+
+  - `csv format for pdr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
+  - `pretty ASCII format for pdr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
+
+.. note::
+
+    Test results have been generated by
+    `FD.io test executor vpp performance job 3n-hsw`_ and
+    `FD.io test executor vpp performance job 3n-skx`_
+    with Robot Framework result
+    files csit-vpp-perf-|srelease|-\*.zip
+    `archived here <../_static/archive/>`_.
 
 Throughput Trending
 -------------------