Code Review / csit.git / blobdiff
? search:
summary | shortlog | log | commit | commitdiff | review | tree
raw | inline | side by side
Report: Add Comparison tables
[csit.git] / docs / report / vpp_performance_tests / comparisons / current_vs_previous_release.rst
diff --git a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
index e251b68..63a342e 100644 (file)
--- a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
+++ b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
@@ -23,16 +23,18 @@ results repeatability vary between the test cases.
 
 .. note::
 
-    Test results have been generated by
-
-    - `FD.io test executor vpp performance job 2n-skx`_,
-    - `FD.io test executor vpp performance job 3n-skx`_,
-    - `FD.io test executor vpp performance job 2n-clx`_,
-    - `FD.io test executor vpp performance job 2n-zn2`_,
-    - `FD.io test executor vpp performance job 2n-dnv`_,
-    - `FD.io test executor vpp performance job 3n-dnv`_,
-    - `FD.io test executor vpp performance job 3n-tsh`_,
-    - `FD.io test executor vpp performance job 2n-tx2`_
+    Test results are stored in
+
+    - `build logs from FD.io vpp performance job 2n-skx`_,
+    - `build logs from FD.io vpp performance job 3n-skx`_,
+    - `build logs from FD.io vpp performance job 2n-clx`_,
+    - `build logs from FD.io vpp performance job 2n-zn2`_,
+    - `build logs from FD.io vpp performance job 2n-dnv`_,
+    - `build logs from FD.io vpp performance job 3n-dnv`_,
+    - `build logs from FD.io vpp performance job 3n-tsh`_,
+    - `build logs from FD.io vpp performance job 2n-tx2`_,
+    - `build logs from FD.io vpp performance job 2n-aws`_,
+    - `build logs from FD.io vpp performance job 3n-aws`_
 
     with RF result files csit-vpp-perf-|srelease|-\*.zip
     `archived here <../../_static/archive/>`_.
@@ -79,6 +81,21 @@ Comparison tables in HTML, ASCII and CSV formats:
   - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr-csv.csv>`_
   - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr-csv.csv>`_
 
+Latency Comparison
+``````````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 2t1c PDR50, direction1, average value comparison <latency-changes-2n-skx-xxv710-2t1c-pdr50-d1-avg.html>`_
+  - `HTML 2t1c PDR90, direction1, average value comparison <latency-changes-2n-skx-xxv710-2t1c-pdr90-d1-avg.html>`_
+  - `HTML 2t1c PDR90, direction1, max value comparison <latency-changes-2n-skx-xxv710-2t1c-pdr90-d1-max.html>`_
+  - `ASCII 2t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-2n-skx-xxv710-2t1c-pdr50-d1-avg.txt>`_
+  - `ASCII 2t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-2n-skx-xxv710-2t1c-pdr90-d1-avg.txt>`_
+  - `ASCII 2t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-2n-skx-xxv710-2t1c-pdr90-d1-max.txt>`_
+  - `CSV 2t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-2n-skx-xxv710-2t1c-pdr50-d1-avg-csv.csv>`_
+  - `CSV 2t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-2n-skx-xxv710-2t1c-pdr90-d1-avg-csv.csv>`_
+  - `CSV 2t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-2n-skx-xxv710-2t1c-pdr90-d1-max-csv.csv>`_
+
 3n-skx
 ~~~~~~
 
@@ -121,6 +138,21 @@ Comparison tables in HTML, ASCII and CSV formats:
   - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-mrr-csv.csv>`_
   - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-mrr-csv.csv>`_
 
+Latency Comparison
+``````````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 2t1c PDR50, direction1, average value comparison <latency-changes-3n-skx-xxv710-2t1c-pdr50-d1-avg.html>`_
+  - `HTML 2t1c PDR90, direction1, average value comparison <latency-changes-3n-skx-xxv710-2t1c-pdr90-d1-avg.html>`_
+  - `HTML 2t1c PDR90, direction1, max value comparison <latency-changes-3n-skx-xxv710-2t1c-pdr90-d1-max.html>`_
+  - `ASCII 2t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-3n-skx-xxv710-2t1c-pdr50-d1-avg.txt>`_
+  - `ASCII 2t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-3n-skx-xxv710-2t1c-pdr90-d1-avg.txt>`_
+  - `ASCII 2t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-3n-skx-xxv710-2t1c-pdr90-d1-max.txt>`_
+  - `CSV 2t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-3n-skx-xxv710-2t1c-pdr50-d1-avg-csv.csv>`_
+  - `CSV 2t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-3n-skx-xxv710-2t1c-pdr90-d1-avg-csv.csv>`_
+  - `CSV 2t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-3n-skx-xxv710-2t1c-pdr90-d1-max-csv.csv>`_
+
 2n-clx-xxv710
 ~~~~~~~~~~~~~
 
@@ -163,6 +195,21 @@ Comparison tables in HTML, ASCII and CSV formats:
   - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-4t2c-mrr-csv.csv>`_
   - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-8t4c-mrr-csv.csv>`_
 
+Latency Comparison
+``````````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 2t1c PDR50, direction1, average value comparison <latency-changes-2n-clx-xxv710-2t1c-pdr50-d1-avg.html>`_
+  - `HTML 2t1c PDR90, direction1, average value comparison <latency-changes-2n-clx-xxv710-2t1c-pdr90-d1-avg.html>`_
+  - `HTML 2t1c PDR90, direction1, max value comparison <latency-changes-2n-clx-xxv710-2t1c-pdr90-d1-max.html>`_
+  - `ASCII 2t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-2n-clx-xxv710-2t1c-pdr50-d1-avg.txt>`_
+  - `ASCII 2t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-2n-clx-xxv710-2t1c-pdr90-d1-avg.txt>`_
+  - `ASCII 2t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-2n-clx-xxv710-2t1c-pdr90-d1-max.txt>`_
+  - `CSV 2t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-2n-clx-xxv710-2t1c-pdr50-d1-avg-csv.csv>`_
+  - `CSV 2t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-2n-clx-xxv710-2t1c-pdr90-d1-avg-csv.csv>`_
+  - `CSV 2t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-2n-clx-xxv710-2t1c-pdr90-d1-max-csv.csv>`_
+
 2n-clx-cx556a
 ~~~~~~~~~~~~~
 
@@ -205,6 +252,21 @@ Comparison tables in HTML, ASCII and CSV formats:
   - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-clx-cx556a-4t2c-mrr-csv.csv>`_
   - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-clx-cx556a-8t4c-mrr-csv.csv>`_
 
+Latency Comparison
+``````````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 2t1c PDR50, direction1, average value comparison <latency-changes-2n-clx-cx556a-2t1c-pdr50-d1-avg.html>`_
+  - `HTML 2t1c PDR90, direction1, average value comparison <latency-changes-2n-clx-cx556a-2t1c-pdr90-d1-avg.html>`_
+  - `HTML 2t1c PDR90, direction1, max value comparison <latency-changes-2n-clx-cx556a-2t1c-pdr90-d1-max.html>`_
+  - `ASCII 2t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-2n-clx-cx556a-2t1c-pdr50-d1-avg.txt>`_
+  - `ASCII 2t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-2n-clx-cx556a-2t1c-pdr90-d1-avg.txt>`_
+  - `ASCII 2t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-2n-clx-cx556a-2t1c-pdr90-d1-max.txt>`_
+  - `CSV 2t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-2n-clx-cx556a-2t1c-pdr50-d1-avg-csv.csv>`_
+  - `CSV 2t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-2n-clx-cx556a-2t1c-pdr90-d1-avg-csv.csv>`_
+  - `CSV 2t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-2n-clx-cx556a-2t1c-pdr90-d1-max-csv.csv>`_
+
 2n-zn2-xxv710
 ~~~~~~~~~~~~~
 
@@ -247,6 +309,21 @@ Comparison tables in HTML, ASCII and CSV formats:
   - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-zn2-xxv710-4t2c-mrr-csv.csv>`_
   - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-zn2-xxv710-8t4c-mrr-csv.csv>`_
 
+Latency Comparison
+``````````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 2t1c PDR50, direction1, average value comparison <latency-changes-2n-zn2-xxv710-2t1c-pdr50-d1-avg.html>`_
+  - `HTML 2t1c PDR90, direction1, average value comparison <latency-changes-2n-zn2-xxv710-2t1c-pdr90-d1-avg.html>`_
+  - `HTML 2t1c PDR90, direction1, max value comparison <latency-changes-2n-zn2-xxv710-2t1c-pdr90-d1-max.html>`_
+  - `ASCII 2t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-2n-zn2-xxv710-2t1c-pdr50-d1-avg.txt>`_
+  - `ASCII 2t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-2n-zn2-xxv710-2t1c-pdr90-d1-avg.txt>`_
+  - `ASCII 2t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-2n-zn2-xxv710-2t1c-pdr90-d1-max.txt>`_
+  - `CSV 2t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-2n-zn2-xxv710-2t1c-pdr50-d1-avg-csv.csv>`_
+  - `CSV 2t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-2n-zn2-xxv710-2t1c-pdr90-d1-avg-csv.csv>`_
+  - `CSV 2t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-2n-zn2-xxv710-2t1c-pdr90-d1-max-csv.csv>`_
+
 2n-dnv
 ~~~~~~
 
@@ -373,6 +450,20 @@ Comparison tables in HTML, ASCII and CSV formats:
   - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr-csv.csv>`_
   - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr-csv.csv>`_
 
+Latency Comparison
+``````````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 1t1c PDR50, direction1, average value comparison <latency-changes-3n-tsh-x520-1t1c-pdr50-d1-avg.html>`_
+  - `HTML 1t1c PDR90, direction1, average value comparison <latency-changes-3n-tsh-x520-1t1c-pdr90-d1-avg.html>`_
+  - `HTML 1t1c PDR90, direction1, max value comparison <latency-changes-3n-tsh-x520-1t1c-pdr90-d1-max.html>`_
+  - `ASCII 1t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-3n-tsh-x520-1t1c-pdr50-d1-avg.txt>`_
+  - `ASCII 1t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-3n-tsh-x520-1t1c-pdr90-d1-avg.txt>`_
+  - `ASCII 1t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-3n-tsh-x520-1t1c-pdr90-d1-max.txt>`_
+  - `CSV 1t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-3n-tsh-x520-1t1c-pdr50-d1-avg-csv.csv>`_
+  - `CSV 1t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-3n-tsh-x520-1t1c-pdr90-d1-avg-csv.csv>`_
+  - `CSV 1t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-3n-tsh-x520-1t1c-pdr90-d1-max-csv.csv>`_
 
 2n-tx2
 ~~~~~~
@@ -415,3 +506,102 @@ Comparison tables in HTML, ASCII and CSV formats:
   - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-tx2-1t1c-mrr-csv.csv>`_
   - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-tx2-2t2c-mrr-csv.csv>`_
   - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-tx2-4t4c-mrr-csv.csv>`_
+
+Latency Comparison
+``````````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 1t1c PDR50, direction1, average value comparison <latency-changes-2n-tx2-xl710-1t1c-pdr50-d1-avg.html>`_
+  - `HTML 1t1c PDR90, direction1, average value comparison <latency-changes-2n-tx2-xl710-1t1c-pdr90-d1-avg.html>`_
+  - `HTML 1t1c PDR90, direction1, max value comparison <latency-changes-2n-tx2-xl710-1t1c-pdr90-d1-max.html>`_
+  - `ASCII 1t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-2n-tx2-xl710-1t1c-pdr50-d1-avg.txt>`_
+  - `ASCII 1t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-2n-tx2-xl710-1t1c-pdr90-d1-avg.txt>`_
+  - `ASCII 1t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-2n-tx2-xl710-1t1c-pdr90-d1-max.txt>`_
+  - `CSV 1t1c PDR50, direction1, average value comparison <../../_static/vpp/latency-changes-2n-tx2-xl710-1t1c-pdr50-d1-avg-csv.csv>`_
+  - `CSV 1t1c PDR90, direction1, average value comparison <../../_static/vpp/latency-changes-2n-tx2-xl710-1t1c-pdr90-d1-avg-csv.csv>`_
+  - `CSV 1t1c PDR90, direction1, max value comparison <../../_static/vpp/latency-changes-2n-tx2-xl710-1t1c-pdr90-d1-max-csv.csv>`_
+
+2n-aws
+~~~~~~
+
+NDR Comparison
+``````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 2t1c NDR comparison <performance-changes-2n-aws-2t1c-ndr.html>`_
+  - `HTML 4t2c NDR comparison <performance-changes-2n-aws-4t2c-ndr.html>`_
+  - `ASCII 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-aws-2t1c-ndr.txt>`_
+  - `ASCII 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-aws-4t2c-ndr.txt>`_
+  - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-aws-2t1c-ndr-csv.csv>`_
+  - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-aws-4t2c-ndr-csv.csv>`_
+
+PDR Comparison
+``````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 2t1c PDR comparison <performance-changes-2n-aws-2t1c-pdr.html>`_
+  - `HTML 4t2c PDR comparison <performance-changes-2n-aws-4t2c-pdr.html>`_
+  - `ASCII 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-aws-2t1c-pdr.txt>`_
+  - `ASCII 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-aws-4t2c-pdr.txt>`_
+  - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-aws-2t1c-pdr-csv.csv>`_
+  - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-aws-4t2c-pdr-csv.csv>`_
+
+MRR Comparison
+``````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 2t1c MRR comparison <performance-changes-2n-aws-2t1c-mrr.html>`_
+  - `HTML 4t2c MRR comparison <performance-changes-2n-aws-4t2c-mrr.html>`_
+  - `HTML 8t4c MRR comparison <performance-changes-2n-aws-8t4c-mrr.html>`_
+  - `ASCII 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-aws-2t1c-mrr.txt>`_
+  - `ASCII 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-aws-4t2c-mrr.txt>`_
+  - `ASCII 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-aws-8t4c-mrr.txt>`_
+  - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-aws-2t1c-mrr-csv.csv>`_
+  - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-aws-4t2c-mrr-csv.csv>`_
+  - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-aws-8t4c-mrr-csv.csv>`_
+
+3n-aws
+~~~~~~
+
+NDR Comparison
+``````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 2t1c NDR comparison <performance-changes-3n-aws-2t1c-ndr.html>`_
+  - `HTML 4t2c NDR comparison <performance-changes-3n-aws-4t2c-ndr.html>`_
+  - `ASCII 2t1c NDR comparison <../../_static/vpp/performance-changes-3n-aws-2t1c-ndr.txt>`_
+  - `ASCII 4t2c NDR comparison <../../_static/vpp/performance-changes-3n-aws-4t2c-ndr.txt>`_
+  - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-3n-aws-2t1c-ndr-csv.csv>`_
+  - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-3n-aws-4t2c-ndr-csv.csv>`_
+
+PDR Comparison
+``````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 2t1c PDR comparison <performance-changes-3n-aws-2t1c-pdr.html>`_
+  - `HTML 4t2c PDR comparison <performance-changes-3n-aws-4t2c-pdr.html>`_
+  - `ASCII 2t1c PDR comparison <../../_static/vpp/performance-changes-3n-aws-2t1c-pdr.txt>`_
+  - `ASCII 4t2c PDR comparison <../../_static/vpp/performance-changes-3n-aws-4t2c-pdr.txt>`_
+  - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-3n-aws-2t1c-pdr-csv.csv>`_
+  - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-3n-aws-4t2c-pdr-csv.csv>`_
+
+MRR Comparison
+``````````````
+
+Comparison tables in HTML, ASCII and CSV formats:
+
+  - `HTML 2t1c MRR comparison <performance-changes-3n-aws-2t1c-mrr.html>`_
+  - `HTML 4t2c MRR comparison <performance-changes-3n-aws-4t2c-mrr.html>`_
+  - `HTML 8t4c MRR comparison <performance-changes-3n-aws-8t4c-mrr.html>`_
+  - `ASCII 2t1c MRR comparison <../../_static/vpp/performance-changes-3n-aws-2t1c-mrr.txt>`_
+  - `ASCII 4t2c MRR comparison <../../_static/vpp/performance-changes-3n-aws-4t2c-mrr.txt>`_
+  - `ASCII 8t4c MRR comparison <../../_static/vpp/performance-changes-3n-aws-8t4c-mrr.txt>`_
+  - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-3n-aws-2t1c-mrr-csv.csv>`_
+  - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-3n-aws-4t2c-mrr-csv.csv>`_
+  - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-3n-aws-8t4c-mrr-csv.csv>`_
Integration tests