Updates to csit rls1704 report related to increased number of test job executions.

Change-Id: If16e312eeb377cabc96fe1a671d9bdfedf5e87b0
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>

diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst
index d781993..97b6abc 100644 (file)
--- a/docs/report/introduction/overview.rst
+++ b/docs/report/introduction/overview.rst
@@ -65,3 +65,11 @@ CSIT |release| report contains following main sections and sub-sections:
    executions using CSIT Robot Framework output files as source data; *VPP\r
    Performance Results*, *Testpmd Performance Results*, *VPP Functional\r
    Results*, *HoneyComb Functional Results*, *VPPtest Functional Results*.\r
    executions using CSIT Robot Framework output files as source data; *VPP\r
    Performance Results*, *Testpmd Performance Results*, *VPP Functional\r
    Results*, *HoneyComb Functional Results*, *VPPtest Functional Results*.\r

+\r
+#. **Test Configuration** - auto-generated DUT configuration data from CSIT jobs\r
+   executions using CSIT Robot Framework output files as source data; *VPP\r
+   Performance Test Configs*, *VPP Functional Test Configs*.\r
+\r
+#. **Test Operational Data** - auto-generated DUT operational data from CSIT jobs\r
+   executions using CSIT Robot Framework output files as source data; *VPP\r
+   Performance Operational Data*.\r

diff --git a/docs/report/testpmd_performance_tests/packet_latency_graphs/index.rst b/docs/report/testpmd_performance_tests/packet_latency_graphs/index.rst
index 4402794..fb16912 100644 (file)
--- a/docs/report/testpmd_performance_tests/packet_latency_graphs/index.rst
+++ b/docs/report/testpmd_performance_tests/packet_latency_graphs/index.rst
@@ -1,6 +1,28 @@
 Packet Latency Graphs
 =====================
 
 Packet Latency Graphs
 =====================
 

+Plotted results are generated from a single execution of CSIT NDR discovery
+test. Box plots are used to show the Minimum, Median and Maximum packet
+latency per test.
+
+*Title of each graph* is a regex (regular expression) matching all
+throughput test cases plotted on this graph, *X-axis labels* are indices
+of individual test suites executed by csit-dpdk-perf-1704-all job that
+created result output file used as data source for the graph, *Y-axis
+labels* are measured packet Latency [uSec] values, and the *Graph
+legend* lists the plotted test suites and their indices. Latency is
+reported for concurrent symmetric bi-directional flows, separately for
+each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
+ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
+
+.. note::
+
+    Test results have been generated by FD.io test executor jobs
+    `csit-dpdk-perf-1704-all
+    <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_,
+    with Robot Framework result files csit-dpdk-perf-1704-all-1.zip
+    `archived here <../../_static/archive/>`_.
+

 .. toctree::
 
     l2
 .. toctree::
 
     l2

diff --git a/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst b/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst
index 95b82d7..51ffea2 100644 (file)
--- a/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst
+++ b/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst
@@ -7,28 +7,6 @@ NDR throughput rate. Latency is reported for Testpmd running in multiple
 configurations of Testpmd pmd thread(s), a.k.a. Testpmd data plane
 thread(s), and their physical CPU core(s) placement.
 
 configurations of Testpmd pmd thread(s), a.k.a. Testpmd data plane
 thread(s), and their physical CPU core(s) placement.
 

-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-dpdk-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-dpdk-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_,
-    with Robot Framework result files csit-dpdk-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_.
-

 Testpmd packet latency - running in configuration of **one worker thread (1t) on one
 physical core (1c)** - is presented in the figure below.
 
 Testpmd packet latency - running in configuration of **one worker thread (1t) on one
 physical core (1c)** - is presented in the figure below.
 

diff --git a/docs/report/testpmd_performance_tests/packet_throughput_graphs/index.rst b/docs/report/testpmd_performance_tests/packet_throughput_graphs/index.rst
index 4e4e87a..d607b6e 100644 (file)
--- a/docs/report/testpmd_performance_tests/packet_throughput_graphs/index.rst
+++ b/docs/report/testpmd_performance_tests/packet_throughput_graphs/index.rst
@@ -1,6 +1,39 @@
 Packet Throughput Graphs
 ========================
 
 Packet Throughput Graphs
 ========================
 

+Plotted results are generated by multiple executions of the same CSIT
+performance tests across three physical testbeds within LF FD.io labs.
+To provide a descriptive summary view, Box-and-Whisker plots are used to
+display variation in measured throughput values, without making any
+assumptions of the underlying statistical distribution.
+
+For each plotted test case, Box-and-Whisker plots show the quartiles
+(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile /
+mean, 3rd quartile / 75th percentile, Max) across collected data set
+(data set size stated in the note below). Outliers are plotted as
+individual points. Min and max values are plotted as bottom and top
+Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and
+top edge of the box. If multiple samples match only two values, and all
+samples fall between them, then no whiskers are plotted. If all samples
+have the same value, only a horizontal line is plotted.
+
+*Title of each graph* is a regex (regular expression) matching all
+throughput test cases plotted on this graph, *X-axis labels* are indices
+of individual test suites executed by csit-dpdk-perf-1704-all jobs that
+created result output files used as data sources for the graph, *Y-axis
+labels* are measured Packets Per Second [pps] values, and the *Graph
+legend* lists the plotted test suites and their indices.
+
+.. note::
+
+    Test results have been generated by FD.io test executor jobs
+    `csit-dpdk-perf-1704-all
+    <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_,
+    with Robot Framework result files csit-dpdk-perf-1704-all-<id>.zip
+    `archived here <../../_static/archive/>`_. Plotted data set size per
+    test case is equal to the number of job executions presented in this
+    report version: **10**.
+

 .. toctree::
 
     l2
 .. toctree::
 
     l2

diff --git a/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst
index 8b13dc9..40f320e 100644 (file)
--- a/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst
+++ b/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst
@@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for Testpmd
 running in multiple configurations of Testpmd pmd thread(s), a.k.a. Testpmd
 data plane thread(s), and their physical CPU core(s) placement.
 
 running in multiple configurations of Testpmd pmd thread(s), a.k.a. Testpmd
 data plane thread(s), and their physical CPU core(s) placement.
 

-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-dpdk-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-dpdk-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_,
-    with Robot Framework result files csit-dpdk-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_. Plotted data set size per
-    test case is equal to the number of job executions presented in this
-    report version: **4**.
-

 NDR Throughput
 ~~~~~~~~~~~~~~
 
 NDR Throughput
 ~~~~~~~~~~~~~~
 

diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst
index f333f23..f717094 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst
@@ -1,6 +1,28 @@
 Packet Latency Graphs
 =====================
 
 Packet Latency Graphs
 =====================
 

+Plotted results are generated from a single execution of CSIT NDR discovery
+test. Box plots are used to show the Minimum, Median and Maximum packet
+latency per test.
+
+*Title of each graph* is a regex (regular expression) matching all
+throughput test cases plotted on this graph, *X-axis labels* are indices
+of individual test suites executed by csit-vpp-perf-1704-all job that
+created result output file used as data source for the graph, *Y-axis
+labels* are measured packet Latency [uSec] values, and the *Graph
+legend* lists the plotted test suites and their indices. Latency is
+reported for concurrent symmetric bi-directional flows, separately for
+each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
+ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
+
+.. note::
+
+    Test results have been generated by FD.io test executor jobs
+    `csit-vpp-perf-1704-all
+    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
+    with Robot Framework result files csit-vpp-perf-1704-all-6.zip
+    `archived here <../../_static/archive/>`_.
+

 .. toctree::
 
     l2
 .. toctree::
 
     l2

diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst
index 0c506f7..db82797 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst
@@ -10,20 +10,6 @@ for VPP running in multiple configurations of VPP worker thread(s),
 a.k.a. VPP data plane thread(s), and their physical CPU core(s)
 placement.
 
 a.k.a. VPP data plane thread(s), and their physical CPU core(s)
 placement.
 

-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-

 .. note::
 
     Test results have been generated by FD.io test executor jobs
 .. note::
 
     Test results have been generated by FD.io test executor jobs

diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst
index 5ddf813..10b22b2 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst
@@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 

-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_.
-

 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html
 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html

diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst
index ab39a97..b14b019 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst
@@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 

-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_.
-

 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html
 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html

diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst
index 93c1bef..fd556a3 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst
@@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 

-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_.
-

 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html
 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html

diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst
index 186bbfd..c650a53 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst
@@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 

-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_.
-

 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html
 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html

diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst
index a017a42..129470d 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst
@@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 

-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_.
-

 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html
 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html

diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst
index d501028..eb9732d 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst
@@ -8,28 +8,6 @@ for VPP running in multiple configurations of VPP worker thread(s),
 a.k.a. VPP data plane thread(s), and their physical CPU core(s)
 placement.
 
 a.k.a. VPP data plane thread(s), and their physical CPU core(s)
 placement.
 

-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_.
-

 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html
 VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
 
 .. raw:: html

diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst
index 4abdf8e..d744ef0 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst
@@ -1,6 +1,39 @@
 Packet Throughput Graphs
 ========================
 
 Packet Throughput Graphs
 ========================
 

+Plotted results are generated by multiple executions of the same CSIT
+performance tests across three physical testbeds within LF FD.io labs.
+To provide a descriptive summary view, Box-and-Whisker plots are used to
+display variation in measured throughput values, without making any
+assumptions of the underlying statistical distribution.
+
+For each plotted test case, Box-and-Whisker plots show the quartiles
+(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile /
+mean, 3rd quartile / 75th percentile, Max) across collected data set
+(data set size stated in the note below). Outliers are plotted as
+individual points. Min and max values are plotted as bottom and top
+Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and
+top edge of the box. If multiple samples match only two values, and all
+samples fall between them, then no whiskers are plotted. If all samples
+have the same value, only a horizontal line is plotted.
+
+*Title of each graph* is a regex (regular expression) matching all
+throughput test cases plotted on this graph, *X-axis labels* are indices
+of individual test suites executed by csit-vpp-perf-1704-all jobs that
+created result output files used as data sources for the graph, *Y-axis
+labels* are measured Packets Per Second [pps] values, and the *Graph
+legend* lists the plotted test suites and their indices.
+
+.. note::
+
+    Test results have been generated by FD.io test executor jobs
+    `csit-vpp-perf-1704-all
+    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
+    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
+    `archived here <../../_static/archive/>`_. Plotted data set size per
+    test case is equal to the number of job executions presented in this
+    report version: **10**.
+

 .. toctree::
 
     l2
 .. toctree::
 
     l2

diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst
index ad71a8a..a76b78f 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst
@@ -10,32 +10,10 @@ Performance is reported for VPP running in multiple configurations of
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 

-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_. Plotted data set size per
-    test case is equal to the number of job executions presented in this
-    report version: **5**.
+Note: only 1,000 scale IPSec tunnel test cases are plotted. Single IPSec
+tunnel test cases are listed in regex outputs, but marked {NOT PLOTTED}
+due to the current limitation of auto-plotting scripts. For all test
+result data please refere to **Detailed Test Results** section.

 
 NDR Throughput
 ~~~~~~~~~~~~~~
 
 NDR Throughput
 ~~~~~~~~~~~~~~


@@ -59,11 +37,11 @@ git repository:
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-ndrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-ndrdisc
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-ndrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-ndrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-ndrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-ndrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-ndrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-ndrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-ndrdisc
     40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrdisc
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-ndrdisc
     40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrdisc
 


@@ -87,11 +65,11 @@ git repository:
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-ndrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-ndrdisc
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-ndrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-ndrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-ndrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-ndrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-ndrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-ndrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-ndrdisc
     40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrdisc
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-ndrdisc
     40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrdisc
 


@@ -117,11 +95,11 @@ git repository:
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-pdrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-pdrdisc
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-pdrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-pdrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-pdrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-pdrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-pdrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-pdrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-pdrdisc
     40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-pdrdisc
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-pdrdisc
     40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-pdrdisc
 


@@ -145,11 +123,11 @@ git repository:
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-pdrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-pdrdisc
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-pdrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-pdrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-pdrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-pdrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-pdrdisc
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-pdrdisc

-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc
+    {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc

     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-pdrdisc
     40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc08-64B-2t2c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-pdrdisc
 
     40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-pdrdisc
     40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc08-64B-2t2c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-pdrdisc
 

diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst
index cd57cbd..bc56f38 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst
@@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP
 running in multiple configurations of VPP worker thread(s), a.k.a. VPP
 data plane thread(s), and their physical CPU core(s) placement.
 
 running in multiple configurations of VPP worker thread(s), a.k.a. VPP
 data plane thread(s), and their physical CPU core(s) placement.
 

-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_. Plotted data set size per
-    test case is equal to the number of job executions presented in this
-    report version: **5**.
-

 NDR Throughput
 ~~~~~~~~~~~~~~
 
 NDR Throughput
 ~~~~~~~~~~~~~~
 

diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst
index 0701ed0..8cd9374 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst
@@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss).  Performance is reported for VPP
 running in multiple configurations of VPP worker thread(s), a.k.a. VPP
 data plane thread(s), and their physical CPU core(s) placement.
 
 running in multiple configurations of VPP worker thread(s), a.k.a. VPP
 data plane thread(s), and their physical CPU core(s) placement.
 

-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_. Plotted data set size per
-    test case is equal to the number of job executions presented in this
-    report version: **5**.
-

 NDR Throughput
 ~~~~~~~~~~~~~~
 
 NDR Throughput
 ~~~~~~~~~~~~~~
 

diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst
index e2b8554..6bcd52f 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst
@@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP
 running in multiple configurations of VPP worker thread(s), a.k.a. VPP
 data plane thread(s), and their physical CPU core(s) placement.
 
 running in multiple configurations of VPP worker thread(s), a.k.a. VPP
 data plane thread(s), and their physical CPU core(s) placement.
 

-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_. Plotted data set size per
-    test case is equal to the number of job executions presented in this
-    report version: **5**.
-

 NDR Throughput
 ~~~~~~~~~~~~~~
 
 NDR Throughput
 ~~~~~~~~~~~~~~
 

diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst
index ebc585e..0baf6dd 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst
@@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP
 running in multiple configurations of VPP worker thread(s), a.k.a. VPP
 data plane thread(s), and their physical CPU core(s) placement.
 
 running in multiple configurations of VPP worker thread(s), a.k.a. VPP
 data plane thread(s), and their physical CPU core(s) placement.
 

-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_. Plotted data set size per
-    test case is equal to the number of job executions presented in this
-    report version: **5**.
-

 NDR Throughput
 ~~~~~~~~~~~~~~
 
 NDR Throughput
 ~~~~~~~~~~~~~~
 

diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst
index 5e8822c..8530911 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst
@@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP
 running in multiple configurations of VPP worker thread(s), a.k.a. VPP
 data plane thread(s), and their physical CPU core(s) placement.
 
 running in multiple configurations of VPP worker thread(s), a.k.a. VPP
 data plane thread(s), and their physical CPU core(s) placement.
 

-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_. Plotted data set size per
-    test case is equal to the number of job executions presented in this
-    report version: **5**.
-

 NDR Throughput
 ~~~~~~~~~~~~~~
 
 NDR Throughput
 ~~~~~~~~~~~~~~
 

diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst
index 818a9de..b932a8e 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst
@@ -8,33 +8,6 @@ packet loss). Performance is reported for VPP running in multiple
 configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s),
 and their physical CPU core(s) placement.
 
 configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s),
 and their physical CPU core(s) placement.
 

-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_. Plotted data set size per
-    test case is equal to the number of job executions presented in this
-    report version: **5**.
-

 NDR Throughput
 ~~~~~~~~~~~~~~
 
 NDR Throughput
 ~~~~~~~~~~~~~~