Code Review / csit.git / blobdiff
? search:
summary | shortlog | log | commit | commitdiff | review | tree
raw | inline | side by side
CSIT-618 Release report update VII
[csit.git] / docs / report / vpp_performance_tests / packet_throughput_graphs / ipsec.rst
diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst
index 6358030..eba3a40 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst
@@ -10,27 +10,6 @@ Performance is reported for VPP running in multiple configurations of
 VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
 physical CPU core(s) placement.
 
-Results are generated by multiple executions of the same CSIT tests.
-In order to display variation in measured throughput values, Box-and-
-whisker plots are used to show their quartiles (Min, 1st quartile / 25th
-percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th
-percentile, Max). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_
-
 NDR Throughput
 ~~~~~~~~~~~~~~
 
@@ -46,21 +25,8 @@ in the graph below.
 CSIT source code for the test cases used for above plots can be found in CSIT
 git repository:
 
-.. code-block:: bash
-
-    $ cd $CSIT/tests/perf
-    $ grep -E "64B-1t1c-.*ipsec.*-ndrdisc" *
-
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-ndrdisc
-    40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrdisc
-
+.. program-output:: cd ../../ && set -x && cd tests/vpp/perf/crypto && grep -E "64B-1t1c-.*ipsec.*-ndrdisc" *
+   :shell:
 
 VPP NDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented
 in the graph below.
@@ -74,20 +40,8 @@ in the graph below.
 CSIT source code for the test cases used for above plots can be found in CSIT
 git repository:
 
-.. code-block:: bash
-
-    $ cd $CSIT/tests/perf
-    $ grep -E "64B-2t2c-.*ipsec.*-ndrdisc" *
-
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-ndrdisc
-    40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrdisc
+.. program-output:: cd ../../ && set -x && cd tests/vpp/perf/crypto && grep -E "64B-2t2c-.*ipsec.*-ndrdisc" *
+   :shell:
 
 PDR Throughput
 ~~~~~~~~~~~~~~
@@ -104,21 +58,8 @@ in the graph below. PDR measured for 0.5% packet loss ratio.
 CSIT source code for the test cases used for above plots can be found in CSIT
 git repository:
 
-.. code-block:: bash
-
-    $ cd $CSIT/tests/perf
-    $ grep -E "64B-1t1c-.*ipsec.*-pdrdisc" *
-
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-pdrdisc
-    40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-pdrdisc
-
+.. program-output:: cd ../../ && set -x && cd tests/vpp/perf/crypto && grep -E "64B-1t1c-.*ipsec.*-pdrdisc" *
+   :shell:
 
 VPP PDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented
 in the graph below. PDR measured for 0.5% packet loss ratio.
@@ -132,18 +73,5 @@ in the graph below. PDR measured for 0.5% packet loss ratio.
 CSIT source code for the test cases used for above plots can be found in CSIT
 git repository:
 
-.. code-block:: bash
-
-    $ cd $CSIT/tests/perf
-    $ grep -E "64B-2t2c-.*ipsec.*-pdrdisc" *
-
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc
-    40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-pdrdisc
-    40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc08-64B-2t2c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-pdrdisc
-
+.. program-output:: cd ../../ && set -x && cd tests/vpp/perf/crypto && grep -E "64B-2t2c-.*ipsec.*-pdrdisc" *
+   :shell:
Integration tests