CSIT-1203: Add parsing of NDRPDR test

[csit.git] / docs / report / vpp_performance_tests / packet_latency_graphs / vm_vhost.rst

diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst
index 2e3071d..dbeca9e 100644 (file)
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst
@@ -1,5 +1,5 @@
-VM vhost Connections
-====================
+KVM VMs vhost-user
+==================

 
 This section includes summary graphs of VPP Phy-to-VM(s)-to-Phy packet
 latency with with VM virtio and VPP vhost-user virtual interfaces
 
 This section includes summary graphs of VPP Phy-to-VM(s)-to-Phy packet
 latency with with VM virtio and VPP vhost-user virtual interfaces


@@ -8,87 +8,314 @@ for VPP running in multiple configurations of VPP worker thread(s),
 a.k.a. VPP data plane thread(s), and their physical CPU core(s)
 placement.
 
 a.k.a. VPP data plane thread(s), and their physical CPU core(s)
 placement.
 

-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
+CSIT source code for the test cases used for plots can be found in
+`CSIT git repository <https://git.fd.io/csit/tree/tests/vpp/perf/vm_vhost?h=rls1807>`_.

 
 

-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
+3n-hsw-x520
+~~~~~~~~~~~

 
 

-.. note::
+64b-1t1c-base_and_scale-l2sw
+----------------------------

 
 

-    Test results have been generated by FD.io test executor jobs
-    `csit-vpp-perf-1704-all
-    <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
-    with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
-    `archived here <../../_static/archive/>`_
+.. raw:: html
+
+    <center><b>
+
+:index:`Latency: vhost-l2sw-3n-hsw-x520-64b-1t1c-base_and_scale-ndr`
+
+.. raw:: html
+
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-l2sw-3n-hsw-x520-64b-1t1c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-l2sw-3n-hsw-x520-64b-1t1c-base_and_scale-ndr-lat}
+            \label{fig:vhost-l2sw-3n-hsw-x520-64b-1t1c-base_and_scale-ndr-lat}
+    \end{figure}
+
+64b-2t2c-base_and_scale-l2sw
+----------------------------
+
+.. raw:: html
+
+    <center><b>
+
+:index:`Latency: vhost-l2sw-3n-hsw-x520-64b-2t2c-base_and_scale-ndr`

 
 

-VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
+.. raw:: html
+
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-l2sw-3n-hsw-x520-64b-2t2c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-l2sw-3n-hsw-x520-64b-2t2c-base_and_scale-ndr-lat}
+            \label{fig:vhost-l2sw-3n-hsw-x520-64b-2t2c-base_and_scale-ndr-lat}
+    \end{figure}
+
+64b-1t1c-base_and_scale-ip4
+---------------------------

 
 .. raw:: html
 
 
 .. raw:: html
 

-    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-vhost-ndrdisc-lat50.html"></iframe>
+    <center><b>

 
 

-*Figure 1. VPP 1thread 1core - packet latency for Phy-to-VM-to-Phy VM vhost-user.*
+:index:`Latency: vhost-ip4-3n-hsw-x520-64b-1t1c-base_and_scale-ndr`

 
 

-CSIT source code for the test cases used for above plots can be found in CSIT
-git repository:
+.. raw:: html

 
 

-.. code-block:: bash
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-ip4-3n-hsw-x520-64b-1t1c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>

 
 

-    $ cd $CSIT/tests/perf
-    $ grep -E "64B-1t1c-.*vhost.*-ndrdisc" *
+.. raw:: latex

 
 

-    10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-eth-4vhost-2vm-ndrdisc
-    10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc
-    10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-4vhost-2vm-ndrdisc
-    10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-    40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-ip4base-eth-4vhost-2vm-ndrdisc
-    40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-    40ge2p1xl710-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc
-    40ge2p1xl710-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-4vhost-2vm-ndrdisc
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-ip4-3n-hsw-x520-64b-1t1c-base_and_scale-ndr-lat}
+            \label{fig:vhost-ip4-3n-hsw-x520-64b-1t1c-base_and_scale-ndr-lat}
+    \end{figure}

 
 

-VPP packet latency in 2t2c setup (2thread, 2core) is presented in the graph below.
+64b-2t2c-base_and_scale-ip4
+---------------------------

 
 .. raw:: html
 
 
 .. raw:: html
 

-    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-vhost-ndrdisc-lat50.html"></iframe>
+    <center><b>
+
+:index:`Latency: vhost-ip4-3n-hsw-x520-64b-2t2c-base_and_scale-ndr`
+
+.. raw:: html
+
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-ip4-3n-hsw-x520-64b-2t2c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-ip4-3n-hsw-x520-64b-2t2c-base_and_scale-ndr-lat}
+            \label{fig:vhost-ip4-3n-hsw-x520-64b-2t2c-base_and_scale-ndr-lat}
+    \end{figure}
+
+3n-hsw-x710
+~~~~~~~~~~~

 
 

-*Figure 2. VPP 2threads 2cores - packet latency for Phy-to-VM-to-Phy VM vhost-user.*
+64b-1t1c-base_and_scale-l2sw
+----------------------------

 
 

-CSIT source code for the test cases used for above plots can be found in CSIT
-git repository:
+.. raw:: html
+
+    <center><b>
+
+:index:`Latency: vhost-l2sw-3n-hsw-x710-64b-1t1c-base_and_scale-ndr`
+
+.. raw:: html
+
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-l2sw-3n-hsw-x710-64b-1t1c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-l2sw-3n-hsw-x710-64b-1t1c-base_and_scale-ndr-lat}
+            \label{fig:vhost-l2sw-3n-hsw-x710-64b-1t1c-base_and_scale-ndr-lat}
+    \end{figure}
+
+64b-2t2c-base_and_scale-l2sw
+----------------------------
+
+.. raw:: html
+
+    <center><b>
+
+:index:`Latency: vhost-l2sw-3n-hsw-x710-64b-2t2c-base_and_scale-ndr`
+
+.. raw:: html
+
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-l2sw-3n-hsw-x710-64b-2t2c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-l2sw-3n-hsw-x710-64b-2t2c-base_and_scale-ndr-lat}
+            \label{fig:vhost-l2sw-3n-hsw-x710-64b-2t2c-base_and_scale-ndr-lat}
+    \end{figure}
+
+64b-1t1c-base_and_scale-ip4
+---------------------------
+
+.. raw:: html
+
+    <center><b>
+
+:index:`Latency: vhost-ip4-3n-hsw-x710-64b-1t1c-base_and_scale-ndr`
+
+.. raw:: html

 
 

-.. code-block:: bash
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-ip4-3n-hsw-x710-64b-1t1c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-ip4-3n-hsw-x710-64b-1t1c-base_and_scale-ndr-lat}
+            \label{fig:vhost-ip4-3n-hsw-x710-64b-1t1c-base_and_scale-ndr-lat}
+    \end{figure}
+
+64b-2t2c-base_and_scale-ip4
+---------------------------
+
+.. raw:: html
+
+    <center><b>
+
+:index:`Latency: vhost-ip4-3n-hsw-x710-64b-2t2c-base_and_scale-ndr`
+
+.. raw:: html
+
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-ip4-3n-hsw-x710-64b-2t2c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-ip4-3n-hsw-x710-64b-2t2c-base_and_scale-ndr-lat}
+            \label{fig:vhost-ip4-3n-hsw-x710-64b-2t2c-base_and_scale-ndr-lat}
+    \end{figure}
+
+3n-hsw-xl710
+~~~~~~~~~~~~
+
+64b-1t1c-base_and_scale-l2sw
+----------------------------
+
+.. raw:: html
+
+    <center><b>
+
+:index:`Latency: vhost-l2sw-3n-hsw-xl710-64b-1t1c-base_and_scale-ndr`
+
+.. raw:: html
+
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-l2sw-3n-hsw-xl710-64b-1t1c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-l2sw-3n-hsw-xl710-64b-1t1c-base_and_scale-ndr-lat}
+            \label{fig:vhost-l2sw-3n-hsw-xl710-64b-1t1c-base_and_scale-ndr-lat}
+    \end{figure}
+
+64b-2t2c-base_and_scale-l2sw
+----------------------------
+
+.. raw:: html
+
+    <center><b>
+
+:index:`Latency: vhost-l2sw-3n-hsw-xl710-64b-2t2c-base_and_scale-ndr`
+
+.. raw:: html
+
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-l2sw-3n-hsw-xl710-64b-2t2c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-l2sw-3n-hsw-xl710-64b-2t2c-base_and_scale-ndr-lat}
+            \label{fig:vhost-l2sw-3n-hsw-xl710-64b-2t2c-base_and_scale-ndr-lat}
+    \end{figure}
+
+64b-1t1c-base_and_scale-ip4
+---------------------------
+
+.. raw:: html
+
+    <center><b>
+
+:index:`Latency: vhost-ip4-3n-hsw-xl710-64b-1t1c-base_and_scale-ndr`
+
+.. raw:: html
+
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-ip4-3n-hsw-xl710-64b-1t1c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-ip4-3n-hsw-xl710-64b-1t1c-base_and_scale-ndr-lat}
+            \label{fig:vhost-ip4-3n-hsw-xl710-64b-1t1c-base_and_scale-ndr-lat}
+    \end{figure}
+
+64b-2t2c-base_and_scale-ip4
+---------------------------
+
+.. raw:: html
+
+    <center><b>
+
+:index:`Latency: vhost-ip4-3n-hsw-xl710-64b-2t2c-base_and_scale-ndr`
+
+.. raw:: html

 
 

-    $ cd $CSIT/tests/perf
-    $ grep -E "64B-2t2c-.*vhost.*-ndrdisc" *
+    </b>
+    <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/vhost-ip4-3n-hsw-xl710-64b-2t2c-base_and_scale-ndr-lat.html"></iframe>
+    <p><br><br></p>
+    </center>

 
 

-    10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-eth-4vhost-2vm-ndrdisc
-    10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc
-    10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
-    10ge2p1x520-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-4vhost-2vm-ndrdisc
-    10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-    40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-ip4base-eth-4vhost-2vm-ndrdisc
-    40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-    40ge2p1xl710-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc
-    40ge2p1xl710-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-4vhost-2vm-ndrdisc
+.. raw:: latex

 
 

+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/vpp/}}
+            \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{vhost-ip4-3n-hsw-xl710-64b-2t2c-base_and_scale-ndr-lat}
+            \label{fig:vhost-ip4-3n-hsw-xl710-64b-2t2c-base_and_scale-ndr-lat}
+    \end{figure}