X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Freport%2Fvpp_performance_tests%2Fpacket_throughput_graphs%2Fvm_vhost.rst;h=e4c6af9f6bd6bfc4800e9c91735fb58ef815bd13;hp=ce2e12961ca3ffe678d546ef653d1b894940ac80;hb=refs%2Fchanges%2F46%2F6346%2F11;hpb=0fc813b1694a6ae70b759e7ca96741f21f81b051 diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst index ce2e12961c..e4c6af9f6b 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst @@ -1,31 +1,39 @@ VM vhost Connections ==================== -Following sections provide a summary of VPP Phy-to-VM-to-Phy VM vhost-user -performance illustrating NDR throughput (zero packet loss) and PDR throughput -(<0.5% packet loss). Performance is reported for VPP running in multiple -configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and -their physical CPU core(s) placement. - -*Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs -that created result output files used as data sources for the graph, -*Y-axis labels* are measured Packets Per Second [pps] values, and the *graph -legend* identifes the plotted test suites. +Following sections include summary graphs of VPP Phy-to-VM(s)-to-Phy +performance with VM virtio and VPP vhost-user virtual interfaces, +including NDR throughput (zero packet loss) and PDR throughput (<0.5% +packet loss). Performance is reported for VPP running in multiple +configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), +and their physical CPU core(s) placement. + +Results are generated by multiple executions of the same CSIT tests. +In order to display variation in measured throughput values, Box-and- +whisker plots are used to show their quartiles (Min, 1st quartile / 25th +percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th +percentile, Max). Outliers are plotted as individual points. + +*Title of each graph* is a regex (regular expression) matching all +throughput test cases plotted on this graph, *X-axis labels* are indices +of individual test suites executed by csit-vpp-perf-1704-all jobs that +created result output files used as data sources for the graph, *Y-axis +labels* are measured Packets Per Second [pps] values, and the *Graph +legend* lists the plotted test suites and their indices. .. note:: - Data sources for reported test results: i) FD.io test executor jobs + Test results have been generated by FD.io test executor jobs `csit-vpp-perf-1704-all - `_ , - ii) archived FD.io jobs test result `output files - <../../_static/archive/>`_. + `_, + with Robot Framework result files csit-vpp-perf-1704-all-.zip + `archived here <../../_static/archive/>`_ NDR Throughput ~~~~~~~~~~~~~~ -VPP NDR Throughput - running in configuration of **one worker thread (1t) on one -physical core (1c)** - is presented in the figure below. +VPP NDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. .. raw:: html @@ -34,31 +42,31 @@ physical core (1c)** - is presented in the figure below. *Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-VM-to-Phy VM vhost-user vhost-user.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash - $ csit/tests/perf + $ cd $CSIT/tests/perf $ grep -E "64B-1t1c-.*vhost.*-ndrdisc" * 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-eth-4vhost-2vm-ndrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 10ge2p1x520-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc 10ge2p1x520-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-4vhost-2vm-ndrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4-ip4base-eth-4vhost-2vm-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-ip4base-eth-4vhost-2vm-ndrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc 40ge2p1xl710-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-eth-4vhost-2vm-ndrdisc + 40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-ip4base-eth-4vhost-2vm-ndrdisc -VPP NDR Throughput - running in configuration of **two worker threads (2t) on -two physical cores (2c)** - is presented in the figure below. +VPP NDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented +in the graph below. .. raw:: html @@ -67,35 +75,34 @@ two physical cores (2c)** - is presented in the figure below. *Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-VM-to-Phy VM vhost-user vhost-user.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash - $ csit/tests/perf + $ cd $CSIT/tests/perf $ grep -E "64B-2t2c-.*vhost.*-ndrdisc" * 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-eth-4vhost-2vm-ndrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 10ge2p1x520-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc 10ge2p1x520-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-4vhost-2vm-ndrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc + 10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-eth-4vhost-2vm-ndrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-ip4base-eth-4vhost-2vm-ndrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc 40ge2p1xl710-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-eth-4vhost-2vm-ndrdisc + 40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-ip4base-eth-4vhost-2vm-ndrdisc PDR Throughput ~~~~~~~~~~~~~~ -VPP PDR Throughput - running in configuration of **one worker thread (1t) on one -physical core (1c)** - is presented in the figure below. PDR at below 0.5% -packet loss ratio. +VPP PDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. .. raw:: html @@ -104,31 +111,31 @@ packet loss ratio. *Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-VM-to-Phy VM vhost-user vhost-user.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash - $ csit/tests/perf + $ cd $CSIT/tests/perf $ grep -E "64B-1t1c-.*vhost.*-pdrdisc" * 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc02-64B-1t1c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-eth-4vhost-2vm-pdrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 10ge2p1x520-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-eth-4vhost-2vm-pdrdisc 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc 10ge2p1x520-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-eth-4vhost-2vm-pdrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4-ip4base-eth-4vhost-2vm-pdrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - 40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-ip4base-eth-4vhost-2vm-pdrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-eth-4vhost-2vm-pdrdisc 40ge2p1xl710-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-eth-4vhost-2vm-pdrdisc + 40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-ip4base-eth-4vhost-2vm-pdrdisc -VPP PDR Throughput - running in configuration of **two worker threads (2t) on -two physical cores (2c)** - is presented in the figure below. +VPP PDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. .. raw:: html @@ -137,26 +144,26 @@ two physical cores (2c)** - is presented in the figure below. *Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-VM-to-Phy VM vhost-user vhost-user.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash - $ csit/tests/perf + $ cd $CSIT/tests/perf $ grep -E "64B-2t2c-.*vhost.*-pdrdisc" * 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc08-64B-2t2c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-eth-4vhost-2vm-pdrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 10ge2p1x520-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-4vhost-2vm-pdrdisc 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc 10ge2p1x520-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-eth-4vhost-2vm-pdrdisc + 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc + 10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc08-64B-2t2c-ethip4-ip4base-eth-4vhost-2vm-pdrdisc + 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - 40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-ip4base-eth-4vhost-2vm-pdrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-4vhost-2vm-pdrdisc 40ge2p1xl710-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-eth-4vhost-2vm-pdrdisc + 40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-ip4base-eth-4vhost-2vm-pdrdisc