X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Freport%2Fvpp_performance_tests%2Fpacket_throughput_graphs%2Fl2.rst;h=5e8822c4a5b66ec8632b36be9e1142f32f43b3e6;hp=89429a6192617f49cd7264aee0ac0491de604198;hb=22f693fb7e67683df3e87791c54c7b48d4e12a0e;hpb=da2aa127ee071e37bc4d6060dd39476179e65365 diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst index 89429a6192..5e8822c4a5 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst @@ -1,190 +1,150 @@ L2 Ethernet Switching ===================== -Following sections provide a summary of VPP Phy-to-Phy L2 Ethernet switching -performance illustrating NDR throughput (zero packet loss) and PDR throughput -(<0.5% packet loss). Performance is reported for VPP running in multiple -configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and -their physical CPU core(s) placement. - -Title of each graph is a regex (regular expression) matching all plotted -test case throughput measurements. +Following sections include summary graphs of VPP Phy-to-Phy performance +with L2 Ethernet switching, including NDR throughput (zero packet loss) +and PDR throughput (<0.5% packet loss). Performance is reported for VPP +running in multiple configurations of VPP worker thread(s), a.k.a. VPP +data plane thread(s), and their physical CPU core(s) placement. + +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. + +*Title of each graph* is a regex (regular expression) matching all +throughput test cases plotted on this graph, *X-axis labels* are indices +of individual test suites executed by csit-vpp-perf-1704-all jobs that +created result output files used as data sources for the graph, *Y-axis +labels* are measured Packets Per Second [pps] values, and the *Graph +legend* lists the plotted test suites and their indices. .. note:: - Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files - <../../_static/archive/>`_. + Test results have been generated by FD.io test executor jobs + `csit-vpp-perf-1704-all + `_, + with Robot Framework result files csit-vpp-perf-1704-all-.zip + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ -VPP NDR Throughput - running in configuration of **one worker thread (1t) on -one physical core (1c)** - is presented in the figure below. +VPP NDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. .. raw:: html - + *Figure 1. VPP 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet Switching.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash - $ csit/tests/perf + $ cd $CSIT/tests/perf $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-dot1ad-l2xcbase-ndrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-dot1q-l2xcbase-ndrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrpdrdisc.robot:| tc01-64B-1t1c-dot1ad-l2xcbase-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrpdrdisc.robot:| tc01-64B-1t1c-dot1q-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-eth-l2xcbase-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2xcbase-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc -VPP NDR Throughput - running in configuration of **two worker threads (2t) on two -physical cores (2c)** - is presented in the figure below. +VPP NDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented +in the graph below. .. raw:: html - + *Figure 2. VPP 2threads 2cores - NDR Throughput for Phy-to-Phy L2 Ethernet Switching.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash - $ csit/tests/perf + $ cd $CSIT/tests/perf $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-dot1ad-l2xcbase-ndrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-dot1q-l2xcbase-ndrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc - -VPP NDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy L2 Ethernet -Switching.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * - - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1ad-l2xcbase-ndrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2xcbase-ndrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrpdrdisc.robot:| tc07-64B-2t2c-dot1ad-l2xcbase-ndrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrpdrdisc.robot:| tc07-64B-2t2c-dot1q-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 10ge2p1x520-eth-l2xcbase-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc + 40ge2p1xl710-eth-l2xcbase-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc PDR Throughput ~~~~~~~~~~~~~~ -VPP PDR Throughput - running in configuration of **one worker thread (1t) on one -physical core (1c)** - is presented in the figure below. PDR at below 0.5% packet -loss ratio. +VPP PDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. .. raw:: html - + -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet Switching.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash - $ csit/tests/perf + $ cd $CSIT/tests/perf $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" * - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-dot1ad-l2xcbase-pdrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-dot1q-l2xcbase-pdrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-pdrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc - 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrpdrdisc.robot:| tc02-64B-1t1c-dot1ad-l2xcbase-pdrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrpdrdisc.robot:| tc02-64B-1t1c-dot1q-l2xcbase-pdrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-eth-l2xcbase-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-pdrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2bdbasemaclrn-pdrdisc -VPP PDR Throughput - running in configuration of **two worker threads (2t) on -two physical cores (2c)** - is presented in the figure below. +VPP PDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. .. raw:: html - + -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet Switching.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash - $ csit/tests/perf + $ cd $CSIT/tests/perf $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" * - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-dot1ad-l2xcbase-pdrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-dot1q-l2xcbase-pdrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-pdrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc - 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc - -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy L2 Ethernet -Switching.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" * - - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1ad-l2xcbase-pdrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1q-l2xcbase-pdrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-pdrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-dot1ad-l2xcbase-ndrpdrdisc.robot:| tc08-64B-2t2c-dot1ad-l2xcbase-pdrdisc + 10ge2p1x520-dot1q-l2xcbase-ndrpdrdisc.robot:| tc08-64B-2t2c-dot1q-l2xcbase-pdrdisc + 10ge2p1x520-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc + 10ge2p1x520-eth-l2xcbase-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-pdrdisc + 10ge2p1x710-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc + 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc