Packet Latency
==============
-Plotted results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Median and Maximum packet
+Latency results are generated from a single execution of NDR discovery
+test across physical testbeds hosted LF FD.io labs: 3n-hsw, 2n-skx, 2n-
+skx. Box plots are used to show the Minimum, Median and Maximum packet
latency per test.
-*Title of each graph* describes the packet path, testbed topology, processor
-architecture, packet size, number of cores and threads used by workers and
-DUT configuration, *X-axis labels* are indices of individual test suites
-executed by FD.io test executor dpdk performance jobs that created result output
-file used as data source for the graph, *Y-axis labels* are measured packet
-Latency [uSec] values, and the *Graph legend* lists the plotted test suites and
-their indices. Latency is reported for concurrent symmetric bi-directional
-flows, separately for each direction: i) West-to-East:
-TGint1-to-SUT1-to-SUT2-to-TGint2, and ii) East-to-West:
-TGint2-to-SUT2-to-SUT1-to-TGint1.
+Additional information about graph data:
+
+#. **Graph Title**: describes tested packet path, testbed topology,
+ processor model, NIC model, packet size, number of cores and
+ threads used by data plane workers and indication of DUT
+ configuration.
+
+#. **X-axis Labels**: indices of individual test suites as listed in
+ Graph Legend and direction of latency flow:
+
+ - West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2.
+ - East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
+
+#. **Y-axis Labels**: measured packet latency values in [uSec].
+
+#. **Graph Legend**: lists X-axis indices with associated CSIT test
+ suites executed to generate graphed test results.
.. note::
Test results have been generated by
`FD.io test executor dpdk performance job 3n-hsw`_,
`FD.io test executor dpdk performance job 3n-skx`_ and
- `FD.io test executor dpdk performance job 2n-skx`_ with Robot Framework
+ `FD.io test executor dpdk performance job 2n-skx`_ with RF
result files csit-dpdk-perf-|srelease|-\*.zip
`archived here <../../_static/archive/>`_.
- Plotted data set size per test case is equal to the number of job executions
- presented in this report version: **10**.
.. toctree::
:maxdepth: 1
Packet Throughput
=================
-Plotted results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution.
-
-For each plotted test case, Box-and-Whisker plots show the quartiles
-(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile /
-mean, 3rd quartile / 75th percentile, Max) across collected data set
-(data set size stated in the note below). Outliers are plotted as
-individual points. Min and max values are plotted as bottom and top
-Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and
-top edge of the box. If multiple samples match only two values, and all
-samples fall between them, then no whiskers are plotted. If all samples
-have the same value, only a horizontal line is plotted.
-
-*Title of each graph* describes the packet path, testbed topology, processor
-architecture, packet size, number of cores and threads used by workers and
-DUT configuration, *X-axis labels* are indices by individual test suites
-executed by FD.io test executor dpdk performance jobs that created result output
-files used as data sources for the graph, *Y-axis labels* are measured Packets
-Per Second [pps] values, and the *Graph legend* lists the plotted test suites
-and their indices.
+Throughput graphs are generated by multiple executions of the same
+performance tests across physical testbeds hosted LF FD.io labs: 3n-hsw,
+2n-skx, 2n-skx. Box-and-Whisker plots are used to display variations in
+measured throughput values, without making any assumptions of the
+underlying statistical distribution.
+
+For each test case, Box-and-Whisker plots show the quartiles (Min, 1st
+quartile / 25th percentile, 2nd quartile / 50th percentile / mean, 3rd
+quartile / 75th percentile, Max) across collected data set. Outliers are
+plotted as individual points.
+
+Additional information about graph data:
+
+#. **Graph Title**: describes tested packet path, testbed topology,
+ processor model, NIC model, packet size, number of cores and
+ threads used by data plane workers and indication of DPDK DUT
+ configuration.
+
+#. **X-axis Labels**: indices of individual test suites as listed in
+ Graph Legend.
+
+#. **Y-axis Labels**: measured Packets Per Second [pps] throughput
+ values.
+
+#. **Graph Legend**: lists X-axis indices with associated CSIT test
+ suites executed to generate graphed test results.
.. note::
Test results have been generated by
`FD.io test executor dpdk performance job 3n-hsw`_,
`FD.io test executor dpdk performance job 3n-skx`_ and
- `FD.io test executor dpdk performance job 2n-skx`_ with Robot Framework
+ `FD.io test executor dpdk performance job 2n-skx`_ with RF
result files csit-dpdk-perf-|srelease|-\*.zip
`archived here <../../_static/archive/>`_.
- Plotted data set size per test case is equal to the number of job executions
- presented in this report version: **10**.
+ Required per test case data set size is **10** and for DPDK tests
+ this is the actual size, as all scheduled test executions completed
+ successfully.
.. toctree::
:maxdepth: 1
HTTP/TCP Performance
====================
-Plotted results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured performance values, without making any
-assumptions of the underlying statistical distribution.
-
-For each plotted test case, Box-and-Whisker plots show the quartiles
-(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile /
-mean, 3rd quartile / 75th percentile, Max) across collected data set
-(data set size stated in the note below). Outliers are plotted as
-individual points. Min and max values are plotted as bottom and top
-Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and
-top edge of the box. If multiple samples match only two values, and all
-samples fall between them, then no whiskers are plotted. If all samples
-have the same value, only a horizontal line is plotted.
+Performance graphs are generated by multiple executions of the same
+performance tests across physical testbeds hosted LF FD.io labs: 3n-hsw.
+Box-and-Whisker plots are used to display variations in measured
+throughput values, without making any assumptions of the underlying
+statistical distribution.
+
+For each test case, Box-and-Whisker plots show the quartiles (Min, 1st
+quartile / 25th percentile, 2nd quartile / 50th percentile / mean, 3rd
+quartile / 75th percentile, Max) across collected data set. Outliers are
+plotted as individual points.
.. note::
Packet Latency
==============
-Plotted results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Median and Maximum packet
+Latency results are generated from a single execution of NDR discovery
+test across physical testbeds hosted LF FD.io labs: 3n-hsw, 2n-skx, 2n-
+skx. Box plots are used to show the Minimum, Median and Maximum packet
latency per test.
-*Title of each graph* describes the packet path, testbed topology, processor
-architecture, packet size, number of cores and threads used by workers and
-DUT configuration, *X-axis labels* are indices of individual test suites
-executed by FD.io test executor vpp performance jobs that created result output
-file used as data source for the graph, *Y-axis labels* are measured packet
-Latency [uSec] values, and the *Graph legend* lists the plotted test suites and
-their indices. Latency is reported for concurrent symmetric bi-directional
-flows, separately for each direction: i) West-to-East:
-TGint1-to-SUT1-to-SUT2-to-TGint2, and ii) East-to-West:
-TGint2-to-SUT2-to-SUT1-to-TGint1.
+Additional information about graph data:
+
+#. **Graph Title**: describes tested packet path, testbed topology,
+ processor model, NIC model, packet size, number of cores and
+ threads used by data plane workers and indication of DUT
+ configuration.
+
+#. **X-axis Labels**: indices of individual test suites as listed in
+ Graph Legend and direction of latency flow:
+
+ - West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2.
+ - East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
+
+#. **Y-axis Labels**: measured packet latency values in [uSec].
+
+#. **Graph Legend**: lists X-axis indices with associated CSIT test
+ suites executed to generate graphed test results.
.. note::
Test results have been generated by
`FD.io test executor vpp performance job 3n-hsw`_,
`FD.io test executor vpp performance job 3n-skx`_ and
- `FD.io test executor vpp performance job 2n-skx`_ with Robot Framework
+ `FD.io test executor vpp performance job 2n-skx`_ with RF
result files csit-vpp-perf-|srelease|-\*.zip
`archived here <../../_static/archive/>`_.
- Plotted data set size per test case is equal to the number of job executions
- presented in this report version: **10**.
.. toctree::
:maxdepth: 1
Packet Throughput
=================
-Plotted results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution.
-
-For each plotted test case, Box-and-Whisker plots show the quartiles
-(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile /
-mean, 3rd quartile / 75th percentile, Max) across collected data set
-(data set size stated in the note below). Outliers are plotted as
-individual points. Min and max values are plotted as bottom and top
-Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and
-top edge of the box. If multiple samples match only two values, and all
-samples fall between them, then no whiskers are plotted. If all samples
-have the same value, only a horizontal line is plotted.
-
-*Title of each graph* describes the packet path, testbed topology, processor
-architecture, packet size, number of cores and threads used by workers and
-DUT configuration, *X-axis labels* are indices of individual test suites
-executed by FD.io test executor vpp performance jobs that created result output
-files used as data sources for the graph, *Y-axis labels* are measured Packets
-Per Second [pps] values, and the *Graph legend* lists the plotted test suites
-and their indices.
+Throughput graphs are generated by multiple executions of the same
+performance tests across physical testbeds hosted LF FD.io labs: 3n-hsw,
+2n-skx, 2n-skx. Box-and-Whisker plots are used to display variations in
+measured throughput values, without making any assumptions of the
+underlying statistical distribution.
+
+For each test case, Box-and-Whisker plots show the quartiles (Min, 1st
+quartile / 25th percentile, 2nd quartile / 50th percentile / mean, 3rd
+quartile / 75th percentile, Max) across collected data set. Outliers are
+plotted as individual points.
+
+Additional information about graph data:
+
+#. **Graph Title**: describes tested packet path, testbed topology,
+ processor model, NIC model, packet size, number of cores and
+ threads used by data plane workers and indication of VPP DUT
+ configuration.
+
+#. **X-axis Labels**: indices of individual test suites as listed in
+ Graph Legend.
+
+#. **Y-axis Labels**: measured Packets Per Second [pps] throughput
+ values.
+
+#. **Graph Legend**: lists X-axis indices with associated CSIT test
+ suites executed to generate graphed test results.
.. note::
Test results have been generated by
`FD.io test executor vpp performance job 3n-hsw`_,
`FD.io test executor vpp performance job 3n-skx`_ and
- `FD.io test executor vpp performance job 2n-skx`_ with Robot Framework
+ `FD.io test executor vpp performance job 2n-skx`_ with RF
result files csit-vpp-perf-|srelease|-\*.zip
`archived here <../../_static/archive/>`_.
- Plotted data set size per test case is equal to the number of job executions
- presented in this report version: **10**.
+ Required per test case data set size is **10**, but for VPP tests
+ the actual size varies per test case and is <=10. This is due to
+ not all test executions completing successfully due to a known
+ `VPP-1361 <https://jira.fd.io/browse/VPP-1361>`_ issue
+ sporadically impacting physical interface up/down state.
.. toctree::
:maxdepth: 1
Speedup Multi-Core
==================
-Throughput Speedup Multi-Core graphs are generated based on results from
-multiple executions of the same CSIT performance tests across the three
-physical testbeds within LF FD.io labs. Plotted grouped bars illustrate
-the 64B/78B packet throughput speedup ratio for 2- and 4-core multi-
+Speedup Multi-Core throughput graphs are generated by multiple
+executions of the same performance tests across physical testbeds hosted
+LF FD.io labs: 3n-hsw, 2n-skx, 2n-skx. Grouped bars illustrate the
+64B/78B packet throughput speedup ratio for 2- and 4-core multi-
threaded VPP configurations relative to 1-core configurations.
.. note::
Test results have been generated by
`FD.io test executor vpp performance job 3n-hsw`_,
`FD.io test executor vpp performance job 3n-skx`_ and
- `FD.io test executor vpp performance job 2n-skx`_ with Robot Framework
+ `FD.io test executor vpp performance job 2n-skx`_ with RF
result files csit-vpp-perf-|srelease|-\*.zip
`archived here <../../_static/archive/>`_.
- Plotted data set size per test case is equal to the number of job executions
- presented in this report version: **10**.
+ Required per test case data set size is **10**, but for VPP tests
+ the actual size varies per test case and is <=10. This is due to
+ not all test executions completing successfully due to a known
+ `VPP-1361 <https://jira.fd.io/browse/VPP-1361>`_ issue
+ sporadically impacting physical interface up/down state.
.. toctree::
:maxdepth: 1
Code Review / csit.git / commitdiff