From: Maciek Konstantynowicz Date: Thu, 27 Apr 2017 11:11:56 +0000 (+0100) Subject: csit rls1704 report: final update to csit_release_notes.rst, packet_*_graphs.rst... X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=commitdiff_plain;h=fa8714742330382e764419c0432099702f182da1 csit rls1704 report: final update to csit_release_notes.rst, packet_*_graphs.rst files. Change-Id: I531706f87716a3db8c5417be6ce4933e02e2a622 Signed-off-by: Maciek Konstantynowicz --- diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 8aa4533972..8ab00d7e8e 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -74,6 +74,8 @@ Non-Drop Rate Throughput discovery tests: +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+ | IPv6 iAcl | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-ndrdisc | 6.5 | 6.1 | 6.9 | 13% | +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+ +| IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-ndrdisc | 5.3 | 4.2 | 4.6 | 10% | ++-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+ PDR Throughput ~~~~~~~~~~~~~~ @@ -98,6 +100,8 @@ Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%: +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+ | IPv6 iAcl | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-pdrdisc | 6.5 | 6.1 | 6.9 | 13% | +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+ +| IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-pdrdisc | 5.3 | 4.2 | 4.6 | 10% | ++-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+ Measured improvements are in line with VPP code optimizations listed in `VPP-17.04 release notes @@ -156,8 +160,6 @@ Non-Drop Rate Throughput discovery tests: +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+ | IPv6 FIB 20k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-ndrdisc | 6.9 | 6.5 | 6.9 | 6% | +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+ -| IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-ndrdisc | 5.3 | 4.2 | 4.6 | 10% | -+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+ PDR Throughput ~~~~~~~~~~~~~~ @@ -196,8 +198,6 @@ Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%: +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+ | IPv6 FIB 20k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-pdrdisc | 6.9 | 6.5 | 6.9 | 6% | +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+ -| IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-pdrdisc | 5.3 | 4.2 | 4.6 | 10% | -+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+ Known Issues ------------ diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst index 82f297ee92..0c506f720c 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst @@ -30,7 +30,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst index fb85fe9464..5ddf813cea 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst @@ -27,7 +27,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst index 42f5ced1cd..ab39a97704 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst @@ -27,7 +27,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst index 7d4cd70885..93c1befe0d 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst @@ -27,7 +27,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst index c54509d3a3..186bbfd386 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst @@ -27,7 +27,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst index f7836feeef..a017a42773 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst @@ -27,7 +27,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst index 2e3071d6d8..d5010289d6 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst @@ -28,7 +28,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst index 63580303ff..ad71a8a07c 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst @@ -10,11 +10,15 @@ Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -29,7 +33,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst index be4af20e84..cd57cbd94c 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst @@ -7,11 +7,15 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -26,7 +30,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst index f99c0a502e..0701ed06ba 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst @@ -7,11 +7,15 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -26,7 +30,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst index c475ab8f18..e2b8554036 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst @@ -7,11 +7,15 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -26,7 +30,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst index 79e77445dd..ebc585ea59 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst @@ -7,11 +7,15 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Rresults are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -26,7 +30,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst index d9a69015e4..5344dd7d51 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst @@ -7,11 +7,15 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -26,7 +30,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst index e4c6af9f6b..818a9de78c 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst @@ -8,11 +8,15 @@ packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -27,7 +31,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all `_, with Robot Framework result files csit-vpp-perf-1704-all-.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~