4 Changes in |csit-release|
5 -------------------------
7 No code changes apart from bug fixes.
12 Relative performance changes in measured packet throughput in |csit-release|
13 are calculated against the results from |csit-release-1|
14 report. Listed mean and standard deviation values are computed based on
15 a series of the same tests executed against respective VPP releases to
16 verify test results repeatability, with percentage change calculated for
17 mean values. Note that the standard deviation is quite high for a small
18 number of packet throughput tests, what indicates poor test results
19 repeatability and makes the relative change of mean throughput value not
20 fully representative for these tests. The root causes behind poor
21 results repeatability vary between the test cases.
26 NDR throughput changes between releases are available in a
27 CSV and pretty ASCII formats:
29 - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
30 - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
31 - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
32 - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
36 Test results have been generated by
37 `FD.io test executor dpdk performance job 3n-hsw`_
38 with Robot Framework result
39 files csit-vpp-perf-|srelease|-\*.zip
40 `archived here <../_static/archive/>`_.
45 PDR throughput changes between releases are available in a
46 CSV and pretty ASCII formats:
48 - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
49 - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
50 - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
51 - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
55 Test results have been generated by
56 `FD.io test executor dpdk performance job 3n-hsw`_
57 with Robot Framework result
58 files csit-vpp-perf-|srelease|-\*.zip
59 `archived here <../_static/archive/>`_.
61 Comparison Across Testbeds
62 --------------------------
64 Relative performance changes in measured packet throughputon 3-Node Skx testbed
65 are calculated against the results measured on 3-Node Hsw testbed.
70 NDR changes between testbeds are available in a
71 CSV and pretty ASCII formats:
73 - `csv format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
74 - `pretty ASCII format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
78 Test results have been generated by
79 `FD.io test executor dpdk performance job 3n-hsw`_ and
80 `FD.io test executor dpdk performance job 3n-skx`_
81 with Robot Framework result
82 files csit-vpp-perf-|srelease|-\*.zip
83 `archived here <../_static/archive/>`_.
88 PDR throughput changes between testbeds are available in a
89 CSV and pretty ASCII formats:
91 - `csv format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
92 - `pretty ASCII format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
96 Test results have been generated by
97 `FD.io test executor dpdk performance job 3n-hsw`_ and
98 `FD.io test executor dpdk performance job 3n-skx`_
99 with Robot Framework result
100 files csit-vpp-perf-|srelease|-\*.zip
101 `archived here <../_static/archive/>`_.
106 Here is the list of known issues in |csit-release| for Testpmd performance tests:
108 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
109 | # | Issue | Jira ID | Description |
110 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
111 | 1 | Testpmd in 1t1c and 2t2c setups - large variation | CSIT-569 | Suspected NIC firmware or DPDK driver issue affecting NDR |
112 | | of discovered NDR throughput values across | | throughput. Applies to XL710 and X710 NICs, no issues observed |
113 | | multiple test runs with xl710 and x710 NICs. | | on x520 NICs. |
114 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
115 | 2 | Lower than expected NDR throughput with xl710 | CSIT-571 | Suspected NIC firmware or DPDK driver issue affecting NDR |
116 | | and x710 NICs, compared to x520 NICs. | | throughput. Applies to XL710 and X710 NICs. |
117 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+