4 Changes in |csit-release|
5 -------------------------
7 #. **DPDK performance tests**
9 - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
10 configurable trial duration and number of consecutive executions.
11 Coverage of MRR tests has been extended across more test
12 scenarios. MRR tests are used for continuous performance trending
13 and for comparison between DPDK releases.
15 - **MLRsearch Throughput**: MLRsearch algorithm has been introduced
16 for all NDR and PDR throughput tests. All tests that previously
17 used binary search got converted to MLRsearch. Coverage of NDR/PDR
18 tests has been extended across more test scenarios.
20 - **2-Node Tests**: A new set of 2-node tests covering DPDK Testpmd
21 and L3fwd, running on new Xeon Skylake testbeds.
23 - **More NICs**: Increased coverage of NIC models tested: Intel-
24 xxv710-da2, Intel-x710-da4.
26 - **Generated tests**: Simplified and unified test structure, semi-
27 autogenerated by generator script. Test generator is currently
28 able to create test combinations with various frame size and
29 cores combinations. All existing test cases were converted to new
32 #. **Presentation and Analytics Layer**
34 - **Performance trending**: Added continuous performance trending with
35 anomaly detection and analysis for DPDK.
40 Relative performance changes in measured NDR, PDR and MRR packet
41 throughput in |csit-release| are calculated against the test results
42 from |csit-release-1| report, for tests running on 3-Node Intel Xeon
43 Haswell testbeds (3n-hsw) in 1-core, 2-core and 4-core (MRR only)
46 Listed mean and standard deviation values are computed based on a series
47 of the same tests executed against respective VPP releases to verify
48 test results repeatability, with percentage change calculated for mean
54 NDR throughput changes between releases are available in CSV and pretty
57 - `CSV 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
58 - `CSV 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
59 - `ASCII 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
60 - `ASCII 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
64 Test results have been generated by
65 `FD.io test executor dpdk performance job 3n-hsw`_
67 files csit-dpdk-perf-|srelease|-\*.zip
68 `archived here <../_static/archive/>`_.
73 PDR throughput changes between releases are available in CSV and pretty
76 - `CSV 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
77 - `CSV 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
78 - `ASCII 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
79 - `ASCII 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
83 Test results have been generated by
84 `FD.io test executor dpdk performance job 3n-hsw`_
86 files csit-dpdk-perf-|srelease|-\*.zip
87 `archived here <../_static/archive/>`_.
92 MRR throughput changes between releases are available in CSV and pretty
95 - `CSV 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.csv>`_,
96 - `CSV 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.csv>`_,
97 - `CSV 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.csv>`_,
98 - `ASCII 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.txt>`_,
99 - `ASCII 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.txt>`_,
100 - `ASCII 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.txt>`_.
104 Test results have been generated by
105 `FD.io test executor dpdk performance job 3n-hsw`_,
107 files csit-vpp-perf-|srelease|-\*.zip
108 `archived here <../_static/archive/>`_.
110 Skx vs. Hsw Comparison
111 ----------------------
113 Relative performance changes in measured packet throughput on 3-Node Skx testbed
114 are calculated against the results measured on 3-Node Hsw testbed.
119 NDR throughput changes between testbeds are available in a CSV and pretty ASCII
122 - `CSV 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
123 - `ASCII 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
127 Test results have been generated by
128 `FD.io test executor dpdk performance job 3n-hsw`_ and
129 `FD.io test executor dpdk performance job 3n-skx`_
131 files csit-dpdk-perf-|srelease|-\*.zip
132 `archived here <../_static/archive/>`_.
137 PDR throughput changes between testbeds are available in a CSV and pretty ASCII
140 - `CSV 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
141 - `ASCII 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
145 Test results have been generated by
146 `FD.io test executor dpdk performance job 3n-hsw`_ and
147 `FD.io test executor dpdk performance job 3n-skx`_
149 files csit-dpdk-perf-|srelease|-\*.zip
150 `archived here <../_static/archive/>`_.
155 MRR throughput changes between testbeds are available in a
156 CSV and pretty ASCII formats:
158 - `CSV 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
159 - `ASCII 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
163 Test results have been generated by
164 `FD.io test executor dpdk performance job 3n-hsw`_ and
165 `FD.io test executor dpdk performance job 3n-skx`_
167 files csit-vpp-perf-|srelease|-\*.zip
168 `archived here <../_static/archive/>`_.
173 Here is the list of known issues in |csit-release| for Testpmd performance tests:
175 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
176 | # | Issue | Jira ID | Description |
177 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
178 | | No known issues | | |
179 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+