4 Changes in |csit-release|
5 -------------------------
7 #. **DPDK performance tests**
9 - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
10 configurable trial duration and number of consecutive executions.
11 Coverage of MRR tests has been extended across more test
12 scenarios. MRR tests are used for continuous performance trending
13 and for comparison between DPDK releases.
15 - **MLRsearch Throughput**: MLRsearch algorithm has been introduced
16 for all NDR and PDR throughput tests. All tests that previously
17 used binary search got converted to MLRsearch. Coverage of NDR/PDR
18 tests has been extended across more test scenarios.
20 - **2-Node Tests**: A new set of 2-node tests covering DPDK Testpmd
21 and L3fwd, running on new Xeon Skylake testbeds.
23 - **More NICs**: Increased coverage of NIC models tested: Intel-
24 xxv710-da2, Intel-x710-da4.
26 - **Generated tests**: Simplified and unified test structure, semi-
27 autogenerated by generator script. Test generator is currently
28 able to create test combinations with various frame size and
29 cores combinations. All existing test cases were converted to new
32 #. **Presentation and Analytics Layer**
34 - **Performance trending**: Added continuous performance trending with
35 anomaly detection and analysis for DPDK.
40 Relative performance changes in measured packet throughput in |csit-release|
41 are calculated against the results from |csit-release-1|
42 report. Listed mean and standard deviation values are computed based on
43 a series of the same tests executed against respective DPDK releases to
44 verify test results repeatability, with percentage change calculated for
50 NDR throughput changes between releases are available in a CSV and pretty ASCII
53 - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
54 - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
55 - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
56 - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
60 Test results have been generated by
61 `FD.io test executor dpdk performance job 3n-hsw`_
62 with Robot Framework result
63 files csit-dpdk-perf-|srelease|-\*.zip
64 `archived here <../_static/archive/>`_.
69 PDR throughput changes between releases are available in a CSV and pretty ASCII
72 - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
73 - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
74 - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
75 - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
79 Test results have been generated by
80 `FD.io test executor dpdk performance job 3n-hsw`_
81 with Robot Framework result
82 files csit-dpdk-perf-|srelease|-\*.zip
83 `archived here <../_static/archive/>`_.
88 MRR small packet throughput changes between releases are available in a
89 CSV and pretty ASCII formats:
91 - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-mrr.csv>`_,
92 - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-mrr.csv>`_,
93 - `csv format for 4t4c <../_static/dpdk/performance-changes-4t4c-mrr.csv>`_,
94 - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-mrr.txt>`_,
95 - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-mrr.txt>`_,
96 - `pretty ASCII format for 4t4c <../_static/dpdk/performance-changes-4t4c-mrr.txt>`_.
100 Test results have been generated by
101 `FD.io test executor dpdk performance job 3n-hsw`_,
102 with Robot Framework result
103 files csit-vpp-perf-|srelease|-\*.zip
104 `archived here <../_static/archive/>`_.
106 Comparison Across Testbeds
107 --------------------------
109 Relative performance changes in measured packet throughput on 3-Node Skx testbed
110 are calculated against the results measured on 3-Node Hsw testbed.
115 NDR throughput changes between testbeds are available in a CSV and pretty ASCII
118 - `csv format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
119 - `pretty ASCII format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
123 Test results have been generated by
124 `FD.io test executor dpdk performance job 3n-hsw`_ and
125 `FD.io test executor dpdk performance job 3n-skx`_
126 with Robot Framework result
127 files csit-dpdk-perf-|srelease|-\*.zip
128 `archived here <../_static/archive/>`_.
133 PDR throughput changes between testbeds are available in a CSV and pretty ASCII
136 - `csv format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
137 - `pretty ASCII format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
141 Test results have been generated by
142 `FD.io test executor dpdk performance job 3n-hsw`_ and
143 `FD.io test executor dpdk performance job 3n-skx`_
144 with Robot Framework result
145 files csit-dpdk-perf-|srelease|-\*.zip
146 `archived here <../_static/archive/>`_.
151 MRR throughput changes between testbeds are available in a
152 CSV and pretty ASCII formats:
154 - `csv format for mrr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
155 - `pretty ASCII format for mrr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
159 Test results have been generated by
160 `FD.io test executor dpdk performance job 3n-hsw`_ and
161 `FD.io test executor dpdk performance job 3n-skx`_
162 with Robot Framework result
163 files csit-vpp-perf-|srelease|-\*.zip
164 `archived here <../_static/archive/>`_.
169 Here is the list of known issues in |csit-release| for Testpmd performance tests:
171 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
172 | # | Issue | Jira ID | Description |
173 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
174 | | No known issues | | |
175 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+