4 Changes in |csit-release|
5 -------------------------
7 #. **DPDK release version change**
9 - |csit-release| tested DPDK-18.05, as used by |vpp-release|.
11 #. **DPDK performance tests**
13 - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
14 configurable trial duration and number of consecutive executions.
15 Coverage of MRR tests has been extended across more test
16 scenarios. MRR tests are used for continuous performance trending
17 and for comparison between DPDK releases.
19 - **MLRsearch Throughput**: MLRsearch algorithm has been introduced
20 for all NDR and PDR throughput tests. All tests that previously
21 used binary search got converted to MLRsearch. Coverage of NDR/PDR
22 tests has been extended across more test scenarios.
24 - **2-Node Tests**: A new set of 2-node tests covering DPDK Testpmd
25 and L3fwd, running on new Xeon Skylake testbeds.
27 - **More NICs**: Increased coverage of NIC models tested: Intel-
28 xxv710-da2, Intel-x710-da4.
30 - **Generated tests**: Simplified and unified test structure, semi-
31 autogenerated by generator script. Test generator is currently
32 able to create test combinations with various frame size and
33 cores combinations. All existing test cases were converted to new
36 #. **Presentation and Analytics Layer**
38 - **Performance trending**: Added continuous performance trending with
39 anomaly detection and analysis for DPDK.
44 Relative performance changes in measured NDR, PDR and MRR packet
45 throughput in |csit-release| are calculated against the test results
46 from |csit-release-1| report, for tests running on 3-Node Intel Xeon
47 Haswell testbeds (3n-hsw) in 1-core, 2-core and 4-core (MRR only)
50 Listed mean and standard deviation values are computed based on a series
51 of the same tests executed against respective VPP releases to verify
52 test results repeatability, with percentage change calculated for mean
58 NDR throughput changes between releases are available in CSV and pretty
61 - `CSV 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
62 - `CSV 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
63 - `ASCII 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
64 - `ASCII 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
68 Test results have been generated by
69 `FD.io test executor dpdk performance job 3n-hsw`_
71 files csit-dpdk-perf-|srelease|-\*.zip
72 `archived here <../_static/archive/>`_.
77 PDR throughput changes between releases are available in CSV and pretty
80 - `CSV 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
81 - `CSV 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
82 - `ASCII 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
83 - `ASCII 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
87 Test results have been generated by
88 `FD.io test executor dpdk performance job 3n-hsw`_
90 files csit-dpdk-perf-|srelease|-\*.zip
91 `archived here <../_static/archive/>`_.
96 MRR throughput changes between releases are available in CSV and pretty
99 - `CSV 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.csv>`_,
100 - `CSV 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.csv>`_,
101 - `CSV 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.csv>`_,
102 - `ASCII 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.txt>`_,
103 - `ASCII 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.txt>`_,
104 - `ASCII 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.txt>`_.
108 Test results have been generated by
109 `FD.io test executor dpdk performance job 3n-hsw`_,
111 files csit-vpp-perf-|srelease|-\*.zip
112 `archived here <../_static/archive/>`_.
114 Skx vs. Hsw Comparison
115 ----------------------
117 Relative performance changes in measured packet throughput on 3-Node Skx testbed
118 are calculated against the results measured on 3-Node Hsw testbed.
123 NDR throughput changes between testbeds are available in a CSV and pretty ASCII
126 - `CSV 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
127 - `ASCII 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
131 Test results have been generated by
132 `FD.io test executor dpdk performance job 3n-hsw`_ and
133 `FD.io test executor dpdk performance job 3n-skx`_
135 files csit-dpdk-perf-|srelease|-\*.zip
136 `archived here <../_static/archive/>`_.
141 PDR throughput changes between testbeds are available in a CSV and pretty ASCII
144 - `CSV 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
145 - `ASCII 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
149 Test results have been generated by
150 `FD.io test executor dpdk performance job 3n-hsw`_ and
151 `FD.io test executor dpdk performance job 3n-skx`_
153 files csit-dpdk-perf-|srelease|-\*.zip
154 `archived here <../_static/archive/>`_.
159 MRR throughput changes between testbeds are available in a
160 CSV and pretty ASCII formats:
162 - `CSV 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
163 - `ASCII 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
167 Test results have been generated by
168 `FD.io test executor dpdk performance job 3n-hsw`_ and
169 `FD.io test executor dpdk performance job 3n-skx`_
171 files csit-vpp-perf-|srelease|-\*.zip
172 `archived here <../_static/archive/>`_.
177 Here is the list of known issues in |csit-release| for Testpmd performance tests:
179 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
180 | # | Issue | Jira ID | Description |
181 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
182 | | No known issues | | |
183 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+