4 Changes in |csit-release|
5 -------------------------
7 #. **DPDK performance tests**
9 - *MRR tests* - Maximum Receive Rate tests measure the packet forwarding rate
10 under the maximum load offered by traffic generator over a set trial
11 duration, regardless of packet loss. MRR tests are used for continuous
12 performance trending and for comparison between releases.
14 - *MLR tests* - NDR and PDR tests measure the packet forwarding rate using
15 MLRsearch library by traffic generator. All tests that previously used
16 binary search were converted to MLRsearch.
18 - *2-node tests* - Set of 2-node tests covering testpmd and l3fwd.
20 - Increased coverage of NIC specific tests (Intel-xxv710-da2, Intel-x710)
22 - *Generated tests* - Simplified and unified test structure,
23 semi-autogenerated by generator script. Test generator is currently able
24 to create test combinations with various frame size and cores combinations.
25 All existing test cases were converted to new format.
31 Relative performance changes in measured packet throughput in |csit-release|
32 are calculated against the results from |csit-release-1|
33 report. Listed mean and standard deviation values are computed based on
34 a series of the same tests executed against respective DPDK releases to
35 verify test results repeatability, with percentage change calculated for
41 NDR throughput changes between releases are available in a CSV and pretty ASCII
44 - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
45 - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
46 - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
47 - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
51 Test results have been generated by
52 `FD.io test executor dpdk performance job 3n-hsw`_
53 with Robot Framework result
54 files csit-dpdk-perf-|srelease|-\*.zip
55 `archived here <../_static/archive/>`_.
60 PDR throughput changes between releases are available in a CSV and pretty ASCII
63 - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
64 - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
65 - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
66 - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
70 Test results have been generated by
71 `FD.io test executor dpdk performance job 3n-hsw`_
72 with Robot Framework result
73 files csit-dpdk-perf-|srelease|-\*.zip
74 `archived here <../_static/archive/>`_.
79 MRR small packet throughput changes between releases are available in a
80 CSV and pretty ASCII formats:
82 - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-mrr.csv>`_,
83 - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-mrr.csv>`_,
84 - `csv format for 4t4c <../_static/dpdk/performance-changes-4t4c-mrr.csv>`_,
85 - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-mrr.txt>`_,
86 - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-mrr.txt>`_,
87 - `pretty ASCII format for 4t4c <../_static/dpdk/performance-changes-4t4c-mrr.txt>`_.
91 Test results have been generated by
92 `FD.io test executor dpdk performance job 3n-hsw`_,
93 with Robot Framework result
94 files csit-vpp-perf-|srelease|-\*.zip
95 `archived here <../_static/archive/>`_.
97 Comparison Across Testbeds
98 --------------------------
100 Relative performance changes in measured packet throughput on 3-Node Skx testbed
101 are calculated against the results measured on 3-Node Hsw testbed.
106 NDR throughput changes between testbeds are available in a CSV and pretty ASCII
109 - `csv format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
110 - `pretty ASCII format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
114 Test results have been generated by
115 `FD.io test executor dpdk performance job 3n-hsw`_ and
116 `FD.io test executor dpdk performance job 3n-skx`_
117 with Robot Framework result
118 files csit-dpdk-perf-|srelease|-\*.zip
119 `archived here <../_static/archive/>`_.
124 PDR throughput changes between testbeds are available in a CSV and pretty ASCII
127 - `csv format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
128 - `pretty ASCII format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
132 Test results have been generated by
133 `FD.io test executor dpdk performance job 3n-hsw`_ and
134 `FD.io test executor dpdk performance job 3n-skx`_
135 with Robot Framework result
136 files csit-dpdk-perf-|srelease|-\*.zip
137 `archived here <../_static/archive/>`_.
142 MRR throughput changes between testbeds are available in a
143 CSV and pretty ASCII formats:
145 - `csv format for mrr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
146 - `pretty ASCII format for mrr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
150 Test results have been generated by
151 `FD.io test executor dpdk performance job 3n-hsw`_ and
152 `FD.io test executor dpdk performance job 3n-skx`_
153 with Robot Framework result
154 files csit-vpp-perf-|srelease|-\*.zip
155 `archived here <../_static/archive/>`_.
160 Here is the list of known issues in |csit-release| for Testpmd performance tests:
162 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
163 | # | Issue | Jira ID | Description |
164 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
165 | | No known issues | | |
166 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+