report 1807: list and table formatting fixes.
[csit.git] / docs / report / dpdk_performance_tests / csit_release_notes.rst
1 Release Notes
2 =============
3
4 Changes in |csit-release|
5 -------------------------
6
7 #. **DPDK performance tests**
8
9    - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
10      configurable trial duration and number of consecutive executions.
11      Coverage of MRR tests has been extended across more test
12      scenarios. MRR tests are used for continuous performance trending
13      and for comparison between DPDK releases.
14
15    - **MLRsearch Throughput**: MLRsearch algorithm has been introduced
16      for all NDR and PDR throughput tests. All tests that previously
17      used binary search got converted to MLRsearch. Coverage of NDR/PDR
18      tests has been extended across more test scenarios.
19
20    - **2-Node Tests**: A new set of 2-node tests covering DPDK Testpmd
21      and L3fwd, running on new Xeon Skylake testbeds.
22
23    - **More NICs**: Increased coverage of NIC models tested: Intel-
24      xxv710-da2, Intel-x710-da4.
25
26    - **Generated tests**: Simplified and unified test structure, semi-
27      autogenerated by generator script. Test generator is currently
28      able to create test combinations with various frame size and
29      cores combinations. All existing test cases were converted to new
30      format.
31
32 #. **Presentation and Analytics Layer**
33
34    - **Performance trending**: Added continuous performance trending with
35      anomaly detection and analysis for DPDK.
36
37 Performance Changes
38 -------------------
39
40 Relative performance changes in measured NDR, PDR and MRR packet
41 throughput in |csit-release| are calculated against the test results
42 from |csit-release-1| report, for tests running on 3-Node Intel Xeon
43 Haswell testbeds (3n-hsw) in 1-core, 2-core and 4-core (MRR only)
44 configurations.
45
46 Listed mean and standard deviation values are computed based on a series
47 of the same tests executed against respective VPP releases to verify
48 test results repeatability, with percentage change calculated for mean
49 values.
50
51 NDR Changes
52 ~~~~~~~~~~~
53
54 NDR throughput changes between releases are available in CSV and pretty
55 ASCII formats:
56
57   - `CSV 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
58   - `CSV 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
59   - `ASCII 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
60   - `ASCII 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
61
62 .. note::
63
64     Test results have been generated by
65     `FD.io test executor dpdk performance job 3n-hsw`_
66     with RF result
67     files csit-dpdk-perf-|srelease|-\*.zip
68     `archived here <../_static/archive/>`_.
69
70 PDR Changes
71 ~~~~~~~~~~~
72
73 PDR throughput changes between releases are available in CSV and pretty
74 ASCII formats:
75
76   - `CSV 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
77   - `CSV 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
78   - `ASCII 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
79   - `ASCII 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
80
81 .. note::
82
83     Test results have been generated by
84     `FD.io test executor dpdk performance job 3n-hsw`_
85     with RF result
86     files csit-dpdk-perf-|srelease|-\*.zip
87     `archived here <../_static/archive/>`_.
88
89 MRR Changes
90 ~~~~~~~~~~~
91
92 MRR throughput changes between releases are available in CSV and pretty
93 ASCII formats:
94
95   - `CSV 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.csv>`_,
96   - `CSV 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.csv>`_,
97   - `CSV 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.csv>`_,
98   - `ASCII 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.txt>`_,
99   - `ASCII 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.txt>`_,
100   - `ASCII 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.txt>`_.
101
102 .. note::
103
104     Test results have been generated by
105     `FD.io test executor dpdk performance job 3n-hsw`_,
106     with RF result
107     files csit-vpp-perf-|srelease|-\*.zip
108     `archived here <../_static/archive/>`_.
109
110 Skx vs. Hsw Comparison
111 ----------------------
112
113 Relative performance changes in measured packet throughput on 3-Node Skx testbed
114 are calculated against the results measured on 3-Node Hsw testbed.
115
116 NDR Comparison
117 ~~~~~~~~~~~~~~
118
119 NDR throughput changes between testbeds are available in a CSV and pretty ASCII
120 formats:
121
122   - `CSV 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
123   - `ASCII 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
124
125 .. note::
126
127     Test results have been generated by
128     `FD.io test executor dpdk performance job 3n-hsw`_ and
129     `FD.io test executor dpdk performance job 3n-skx`_
130     with RF result
131     files csit-dpdk-perf-|srelease|-\*.zip
132     `archived here <../_static/archive/>`_.
133
134 PDR Comparison
135 ~~~~~~~~~~~~~~
136
137 PDR throughput changes between testbeds are available in a CSV and pretty ASCII
138 formats:
139
140   - `CSV 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
141   - `ASCII 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
142
143 .. note::
144
145     Test results have been generated by
146     `FD.io test executor dpdk performance job 3n-hsw`_ and
147     `FD.io test executor dpdk performance job 3n-skx`_
148     with RF result
149     files csit-dpdk-perf-|srelease|-\*.zip
150     `archived here <../_static/archive/>`_.
151
152 MRR Comparison
153 ~~~~~~~~~~~~~~
154
155 MRR throughput changes between testbeds are available in a
156 CSV and pretty ASCII formats:
157
158   - `CSV 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
159   - `ASCII 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
160
161 .. note::
162
163     Test results have been generated by
164     `FD.io test executor dpdk performance job 3n-hsw`_ and
165     `FD.io test executor dpdk performance job 3n-skx`_
166     with RF result
167     files csit-vpp-perf-|srelease|-\*.zip
168     `archived here <../_static/archive/>`_.
169
170 Known Issues
171 ------------
172
173 Here is the list of known issues in |csit-release| for Testpmd performance tests:
174
175 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
176 | # | Issue                                             | Jira ID    | Description                                                     |
177 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
178 |   | No known issues                                   |            |                                                                 |
179 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+

©2016 FD.io a Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation. Linux is a registered trademark of Linus Torvalds.
Please see our privacy policy and terms of use.