d029bd0e74fcca4cd000f316c8b0618daee15c9c
[csit.git] / docs / report / vpp_performance_tests / csit_release_notes.rst
1 CSIT Release Notes\r
2 ==================\r
3 \r
4 Changes in CSIT |release|\r
5 -------------------------\r
6 \r
7 #. Added VPP performance tests\r
8 \r
9    - **Container Topologies Orchestrated by K8s with VPP memif tests**\r
10 \r
11      - Added tests with VPP in L2 Cross-Connect and Bridge-Domain configurations\r
12        connecting over memif virtual interfaces VPPs running in Docker\r
13        containers, with service chain topologies orchestrated by Kubernetes.\r
14        Chained, horizontal and standalone linked topologies added;\r
15 \r
16    - **VPP TCP/IP stack**\r
17 \r
18      - Added tests with VPP builtin TCP based HTTP server. WRK traffic generator\r
19        is used.\r
20 \r
21    - **SRv6 tests**\r
22 \r
23      - [placeholder]\r
24 \r
25    - **IPSecSW tests**\r
26 \r
27      - SW computed IPSec encryption with AES-GCM, CBC-SHA1 ciphers, in\r
28        combination with IPv4 routed-forwarding.\r
29 \r
30 #. Presentation and Analytics Layer\r
31 \r
32      - Added multi-core scale-up analysis into Presentation and Analytics Layer\r
33        (PAL) for automated CSIT test results analysis;\r
34 \r
35 \r
36 #. Other improvements\r
37 \r
38      - **Framework optimizations**\r
39 \r
40        - Ability to run CSIT framework on ARM architecture.\r
41 \r
42        - Overall stability improvements.\r
43 \r
44 Performance Improvements\r
45 ------------------------\r
46 \r
47 Substantial improvements in measured packet throughput have been observed in a\r
48 number of CSIT |release| tests listed below, with relative increase of\r
49 double-digit percentage points. Relative improvements for this release are\r
50 calculated against the test results listed in CSIT |release-1| report. The\r
51 comparison is calculated between the mean values based on collected and\r
52 archived test results' samples for involved VPP releases. Standard deviation\r
53 has been also listed for CSIT |release|. Performance numbers since release\r
54 VPP-16.09 are provided for reference.\r
55 \r
56 NDR Throughput\r
57 ~~~~~~~~~~~~~~\r
58 \r
59 Non-Drop Rate Throughput discovery tests:\r
60 \r
61 .. only:: html\r
62 \r
63    .. csv-table::\r
64       :align: center\r
65       :file: performance_improvements/performance_improvements_ndr_top.csv\r
66 \r
67 .. only:: latex\r
68 \r
69    .. raw:: latex\r
70 \r
71       \makeatletter\r
72       \csvset{\r
73         perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
74       }\r
75       \makeatother\r
76 \r
77       {\tiny\r
78       \csvautobooklongtable[separator=comma,\r
79         respect all,\r
80         no check column count,\r
81         perfimprovements column width=1cm,\r
82         late after line={\\\hline},\r
83         late after last line={\end{longtable}}\r
84         ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_ndr_top.csv}\r
85       }\r
86 \r
87 \r
88 PDR Throughput\r
89 ~~~~~~~~~~~~~~\r
90 \r
91 Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%:\r
92 \r
93 .. only:: html\r
94 \r
95    .. csv-table::\r
96       :align: center\r
97       :file: performance_improvements/performance_improvements_pdr_top.csv\r
98 \r
99 .. only:: latex\r
100 \r
101    .. raw:: latex\r
102 \r
103       \makeatletter\r
104       \csvset{\r
105         perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
106       }\r
107       \makeatother\r
108 \r
109       {\tiny\r
110       \csvautobooklongtable[separator=comma,\r
111         respect all,\r
112         no check column count,\r
113         perfimprovements column width=1cm,\r
114         late after line={\\\hline},\r
115         late after last line={\end{longtable}}\r
116         ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_pdr_top.csv}\r
117       }\r
118 \r
119 \r
120 Measured improvements are in line with VPP code optimizations listed in\r
121 `VPP-17.10 release notes\r
122 <https://docs.fd.io/vpp/17.10/release_notes_1710.html>`_.\r
123 \r
124 Other Performance Changes\r
125 -------------------------\r
126 \r
127 Other changes in measured packet throughput, with either minor relative increase\r
128 or decrease, have been observed in a number of CSIT |release| tests listed\r
129 below. Relative changes are calculated against the test results listed in CSIT\r
130 |release-1| report.\r
131 \r
132 NDR Throughput\r
133 ~~~~~~~~~~~~~~\r
134 \r
135 Non-Drop Rate Throughput discovery tests:\r
136 \r
137 .. only:: html\r
138 \r
139    .. csv-table::\r
140       :align: center\r
141       :file: performance_improvements/performance_improvements_ndr_low.csv\r
142 \r
143 .. only:: latex\r
144 \r
145    .. raw:: latex\r
146 \r
147       \makeatletter\r
148       \csvset{\r
149         perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
150       }\r
151       \makeatother\r
152 \r
153       {\tiny\r
154       \csvautobooklongtable[separator=comma,\r
155         respect all,\r
156         no check column count,\r
157         perfimprovements column width=1cm,\r
158         late after line={\\\hline},\r
159         late after last line={\end{longtable}}\r
160         ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_ndr_low.csv}\r
161       }\r
162 \r
163 \r
164 PDR Throughput\r
165 ~~~~~~~~~~~~~~\r
166 \r
167 Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%:\r
168 \r
169 .. only:: html\r
170 \r
171    .. csv-table::\r
172       :align: center\r
173       :file: performance_improvements/performance_improvements_pdr_low.csv\r
174 \r
175 .. only:: latex\r
176 \r
177    .. raw:: latex\r
178 \r
179       \makeatletter\r
180       \csvset{\r
181         perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
182       }\r
183       \makeatother\r
184 \r
185       {\tiny\r
186       \csvautobooklongtable[separator=comma,\r
187         respect all,\r
188         no check column count,\r
189         perfimprovements column width=1cm,\r
190         late after line={\\\hline},\r
191         late after last line={\end{longtable}}\r
192         ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_pdr_low.csv}\r
193       }\r
194 \r
195 \r
196 Known Issues\r
197 ------------\r
198 \r
199 Here is the list of known issues in CSIT |release| for VPP performance tests:\r
200 \r
201 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
202 | # | Issue                                           | Jira ID    | Description                                                     |\r
203 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
204 | 1 | Vic1385 and Vic1227 low performance.            | VPP-664    | Low NDR performance.                                            |\r
205 |   |                                                 |            |                                                                 |\r
206 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
207 | 2 | Sporadic NDR discovery test failures on x520.   | CSIT-750   | Suspected issue with HW combination of X710-X520 in LF          |\r
208 |   |                                                 |            | infrastructure. Issue can't be replicated outside LF.           |\r
209 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
210 | 3 | VPP in 2t2c setups - large variation            | CSIT-568   | Suspected NIC firmware or DPDK driver issue affecting NDR       |\r
211 |   | of discovered NDR throughput values across      |            | throughput. Applies to XL710 and X710 NICs, x520 NICs are fine. |\r
212 |   | multiple test runs with xl710 and x710 NICs.    |            |                                                                 |\r
213 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
214 | 4 | Lower than expected NDR throughput with         | CSIT-569   | Suspected NIC firmware or DPDK driver issue affecting NDR and   |\r
215 |   | xl710 and x710 NICs, compared to x520 NICs.     |            | PDR throughput. Applies to XL710 and X710 NICs.                 |\r
216 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
217 \r

©2016 FD.io a Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation. Linux is a registered trademark of Linus Torvalds.
Please see our privacy policy and terms of use.