rls1710: rls_notes update.
[csit.git] / docs / report / vpp_performance_tests / csit_release_notes.rst
1 CSIT Release Notes\r
2 ==================\r
3 \r
4 Changes in CSIT |release|\r
5 -------------------------\r
6 \r
7 #. Added VPP performance tests\r
8 \r
9    - **L2BD MAC scale tests**\r
10 \r
11      - VPP L2 Bridge-Domain with MAC learning and large size L2FIB (10k,\r
12        100k, 1M MACs), tested in NIC-to-NIC and VM vhost topologies.\r
13 \r
14    - **Linux Container VPP memif tests**\r
15 \r
16      - Tests with VPP in L2 Bridge-Domain configuration connecting over\r
17        memif virtual interfaces to VPPs running in LXCs;\r
18 \r
19    - **Docker Container VPP memif tests**\r
20 \r
21      - Tests with VPP in L2 Cross-Connect configuration connecting over\r
22        memif virtual interfaces VPPs running in Docker containers;\r
23 \r
24    - **Container Topologies Orchestrated by K8s with VPP memif tests**\r
25 \r
26      - Tests with VPP in L2 Cross-Connect and Bridge-Domain configurations\r
27        connecting over memif virtual interfaces VPPs running in Docker\r
28        containers, with service chain topologies orchestrated by Kubernetes;\r
29 \r
30    - **Stateful Security Groups**\r
31 \r
32      - m-thread m-core VPP stateful and stateless security-groups tests;\r
33 \r
34    - **MAC-IP binding**\r
35 \r
36      - MACIP input access-lists, single-thread single-core and m-thread\r
37        m-core tests;\r
38 \r
39 #. Presentation and Analytics Layer\r
40 \r
41      - New Presentation and Analytics Layer (PAL) for automated CSIT test\r
42        results analysis and presentation, including statistical analysis\r
43        of results repeatibility and test report auto-generation;\r
44 \r
45 Performance Improvements\r
46 ------------------------\r
47 \r
48 Substantial improvements in measured packet throughput have been observed in a\r
49 number of CSIT |release| tests listed below, with relative increase of\r
50 double-digit percentage points. Relative improvements for this release are\r
51 calculated against the test results listed in CSIT |release-1| report. The\r
52 comparison is calculated between the mean values based on collected and\r
53 archived test results' samples for involved VPP releases. Standard deviation\r
54 has been also listed for CSIT |release|. Performance numbers since release\r
55 VPP-16.09 are provided for reference.\r
56 \r
57 NDR Throughput\r
58 ~~~~~~~~~~~~~~\r
59 \r
60 Non-Drop Rate Throughput discovery tests:\r
61 \r
62 .. only:: html\r
63 \r
64    .. csv-table::\r
65       :align: center\r
66       :file: performance_improvements/performance_improvements_ndr_top.csv\r
67 \r
68 .. only:: latex\r
69 \r
70    .. raw:: latex\r
71 \r
72       \makeatletter\r
73       \csvset{\r
74         perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
75       }\r
76       \makeatother\r
77 \r
78       {\tiny\r
79       \csvautobooklongtable[separator=comma,\r
80         respect all,\r
81         no check column count,\r
82         perfimprovements column width=1cm,\r
83         late after line={\\\hline},\r
84         late after last line={\end{longtable}}\r
85         ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_ndr_top.csv}\r
86       }\r
87 \r
88 \r
89 PDR Throughput\r
90 ~~~~~~~~~~~~~~\r
91 \r
92 Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%:\r
93 \r
94 .. only:: html\r
95 \r
96    .. csv-table::\r
97       :align: center\r
98       :file: performance_improvements/performance_improvements_pdr_top.csv\r
99 \r
100 .. only:: latex\r
101 \r
102    .. raw:: latex\r
103 \r
104       \makeatletter\r
105       \csvset{\r
106         perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
107       }\r
108       \makeatother\r
109 \r
110       {\tiny\r
111       \csvautobooklongtable[separator=comma,\r
112         respect all,\r
113         no check column count,\r
114         perfimprovements column width=1cm,\r
115         late after line={\\\hline},\r
116         late after last line={\end{longtable}}\r
117         ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_pdr_top.csv}\r
118       }\r
119 \r
120 \r
121 Measured improvements are in line with VPP code optimizations listed in\r
122 `VPP-17.10 release notes\r
123 <https://docs.fd.io/vpp/17.10/release_notes_1710.html>`_.\r
124 \r
125 Other Performance Changes\r
126 -------------------------\r
127 \r
128 Other changes in measured packet throughput, with either minor relative increase\r
129 or decrease, have been observed in a number of CSIT |release| tests listed\r
130 below. Relative changes are calculated against the test results listed in CSIT\r
131 |release-1| report.\r
132 \r
133 NDR Throughput\r
134 ~~~~~~~~~~~~~~\r
135 \r
136 Non-Drop Rate Throughput discovery tests:\r
137 \r
138 .. only:: html\r
139 \r
140    .. csv-table::\r
141       :align: center\r
142       :file: performance_improvements/performance_improvements_ndr_low.csv\r
143 \r
144 .. only:: latex\r
145 \r
146    .. raw:: latex\r
147 \r
148       \makeatletter\r
149       \csvset{\r
150         perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
151       }\r
152       \makeatother\r
153 \r
154       {\tiny\r
155       \csvautobooklongtable[separator=comma,\r
156         respect all,\r
157         no check column count,\r
158         perfimprovements column width=1cm,\r
159         late after line={\\\hline},\r
160         late after last line={\end{longtable}}\r
161         ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_ndr_low.csv}\r
162       }\r
163 \r
164 \r
165 PDR Throughput\r
166 ~~~~~~~~~~~~~~\r
167 \r
168 Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%:\r
169 \r
170 .. only:: html\r
171 \r
172    .. csv-table::\r
173       :align: center\r
174       :file: performance_improvements/performance_improvements_pdr_low.csv\r
175 \r
176 .. only:: latex\r
177 \r
178    .. raw:: latex\r
179 \r
180       \makeatletter\r
181       \csvset{\r
182         perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},\r
183       }\r
184       \makeatother\r
185 \r
186       {\tiny\r
187       \csvautobooklongtable[separator=comma,\r
188         respect all,\r
189         no check column count,\r
190         perfimprovements column width=1cm,\r
191         late after line={\\\hline},\r
192         late after last line={\end{longtable}}\r
193         ]{../_tmp/src/vpp_performance_tests/performance_improvements/performance_improvements_pdr_low.csv}\r
194       }\r
195 \r
196 \r
197 Known Issues\r
198 ------------\r
199 \r
200 Here is the list of known issues in CSIT |release| for VPP performance tests:\r
201 \r
202 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
203 | # | Issue                                           | Jira ID    | Description                                                     |\r
204 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
205 | 1 | Vic1385 and Vic1227 low performance.            | VPP-664    | Low NDR performance.                                            |\r
206 |   |                                                 |            |                                                                 |\r
207 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
208 | 2 | Sporadic NDR discovery test failures on x520.   | CSIT-750   | Suspected issue with HW settings (BIOS, FW) in LF               |\r
209 |   |                                                 |            | infrastructure. Issue can't be replicated outside LF.           |\r
210 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
211 | 3 | VPP in 2t2c setups - large variation            | CSIT-568   | Suspected NIC firmware or DPDK driver issue affecting NDR       |\r
212 |   | of discovered NDR throughput values across      |            | throughput. Applies to XL710 and X710 NICs, x520 NICs are fine. |\r
213 |   | multiple test runs with xl710 and x710 NICs.    |            |                                                                 |\r
214 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
215 | 4 | Lower than expected NDR throughput with         | CSIT-569   | Suspected NIC firmware or DPDK driver issue affecting NDR and   |\r
216 |   | xl710 and x710 NICs, compared to x520 NICs.     |            | PDR throughput. Applies to XL710 and X710 NICs.                 |\r
217 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
218 \r

©2016 FD.io a Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation. Linux is a registered trademark of Linus Torvalds.
Please see our privacy policy and terms of use.