4 Changes in CSIT |release|
\r
5 -------------------------
\r
7 #. Naming change for all VPP performance test suites and test cases.
\r
8 #. VPP performance test environment changes
\r
10 - upgrade to Ubuntu 16.04
\r
11 - VM and vhost-user test environment optimizations
\r
12 - addition of HW cryptodev devices in LFD FD.io physical testbed
\r
17 - new NICs - Intel x710, Cisco VIC1385, Cisco VIC1227
\r
18 - more VM vhost-user tests
\r
22 Performance Tests Naming
\r
23 ------------------------
\r
25 CSIT |release| introduced a common structured naming convention for all
\r
26 performance and functional tests. This change was driven by substantially
\r
27 growing number and type of CSIT test cases. Firstly, the original practice did
\r
28 not always follow any strict naming convention. Secondly test names did not
\r
29 always clearly capture tested packet encapsulations, and the actual type or
\r
30 content of the tests. Thirdly HW configurations in terms of NICs, ports and
\r
31 their locality were not captured either. These were but few reasons that drove
\r
32 the decision to change and define a new more complete and stricter test naming
\r
33 convention, and to apply this to all existing and new test cases.
\r
35 The new naming should be intuitive for majority of the tests. The complete
\r
36 description of CSIT test naming convention is provided on `CSIT test naming wiki
\r
37 <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
\r
39 Here few illustrative examples of the new naming usage for performance test
\r
42 #. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P**
\r
44 - *PortNICConfig-WireEncapsulation-PacketForwardingFunction-
\r
45 PacketProcessingFunction1-...-PacketProcessingFunctionN-TestType*
\r
46 - *10ge2p1x520-dot1q-l2bdbasemaclrn-ndrdisc.robot* => 2 ports of 10GE on
\r
47 Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain baseline switching
\r
48 with MAC learning, NDR throughput discovery.
\r
49 - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrchk.robot* => 2 ports of 10GE
\r
50 on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain baseline
\r
51 switching with MAC learning, NDR throughput discovery.
\r
52 - *10ge2p1x520-ethip4-ip4base-ndrdisc.robot* => 2 ports of 10GE on Intel
\r
53 x520 NIC, IPv4 baseline routed forwarding, NDR throughput discovery.
\r
54 - *10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot* => 2 ports of 10GE on
\r
55 Intel x520 NIC, IPv6 scaled up routed forwarding, NDR throughput
\r
58 #. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC,
\r
59 P2V2P, NIC2VMchain2NIC, P2V2V2P**
\r
61 - *PortNICConfig-WireEncapsulation-PacketForwardingFunction-
\r
62 PacketProcessingFunction1-...-PacketProcessingFunctionN-VirtEncapsulation-
\r
63 VirtPortConfig-VMconfig-TestType*
\r
64 - *10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 ports
\r
65 of 10GE on Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain
\r
66 switching to/from two vhost interfaces and one VM, NDR throughput
\r
68 - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2
\r
69 ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain
\r
70 switching to/from two vhost interfaces and one VM, NDR throughput
\r
72 - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc.robot* => 2
\r
73 ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain
\r
74 switching to/from four vhost interfaces and two VMs, NDR throughput
\r
77 Measured Performance Improvements
\r
78 ---------------------------------
\r
80 Substantial improvements in measured packet throughput have been observed
\r
81 in VPP-17.01 for the following CSIT |release| tests:
\r
83 +-------------------+----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
84 | VPP Functionality | Test Name | VPP-16.09 | VPP-17.01 | Relative Improvement |
\r
85 +===================+================================================================+===========+=================+======================+
\r
86 | L2XC | 10ge2p1x520:64B-1t1c-eth-l2xcbase-ndrdisc | 9.4 Mpps | 12.6..12.9 Mpps | 34..37% |
\r
87 +-------------------+----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
88 | L2XC | 10ge2p1xl710:64B-1t1c-eth-l2xcbase-ndrdisc | 9.5 Mpps | 12.1..12.4 Mpps | 27..30% |
\r
89 +-------------------+----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
90 | L2BD | 10ge2p1x520:64B-1t1c-eth-l2bdbasemaclrn-ndrdisc | 7.8 Mpps | 10.6 Mpps | 36% |
\r
91 +-------------------+----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
92 | L2BD-vhost-VM | 10ge2p1x520:64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc | 0.4 Mpps | 2.8 Mpps | 600% |
\r
93 +-------------------+----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
94 | L2XC-vhost-VM | 10ge2p1x520:64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc | 0.4 Mpps | 3.2 Mpps | 700% |
\r
95 +-------------------+----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
96 | IPv4 | 10ge2p1x520:64B-1t1c-ethip4-ip4base-ndrdisc | 8.7 Mpps | 9.7 Mpps | 12% |
\r
97 +-------------------+----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
99 Non-drop rate search:
\r
101 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
102 | VPP Functionality | Test Name | VPP-16.09 | VPP-17.01 | Relative Improvement |
\r
103 +===================+=================================================================+===========+=================+======================+
\r
104 | L2XC | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-ndrdisc | 9.4 Mpps | 12.7 Mpps | 35% |
\r
105 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
106 | L2XC | 10ge2p1xl710: 64B-1t1c-eth-l2xcbase-ndrdisc | 9.5 Mpps | 12.2..12.4 Mpps | 28..30% |
\r
107 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
108 | L2XC dot1ad | 10ge2p1x520: 64B-1t1c-dot1ad-l2xcbase-ndrdisc | 7.4 Mpps | 8.8..9.0 Mpps | 19..23% |
\r
109 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
110 | L2XC dot1q | 10ge2p1x520: 64B-1t1c-dot1q-l2xcbase-ndrdisc | 7.5 Mpps | 8.8..9.0 Mpps | 17..20% |
\r
111 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
112 | L2XC VxLAN | 10ge2p1x520: 64B-1t1c-ethip4vxlan-l2xcbase-ndrdisc | 5.4 Mpps | 6.5 Mpps | 20% |
\r
113 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
114 | L2XC-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc | 0.5 Mpps | 2.8..3.2 Mpps | 460..540% |
\r
115 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
116 | L2BD | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-ndrdisc | 7.8 Mpps | 10.4..10.6 Mpps | 33..36% |
\r
117 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
118 | L2BD-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc | 0.4 Mpps | 2.7..2.8 Mpps | 575..600% |
\r
119 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
120 | IPv4 | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ndrdisc | 8.7 Mpps | 9.7 Mpps | 11% |
\r
121 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
122 | IPv4 COP | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc | 7.1 Mpps | 8.3..8.5 Mpps | 17..20% |
\r
123 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
124 | IPv4 FIB 200k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale200k-ndrdisc | 8.5 Mpps | 9.0 Mpps | 6% |
\r
125 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
126 | IPv4 FIB 20k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale20k-ndrdisc | 8.5 Mpps | 9.0..9.2 Mpps | 6..8% |
\r
127 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
128 | IPv4 FIB 2M | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale2m-ndrdisc | 8.5 Mpps | 7.8..8.1 Mpps | -8..-5% |
\r
129 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
130 | IPv4 iAcl | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc | 6.9 Mpps | 7.6..7.8 Mpps | 10..13% |
\r
131 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
132 | IPv4 Policer | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc | 6.9 Mpps | 7.4..7.6 Mpps | 7..10% |
\r
133 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
134 | IPv4 LISP | 10ge2p1x520: 64B-1t1c-ethip4lispip4-ip4base-ndrdisc | 4.4 Mpps | 4.8 Mpps | 9% |
\r
135 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
136 | IPv4 vhost | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc | 0.3 Mpps | 2.6 Mpps | 767% |
\r
137 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
138 | IPv6 | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-ndrdisc | 3.0 Mpps | 7.3..7.7 Mpps | 143..157% |
\r
139 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
140 | IPv6 COP | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-copwhtlistbase-ndrdisc | 6.1 Mpps | 6.1..6.5 Mpps | 0..7% |
\r
141 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
142 | IPv6 FIB 200k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale200k-ndrdisc | 6.5 Mpps | 5.3..5.7 Mpps | -18..-12% |
\r
143 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
144 | IPv6 FIB 20k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-ndrdisc | 6.9 Mpps | 6.5 Mpps | -6% |
\r
145 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
146 | IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-ndrdisc | 5.3 Mpps | 4.2 Mpps | -21% |
\r
147 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
148 | IPv6 iAcl | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-ndrdisc | 6.5 Mpps | 6.1..6.5 Mpps | -6..0% |
\r
149 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
151 Partial drop rate search, LT = 0.5%:
\r
153 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
154 | VPP Functionality | Test Name | VPP-16.09 | VPP-17.01 | Relative Improvement |
\r
155 +===================+=================================================================+===========+=================+======================+
\r
156 | L2XC | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-pdrdisc | 9.4 Mpps | 12.7..12.9 Mpps | 35..37% |
\r
157 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
158 | L2XC | 10ge2p1xl710: 64B-1t1c-eth-l2xcbase-pdrdisc | no data | no data | |
\r
159 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
160 | L2XC dot1ad | 10ge2p1x520: 64B-1t1c-dot1ad-l2xcbase-pdrdisc | 7.4 Mpps | 8.8..9.1 Mpps | 19..23% |
\r
161 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
162 | L2XC dot1q | 10ge2p1x520: 64B-1t1c-dot1q-l2xcbase-pdrdisc | 7.5 Mpps | 8.8..9.0 Mpps | 17..20% |
\r
163 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
164 | L2XC VxLAN | 10ge2p1x520: 64B-1t1c-ethip4vxlan-l2xcbase-pdrdisc | 5.4 Mpps | 6.5 Mpps | 20% |
\r
165 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
166 | L2XC-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc | 2.6 Mpps | 3.2..3.3 Mpps | 23..26% |
\r
167 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
168 | L2BD | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-pdrdisc | 7.8 Mpps | 10.6 Mpps | 36% |
\r
169 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
170 | L2BD-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc | 2.1 Mpps | 2.9 Mpps | 38% |
\r
171 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
172 | IPv4 | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-pdrdisc | 8.7 Mpps | 9.7 Mpps | 11% |
\r
173 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
174 | IPv4 COP | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-copwhtlistbase-pdrdisc | 7.1 Mpps | 8.3..8.5 Mpps | 17..20% |
\r
175 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
176 | IPv4 FIB 200k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale200k-pdrdisc | 8.5 Mpps | 9.0 Mpps | 6% |
\r
177 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
178 | IPv4 FIB 20k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale20k-pdrdisc | 8.5 Mpps | 9.0..9.2 Mpps | 6..8% |
\r
179 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
180 | IPv4 FIB 2M | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale2m-pdrdisc | 8.3 Mpps | 8.1 Mpps | -2% |
\r
181 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
182 | IPv4 iAcl | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-iacldstbase-pdrdisc | 7.1 Mpps | 7.6..7.8 Mpps | 7..10% |
\r
183 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
184 | IPv4 Policer | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ipolicemarkbase-pdrdisc | 7.1 Mpps | 7.4..7.6 Mpps | 4..7% |
\r
185 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
186 | IPv4 LISP | 10ge2p1x520: 64B-1t1c-ethip4lispip4-ip4base-pdrdisc | 4.6 Mpps | 4.8 Mpps | 9% |
\r
187 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
188 | IPv4 vhost | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc | 2.0 Mpps | 2.7 Mpps | 35% |
\r
189 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
190 | IPv6 | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-pdrdisc | 7.7 Mpps | 7.3..7.7 Mpps | -5..0% |
\r
191 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
192 | IPv6 COP | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-copwhtlistbase-pdrdisc | 6.1 Mpps | 6.1..6.5 Mpps | 0..7% |
\r
193 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
194 | IPv6 FIB 200k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale200k-pdrdisc | 6.9 Mpps | 5.3..5.7 Mpps | -23..-17% |
\r
195 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
196 | IPv6 FIB 20k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-pdrdisc | 6.9 Mpps | 6.5 Mpps | -6% |
\r
197 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
198 | IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-pdrdisc | 5.3 Mpps | 4.2 Mpps | -21% |
\r
199 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
200 | IPv6 iAcl | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-pdrdisc | 6.5 Mpps | 6.1..6.5 Mpps | -6..0% |
\r
201 +-------------------+-----------------------------------------------------------------+-----------+-----------------+----------------------+
\r
203 This is in line with VPP code optimizations listed in `VPP-17.01 release notes
\r
204 <https://docs.fd.io/vpp/17.01/release_notes_1701.html>`_.
\r
206 vhost-VM performance improvement is also due to Linux KVM test environment
\r
207 optimization for vhost-VM tests - see section below "VM vhost-user
\r
208 Throughput Measurements".
\r
210 Multi-Thread and Multi-Core Measurements
\r
211 ----------------------------------------
\r
213 **HyperThreading** - CSIT |release| performance tests are executed with SUT
\r
214 servers' Intel XEON CPUs configured in HyperThreading Disabled mode (BIOS
\r
215 settings). This is the simplest configuration used to establish baseline
\r
216 single-thread single-core SW packet processing and forwarding performance.
\r
217 Subsequent releases of CSIT will add performance tests with Intel
\r
218 HyperThreading Enabled (requires BIOS settings change and hard reboot).
\r
220 **Multi-core Test** - CSIT |release| multi-core tests are executed in the
\r
221 following VPP thread and core configurations:
\r
223 #. 1t1c - 1 VPP worker thread on 1 CPU physical core.
\r
224 #. 2t2c - 2 VPP worker threads on 2 CPU physical cores.
\r
225 #. 4t4c - 4 VPP threads on 4 CPU physical cores.
\r
227 Note that in quite a few test cases running VPP on 2 or 4 physical cores hits
\r
228 the tested NIC I/O bandwidth or packets-per-second limit.
\r
230 Packet Throughput Measurements
\r
231 ------------------------------
\r
233 Following values are measured and reported for packet throughput tests:
\r
235 - NDR binary search per RFC2544:
\r
237 - Packet rate: "RATE: <aggregate packet rate in packets-per-second> pps
\r
238 (2x <per direction packets-per-second>)"
\r
239 - Aggregate bandwidth: "BANDWIDTH: <aggregate bandwidth in Gigabits per
\r
240 second> Gbps (untagged)"
\r
242 - PDR binary search per RFC2544:
\r
244 - Packet rate: "RATE: <aggregate packet rate in packets-per-second> pps (2x
\r
245 <per direction packets-per-second>)"
\r
246 - Aggregate bandwidth: "BANDWIDTH: <aggregate bandwidth in Gigabits per
\r
247 second> Gbps (untagged)"
\r
248 - Packet loss tolerance: "LOSS_ACCEPTANCE <accepted percentage of packets
\r
249 lost at PDR rate>""
\r
251 - NDR and PDR are measured for the following L2 frame sizes:
\r
253 - IPv4: 64B, IMIX_v4_1 (28x64B,16x570B,4x1518B), 1518B, 9000B.
\r
254 - IPv6: 78B, 1518B, 9000B.
\r
257 Packet Latency Measurements
\r
258 ---------------------------
\r
260 TRex Traffic Generator (TG) is used for measuring latency of VPP DUTs. Reported
\r
261 latency values are measured using following methodology:
\r
263 - Latency tests are performed at 10%, 50% of discovered NDR rate (non drop rate)
\r
264 for each NDR throughput test and packet size (except IMIX).
\r
265 - TG sends dedicated latency streams, one per direction, each at the rate of
\r
266 10kpps at the prescribed packet size; these are sent in addition to the main
\r
268 - TG reports min/avg/max latency values per stream direction, hence two sets
\r
269 of latency values are reported per test case; future release of TRex is
\r
270 expected to report latency percentiles.
\r
271 - Reported latency values are aggregate across two SUTs due to three node
\r
272 topology used for all performance tests; for per SUT latency, reported value
\r
273 should be divided by two.
\r
274 - 1usec is the measurement accuracy advertised by TRex TG for the setup used in
\r
275 FD.io labs used by CSIT project.
\r
276 - TRex setup introduces an always-on error of about 2*2usec per latency flow -
\r
277 additonal Tx/Rx interface latency induced by TRex SW writing and reading
\r
278 packet timestamps on CPU cores without HW acceleration on NICs closer to the
\r
282 KVM VM vhost Measurements
\r
283 -------------------------
\r
285 CSIT |release| introduced environment configuration changes to KVM Qemu vhost-
\r
286 user tests in order to more representatively measure VPP-17.01 performance in
\r
287 configurations with vhost-user interfaces and VMs.
\r
289 Current setup of CSIT FD.io performance lab is using tuned settings for more
\r
290 optimal performance of KVM Qemu:
\r
292 - Default Qemu virtio queue size of 256 descriptors.
\r
293 - Adjusted Linux kernel CFS scheduler settings, as detailed on this CSIT wiki
\r
294 page: https://wiki.fd.io/view/CSIT/VM-vhost-env-tuning.
\r
296 Adjusted Linux kernel CFS settings make the NDR and PDR throughput performance
\r
297 of VPP+VM system less sensitive to other Linux OS system tasks by reducing
\r
298 their interference on CPU cores that are designated for critical software
\r
299 tasks under test, namely VPP worker threads in host and Testpmd threads in
\r
300 guest dealing with data plan.
\r
302 Report Addendum Tests - Cryptodev
\r
303 ---------------------------------
\r
305 DPDK Cryptodev functionality support for both SW and HW crypto devices has
\r
306 been introduced in VPP-17.01 release. CSIT functional and performance tests
\r
307 have been also developed and merged. However due to the factors beyond CSIT
\r
308 project control execution of those tests within the LF FD.io test environment
\r
309 still need to complete. Once the results become available, they will be
\r
310 published as an addendum to the current version of CSIT |release| report.
\r
312 Report Addendum Tests - Centos
\r
313 ------------------------------
\r
315 CSIT |release| added Centos functional test execution environment in FD.io
\r
316 VIRL testbeds.However due to the factors beyond CSIT project control execution
\r
317 of those tests within the LF FD.io test environment still need to complete.
\r
318 Once the results become available, they will be published as an addendum to
\r
319 the current version of CSIT |release| report.
\r
321 Report Addendum Tests - SNAT44
\r
322 ------------------------------
\r
324 VPP SNAT44 functionality has been introduced in VPP-17.01 release. CSIT
\r
325 performance tests are still in development and integration into LFD FD.io test
\r
326 environment. Once the tests are fully integrated and results become available,
\r
327 they will be published as an addendum to the current version of CSIT |release|
\r
333 Here is the list of known issues in CSIT |release| for VPP performance tests in physical HW testbed:
\r
335 +---+-------------------------------------------------+-----------------------------------------------------------------+
\r
336 | # | Issue | Description |
\r
337 +---+-------------------------------------------------+-----------------------------------------------------------------+
\r
338 | 1 | Sporadic IPv4 routed-forwwarding NDR discovery | Suspected issue with DPDK IPv4 checksum calculation, VPP jira # |
\r
339 | | test failures for 1518B frame size | Observed frequency: sporadic, ca. 20% to 30% of test runs |
\r
340 +---+-------------------------------------------------+-----------------------------------------------------------------+
\r
342 +---+-------------------------------------------------+-----------------------------------------------------------------+
\r
344 +---+-------------------------------------------------+-----------------------------------------------------------------+
\r
Code Review / csit.git / blob