4 Changes in CSIT |release| - to be updated
\r
5 -------------------------
\r
7 #. VPP performance test environment changes
\r
9 - further VM and vhost-user test environment optimizations - Qemu virtio queue size increased from default value of 256 to 1024
\r
10 - addition of HW cryptodev devices in all three LF FD.io physical testbeds
\r
12 #. Added tests (to be updated)
\r
16 - Intel x710 (L2 Bridge Domain switched-forwarding, vhost-user)
\r
17 - Cisco VIC1385 (L2 Bridge Domain switched-forwarding)
\r
18 - Cisco VIC1227 (L2 Bridge Domain switched-forwarding)
\r
20 - vhost-user tests with VM
\r
22 - L2 Bridge Domain switched-forwarding with Intel x710 NIC
\r
23 - L2 Bridge Domain switched-forwarding with VxLAN and Intel x520 NIC
\r
24 - L2 Bridge Domain switched-forwarding with Intel xl710 NIC
\r
28 - L2 Bridge Domain switched-forwarding with Intel x520 NIC
\r
29 - L2 Bridge Domain switched-forwarding with vhost-user, VM and Intel x520
\r
32 Performance Improvements - to be updated
\r
33 ------------------------
\r
35 Substantial improvements in measured packet throughput, with relative increase
\r
36 of double-digit percentage points, have been observed in a number of CSIT
\r
37 |release| tests listed below. Relative improvements are calculated against the
\r
38 test results in CSIT rls1701 report.
\r
43 Non-Drop Rate Throughput discovery tests:
\r
45 +-------------------+-----------------------------------------------------------------+------------+-----------+----------------------------+
\r
46 | VPP Functionality | Test Name | VPP-16.09 | VPP-17.01 | VPP-17.04 | 17.01 to 17.04 |
\r
47 | | | [Mpps] | [Mpps] | [Mpps] | Improvement |
\r
48 +===================+=================================================================+============+===========+===========+================+
\r
49 | L2XC | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-ndrdisc | 9.4 | 12.7 | 13.4 | 6% |
\r
50 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
51 | L2XC | 10ge2p1xl710: 64B-1t1c-eth-l2xcbase-ndrdisc | 9.5 | 12.2 | 12.4 | 2% |
\r
52 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
53 | L2XC dot1ad | 10ge2p1x520: 64B-1t1c-dot1ad-l2xcbase-ndrdisc | 7.4 | 8.8 | 9.3 | 6% |
\r
54 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
55 | L2XC dot1q | 10ge2p1x520: 64B-1t1c-dot1q-l2xcbase-ndrdisc | 7.5 | 8.8 | 9.2 | 5% |
\r
56 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
57 | L2XC VxLAN | 10ge2p1x520: 64B-1t1c-ethip4vxlan-l2xcbase-ndrdisc | 5.4 | 6.5 | 6.8 | 5% |
\r
58 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
59 | L2XC-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc | 0.5 | 2.8 | 3.2 | 14% |
\r
60 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
61 | L2BD | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-ndrdisc | 7.8 | 10.4 | 10.8 | 4% |
\r
62 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
63 | L2BD-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc | 0.4 | 2.7 | 3.4 | 26% |
\r
64 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
65 | IPv4 | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ndrdisc | 8.7 | 9.7 | 10.6 | 9% |
\r
66 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
67 | IPv4 COP | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc | 7.1 | 8.3 | 9.0 | 8% |
\r
68 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
69 | IPv4 iAcl | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc | 6.9 | 7.6 | 8.3 | 9% |
\r
70 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
71 | IPv4 vhost | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc | 0.3 | 2.6 | 3.1 | 19% |
\r
72 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
73 | IPv6 | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-ndrdisc | 3.0 | 7.3 | 8.5 | 16% |
\r
74 +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+----------------+
\r
79 Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%:
\r
81 +-------------------+-----------------------------------------------------------------+-----------+-----------+-------------------------+
\r
82 | VPP Functionality | Test Name | VPP-16.09 | VPP-17.01 | VPP-17.04 | Relative |
\r
83 | | | [Mpps] | [Mpps] | [Mpps] | Improvement |
\r
84 +===================+=================================================================+===========+===========+===========+=============+
\r
85 | L2XC | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-pdrdisc | 9.4 | 12.7 | 13.4 | % |
\r
86 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
87 | L2XC dot1ad | 10ge2p1x520: 64B-1t1c-dot1ad-l2xcbase-pdrdisc | 7.4 | 8.8 | 9.3 | % |
\r
88 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
89 | L2XC dot1q | 10ge2p1x520: 64B-1t1c-dot1q-l2xcbase-pdrdisc | 7.5 | 8.8 | 9.5 | % |
\r
90 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
91 | L2XC VxLAN | 10ge2p1x520: 64B-1t1c-ethip4vxlan-l2xcbase-pdrdisc | 5.4 | 6.5 | 6.8 | % |
\r
92 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
93 | L2XC-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc | 2.6 | 3.2 | 3.2 | % |
\r
94 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
95 | L2BD | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-pdrdisc | 7.8 | 10.6 | 11.1 | % |
\r
96 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
97 | L2BD-vhost-VM | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc | 2.1 | 2.9 | 3.2 | % |
\r
98 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
99 | IPv4 | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-pdrdisc | 8.7 | 9.7 | 10.6 | % |
\r
100 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
101 | IPv4 COP | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-copwhtlistbase-pdrdisc | 7.1 | 8.3 | 9.2 | % |
\r
102 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
103 | IPv4 vhost | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc | 2.0 | 2.7 | 3.2 | % |
\r
104 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
106 Measured improvements are in line with VPP code optimizations listed in
\r
107 `VPP-17.04 release notes
\r
108 <https://docs.fd.io/vpp/17.04/release_notes_1704.html>`_.
\r
110 Additionally, vhost-VM performance improvements are due to both VPP code
\r
111 optimizations as well as due to the FD.io CSIT Linux KVM test environment
\r
112 optimizations for vhost-VM tests - see section "2.1.7. Methodology: KVM VM
\r
116 Other Performance Changes - to be updated
\r
117 -------------------------
\r
119 Other changes in measured packet throughput, with either minor relative
\r
120 increase or decrease, have been observed in a number of CSIT |release| tests
\r
121 listed below. Relative changes are calculated against the test results in CSIT
\r
127 Non-Drop Rate Throughput discovery tests:
\r
129 +-------------------+-----------------------------------------------------------------+-----------+-----------+-------------------------+
\r
130 | VPP Functionality | Test Name | VPP-16.09 | VPP-17.01 | VPP-17.04 | Relative |
\r
131 | | | [Mpps] | [Mpps] | [Mpps] | Change |
\r
132 +===================+=================================================================+===========+===========+===========+=============+
\r
133 | IPv4 FIB 200k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale200k-ndrdisc | 8.5 | 9.0 | | % |
\r
134 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
135 | IPv4 FIB 20k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale20k-ndrdisc | 8.5 | 9.0 | | % |
\r
136 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
137 | IPv4 FIB 2M | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale2m-ndrdisc | 8.5 | 7.8 | | % |
\r
138 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
139 | IPv4 Policer | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc | 6.9 | 7.4 | | % |
\r
140 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
141 | IPv4 LISP | 10ge2p1x520: 64B-1t1c-ethip4lispip4-ip4base-ndrdisc | 4.4 | 4.8 | | % |
\r
142 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
143 | IPv6 COP | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-copwhtlistbase-ndrdisc | 6.1 | 6.1 | | % |
\r
144 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
145 | IPv6 FIB 200k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale200k-ndrdisc | 6.5 | 5.3 | | % |
\r
146 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
147 | IPv6 FIB 20k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-ndrdisc | 6.9 | 6.5 | | % |
\r
148 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
149 | IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-ndrdisc | 5.3 | 4.2 | | % |
\r
150 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
151 | IPv6 iAcl | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-ndrdisc | 6.5 | 6.1 | | % |
\r
152 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
157 Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%:
\r
159 +-------------------+-----------------------------------------------------------------+-----------+-----------+-------------------------+
\r
160 | VPP Functionality | Test Name | VPP-16.09 | VPP-17.01 | VPP-17.04 | Relative |
\r
161 | | | [Mpps] | [Mpps] | [Mpps] | Change |
\r
162 +===================+=================================================================+===========+===========+===========+=============+
\r
163 | IPv4 FIB 200k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale200k-pdrdisc | 8.5 | 9.0 | | % |
\r
164 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
165 | IPv4 FIB 20k | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale20k-pdrdisc | 8.5 | 9.0 | | % |
\r
166 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
167 | IPv4 FIB 2M | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale2m-pdrdisc | 8.3 | 8.1 | | % |
\r
168 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
169 | IPv4 iAcl | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-iacldstbase-pdrdisc | 7.1 | 7.6 | | % |
\r
170 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
171 | IPv4 Policer | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ipolicemarkbase-pdrdisc | 7.1 | 7.4 | | % |
\r
172 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
173 | IPv4 LISP | 10ge2p1x520: 64B-1t1c-ethip4lispip4-ip4base-pdrdisc | 4.6 | 4.8 | | % |
\r
174 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
175 | IPv6 | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-pdrdisc | 7.7 | 7.3 | | % |
\r
176 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
177 | IPv6 COP | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-copwhtlistbase-pdrdisc | 6.1 | 6.1 | | % |
\r
178 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
179 | IPv6 FIB 200k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale200k-pdrdisc | 6.9 | 5.3 | | % |
\r
180 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
181 | IPv6 FIB 20k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-pdrdisc | 6.9 | 6.5 | | % |
\r
182 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
183 | IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-pdrdisc | 5.3 | 4.2 | | % |
\r
184 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
185 | IPv6 iAcl | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-pdrdisc | 6.5 | 6.1 | | % |
\r
186 +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-------------+
\r
188 Known Issues - to be updated
\r
191 Here is the list of known issues in CSIT |release| for VPP performance tests: (to be updated)
\r
193 +---+-------------------------------------------------+-----------------------------------------------------------------+
\r
194 | # | Issue | Description |
\r
195 +---+-------------------------------------------------+-----------------------------------------------------------------+
\r
196 | 1 | Sporadic IPv4 routed-forwarding NDR discovery | Suspected issue with DPDK IPv4 checksum calculation. |
\r
197 | | test failures for 1518B frame size | Observed frequency: sporadic, ca. 20% to 30% of test runs. |
\r
198 +---+-------------------------------------------------+-----------------------------------------------------------------+
\r
199 | 3 | Vic1385 and Vic1227 performance | Low performance of NDR results. Big difference between NDR and |
\r
201 +---+-------------------------------------------------+-----------------------------------------------------------------+
\r
202 | 4 | Sporadic NDR discovery test failures on x520 | Suspected issue with HW settings (BIOS, FW) in LF |
\r
203 | | | infrastructure. Issue can't be replicated outside LF. |
\r
204 +---+-------------------------------------------------+-----------------------------------------------------------------+
\r
205 | 5 | Non-repeatible zig-zagging NDR throughput | Suspected NIC driver issue in multi-thread multi-core operation |
\r
206 | | results for multi-thread multi-core tests | affectin NDR throughput. Applies to XL710 and X710 NICs tested. |
\r
207 | | - 2t2c, 4t4c - for some tested NICs. | |
\r
208 +---+-------------------------------------------------+-----------------------------------------------------------------+
\r