4 CSIT VPP functional tests are executed in FD.io VIRL testbeds. The
5 physical VIRL testbed infrastructure consists of three VIRL servers:
11 - VIRL STD server version: 0.10.32.16
12 - VIRL UWM server version: 0.10.32.16
18 - VIRL STD server version: 0.10.32.16
19 - VIRL UWM server version: 0.10.32.16
25 - VIRL STD server version: 0.10.32.19
26 - VIRL UWM server version: 0.10.32.19
28 - VIRL hosts: Cisco UCS C240-M4, each with 2x Intel Xeon E5-2699
29 v3 (2.30 GHz, 18c), 512GB RAM.
31 Whenever a patch is submitted to gerrit for review, parallel VIRL
32 simulations are started to reduce the time of execution of all
33 functional tests. The number of parallel VIRL simulations is equal to a
34 number of test groups defined by TEST_GROUPS variable in
35 :file:`csit/bootstrap.sh` file. VIRL host to run VIRL simulation is
36 selected based on least load algorithm per VIRL simulation.
38 Every VIRL simulation uses the same three-node logical ring topology -
39 Traffic Generator (TG node) and two Systems Under Test (SUT1 and SUT2).
40 The appropriate pre-built VPP packages built by Jenkins for the patch
41 under review are then installed on the two SUTs, along with their
42 :file:`/etc/vpp/startup.conf` file, in all VIRL simulations.
44 SUT Settings - VIRL Guest VM
45 ----------------------------
47 SUT VMs' settings are defined in `VIRL topologies directory`_
49 - List of SUT VM interfaces:
51 <interface id="0" name="GigabitEthernet0/4/0"/>
52 <interface id="1" name="GigabitEthernet0/5/0"/>
53 <interface id="2" name="GigabitEthernet0/6/0"/>
54 <interface id="3" name="GigabitEthernet0/7/0"/>
56 - Number of 2MB hugepages: 1024.
58 - Maximum number of memory map areas: 20000.
60 - Kernel Shared Memory Max: 2147483648 (vm.nr_hugepages * 2 * 1024 * 1024).
62 SUT Settings - VIRL Guest OS Linux
63 ----------------------------------
65 In CSIT terminology, the VM operating system for both SUTs that |vpp-release|
66 has been tested with, is the following:
70 This image implies Ubuntu 16.04.1 LTS, current as of yyyy-mm-dd (that is,
71 package versions are those that would have been installed by a
72 :command:`apt-get update`, :command:`apt-get upgrade` on that day), produced
73 by CSIT disk image build scripts.
75 The exact list of installed packages and their versions (including the Linux
76 kernel package version) are included in `VIRL ubuntu images lists`_.
78 A replica of this VM image can be built by running the :command:`build.sh`
79 script in CSIT repository.
83 This image implies Centos 7.4-1711, current as of yyyy-mm-dd (that is,
84 package versions are those that would have been installed by a
85 :command:`yum update`, :command:`yum upgrade` on that day), produced
86 by CSIT disk image build scripts.
88 The exact list of installed packages and their versions (including the Linux
89 kernel package version) are included in `VIRL centos images lists`_.
91 A replica of this VM image can be built by running the :command:`build.sh`
92 script in CSIT repository.
96 In addition to the "main" VM image, tests which require VPP to communicate to
97 a VM over a vhost-user interface, utilize a "nested" VM image.
99 This "nested" VM is dynamically created and destroyed as part of a test case,
100 and therefore the "nested" VM image is optimized to be small, lightweight and
101 have a short boot time. The "nested" VM image is not built around any
102 established Linux distribution, but is based on `BuildRoot
103 <https://buildroot.org/>`_, a tool for building embedded Linux systems. Just
104 as for the "main" image, scripts to produce an identical replica of the
105 "nested" image are included in CSIT GIT repository, and the image can be
106 rebuilt using the "build.sh" script at `VIRL nested`_.
111 Every System Under Test runs VPP SW application in Linux user-mode as a Device
112 Under Test (DUT) node.
114 DUT Port Configuration
115 ~~~~~~~~~~~~~~~~~~~~~~
117 Port configuration of DUTs is defined in topology file that is generated per
118 VIRL simulation based on the definition stored in `VIRL topologies directory`_.
120 Example of DUT nodes configuration:
136 -----BEGIN RSA PRIVATE KEY-----
137 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
138 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
139 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
140 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
141 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
142 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
143 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
144 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
145 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
146 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
147 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
148 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
149 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
150 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
151 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
152 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
153 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
154 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
155 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
156 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
157 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
158 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
159 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
160 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
161 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
162 -----END RSA PRIVATE KEY-----
166 mac_address: "fa:16:3e:9b:89:52"
167 pci_address: "0000:00:04.0"
170 mac_address: "fa:16:3e:7a:33:60"
171 pci_address: "0000:00:05.0"
174 mac_address: "fa:16:3e:29:b7:ae"
175 pci_address: "0000:00:06.0"
178 mac_address: "fa:16:3e:76:8d:ff"
179 pci_address: "0000:00:07.0"
193 -----BEGIN RSA PRIVATE KEY-----
194 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
195 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
196 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
197 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
198 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
199 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
200 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
201 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
202 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
203 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
204 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
205 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
206 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
207 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
208 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
209 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
210 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
211 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
212 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
213 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
214 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
215 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
216 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
217 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
218 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
219 -----END RSA PRIVATE KEY-----
223 mac_address: "fa:16:3e:ad:6c:7d"
224 pci_address: "0000:00:04.0"
227 mac_address: "fa:16:3e:94:a4:99"
228 pci_address: "0000:00:05.0"
231 mac_address: "fa:16:3e:75:92:da"
232 pci_address: "0000:00:06.0"
235 mac_address: "fa:16:3e:2c:b1:2a"
236 pci_address: "0000:00:07.0"
244 VPP Installed Packages - Ubuntu
245 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
250 ii libvppinfra 19.08-release amd64 Vector Packet Processing--runtime libraries
251 ii libvppinfra-dev 19.08-release amd64 Vector Packet Processing--runtime libraries
252 ii python3-vpp-api 19.08-release amd64 VPP Python3 API bindings
253 ii vpp 19.08-release amd64 Vector Packet Processing--executables
254 ii vpp-api-python 19.08-release amd64 VPP Python API bindings
255 ii vpp-dbg 19.08-release amd64 Vector Packet Processing--debug symbols
256 ii vpp-dev 19.08-release amd64 Vector Packet Processing--development support
257 ii vpp-plugin-core 19.08-release amd64 Vector Packet Processing--runtime core plugins
258 ii vpp-plugin-dpdk 19.08-release amd64 Vector Packet Processing--runtime dpdk plugin
260 VPP Installed Packages - Centos
261 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
270 Install Date: Wed 28 Aug 2019 01:35:57 AM EDT
271 Group : Development/Libraries
275 Source RPM : vpp-19.08-release.src.rpm
276 Build Date : Wed 21 Aug 2019 02:30:08 PM EDT
277 Build Host : 05ab23f629ba
278 Relocations : (not relocatable)
279 Summary : VPP api python bindings
281 This package contains the python bindings for the vpp api
286 Install Date: Wed 28 Aug 2019 01:35:57 AM EDT
291 Source RPM : vpp-19.08-release.src.rpm
292 Build Date : Wed 21 Aug 2019 02:30:08 PM EDT
293 Build Host : 05ab23f629ba
294 Relocations : (not relocatable)
295 Summary : Vector Packet Processing
297 This package provides VPP executables: vpp, vpp_api_test, vpp_json_test
298 vpp - the vector packet engine
299 vpp_api_test - vector packet engine API test tool
300 vpp_json_test - vector packet engine JSON test tool
305 Install Date: Wed 28 Aug 2019 01:35:57 AM EDT
306 Group : System Environment/Libraries
310 Source RPM : vpp-19.08-release.src.rpm
311 Build Date : Wed 21 Aug 2019 02:30:08 PM EDT
312 Build Host : 05ab23f629ba
313 Relocations : (not relocatable)
314 Summary : VPP libraries
316 This package contains the VPP shared libraries, including:
317 vppinfra - foundation library supporting vectors, hashes, bitmaps, pools, and string formatting.
319 vlib - vector processing library
320 vlib-api - binary API library
321 vnet - network stack library
326 Install Date: Wed 28 Aug 2019 01:35:59 AM EDT
327 Group : Development/Libraries
331 Source RPM : vpp-19.08-release.src.rpm
332 Build Date : Wed 21 Aug 2019 02:30:08 PM EDT
333 Build Host : 05ab23f629ba
334 Relocations : (not relocatable)
335 Summary : VPP header files, static libraries
337 This package contains the header files for VPP.
338 Install this package if you want to write a
339 program for compilation and linking with vpp lib.
342 vnet - devices, classify, dhcp, ethernet flow, gre, ip, etc.
345 Name : vpp-selinux-policy
349 Install Date: Wed 28 Aug 2019 01:35:56 AM EDT
350 Group : System Environment/Base
354 Source RPM : vpp-19.08-release.src.rpm
355 Build Date : Wed 21 Aug 2019 02:30:08 PM EDT
356 Build Host : 05ab23f629ba
357 Relocations : (not relocatable)
358 Summary : VPP Security-Enhanced Linux (SELinux) policy
360 This package contains a tailored VPP SELinux policy
365 Install Date: Wed 28 Aug 2019 01:35:59 AM EDT
366 Group : System Environment/Libraries
370 Source RPM : vpp-19.08-release.src.rpm
371 Build Date : Wed 21 Aug 2019 02:30:08 PM EDT
372 Build Host : 05ab23f629ba
373 Relocations : (not relocatable)
374 Summary : Vector Packet Processing--runtime plugins
376 This package contains VPP plugins
378 VPP Startup Configuration
379 ~~~~~~~~~~~~~~~~~~~~~~~~~
381 VPP startup configuration is common for all test cases.
383 **Common Configuration**
385 There is used the default startup configuration as defined in `VPP startup.conf`_
390 Traffic Generator node is VM running the same OS Linux as SUTs. Ports of this
391 VM are used as source (Tx) and destination (Rx) ports for the traffic.
393 Traffic scripts of test cases are executed on this VM.
398 Configuration of the TG VMs is defined in `VIRL topologies directory`_.
400 /csit/resources/tools/virl/topologies/double-ring-nested.xenial.virl
402 - List of TG VM interfaces:::
404 <interface id="0" name="eth1"/>
405 <interface id="1" name="eth2"/>
406 <interface id="2" name="eth3"/>
407 <interface id="3" name="eth4"/>
408 <interface id="4" name="eth5"/>
409 <interface id="5" name="eth6"/>
411 TG Port Configuration
412 ~~~~~~~~~~~~~~~~~~~~~
414 Port configuration of TG is defined in topology file that is generated per VIRL
415 simulation based on the definition stored in `VIRL topologies directory`_.
417 Example of TG node configuration:::
426 -----BEGIN RSA PRIVATE KEY-----
427 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
428 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
429 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
430 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
431 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
432 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
433 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
434 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
435 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
436 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
437 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
438 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
439 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
440 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
441 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
442 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
443 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
444 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
445 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
446 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
447 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
448 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
449 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
450 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
451 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
452 -----END RSA PRIVATE KEY-----
456 mac_address: "fa:16:3e:b9:e1:27"
457 pci_address: "0000:00:06.0"
461 mac_address: "fa:16:3e:e9:c8:68"
462 pci_address: "0000:00:07.0"
466 mac_address: "fa:16:3e:e8:d3:47"
467 pci_address: "0000:00:08.0"
471 mac_address: "fa:16:3e:cf:ca:58"
472 pci_address: "0000:00:09.0"
479 Functional tests utilize Scapy as a traffic generator. Scapy v2.3.1 is
480 used for |vpp-release| tests.