4 CSIT VPP functional tests are executed in FD.io VIRL testbeds. The
5 physical VIRL testbed infrastructure consists of three VIRL servers:
11 - VIRL STD server version: 0.10.32.16
12 - VIRL UWM server version: 0.10.32.16
18 - VIRL STD server version: 0.10.32.16
19 - VIRL UWM server version: 0.10.32.16
25 - VIRL STD server version: 0.10.32.19
26 - VIRL UWM server version: 0.10.32.19
28 - VIRL hosts: Cisco UCS C240-M4, each with 2x Intel Xeon E5-2699
29 v3 (2.30 GHz, 18c), 512GB RAM.
31 Whenever a patch is submitted to gerrit for review, parallel VIRL
32 simulations are started to reduce the time of execution of all
33 functional tests. The number of parallel VIRL simulations is equal to a
34 number of test groups defined by TEST_GROUPS variable in
35 :file:`csit/bootstrap.sh` file. VIRL host to run VIRL simulation is
36 selected based on least load algorithm per VIRL simulation.
38 Every VIRL simulation uses the same three-node logical ring topology -
39 Traffic Generator (TG node) and two Systems Under Test (SUT1 and SUT2).
40 The appropriate pre-built VPP packages built by Jenkins for the patch
41 under review are then installed on the two SUTs, along with their
42 :file:`/etc/vpp/startup.conf` file, in all VIRL simulations.
44 SUT Settings - VIRL Guest VM
45 ----------------------------
47 SUT VMs' settings are defined in `VIRL topologies directory`_
49 - List of SUT VM interfaces:
51 <interface id="0" name="GigabitEthernet0/4/0"/>
52 <interface id="1" name="GigabitEthernet0/5/0"/>
53 <interface id="2" name="GigabitEthernet0/6/0"/>
54 <interface id="3" name="GigabitEthernet0/7/0"/>
56 - Number of 2MB hugepages: 1024.
58 - Maximum number of memory map areas: 20000.
60 - Kernel Shared Memory Max: 2147483648 (vm.nr_hugepages * 2 * 1024 * 1024).
62 SUT Settings - VIRL Guest OS Linux
63 ----------------------------------
65 In CSIT terminology, the VM operating system for both SUTs that |vpp-release|
66 has been tested with, is the following:
70 This image implies Ubuntu 16.04.1 LTS, current as of yyyy-mm-dd (that is,
71 package versions are those that would have been installed by a
72 :command:`apt-get update`, :command:`apt-get upgrade` on that day), produced
73 by CSIT disk image build scripts.
75 The exact list of installed packages and their versions (including the Linux
76 kernel package version) are included in `VIRL ubuntu images lists`_.
78 A replica of this VM image can be built by running the :command:`build.sh`
79 script in CSIT repository.
83 This image implies Centos 7.4-1711, current as of yyyy-mm-dd (that is,
84 package versions are those that would have been installed by a
85 :command:`yum update`, :command:`yum upgrade` on that day), produced
86 by CSIT disk image build scripts.
88 The exact list of installed packages and their versions (including the Linux
89 kernel package version) are included in `VIRL centos images lists`_.
91 A replica of this VM image can be built by running the :command:`build.sh`
92 script in CSIT repository.
96 In addition to the "main" VM image, tests which require VPP to communicate to
97 a VM over a vhost-user interface, utilize a "nested" VM image.
99 This "nested" VM is dynamically created and destroyed as part of a test case,
100 and therefore the "nested" VM image is optimized to be small, lightweight and
101 have a short boot time. The "nested" VM image is not built around any
102 established Linux distribution, but is based on `BuildRoot
103 <https://buildroot.org/>`_, a tool for building embedded Linux systems. Just
104 as for the "main" image, scripts to produce an identical replica of the
105 "nested" image are included in CSIT GIT repository, and the image can be
106 rebuilt using the "build.sh" script at `VIRL nested`_.
111 Every System Under Test runs VPP SW application in Linux user-mode as a Device
112 Under Test (DUT) node.
114 DUT Port Configuration
115 ~~~~~~~~~~~~~~~~~~~~~~
117 Port configuration of DUTs is defined in topology file that is generated per
118 VIRL simulation based on the definition stored in `VIRL topologies directory`_.
120 Example of DUT nodes configuration:
136 -----BEGIN RSA PRIVATE KEY-----
137 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
138 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
139 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
140 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
141 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
142 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
143 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
144 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
145 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
146 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
147 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
148 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
149 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
150 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
151 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
152 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
153 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
154 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
155 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
156 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
157 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
158 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
159 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
160 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
161 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
162 -----END RSA PRIVATE KEY-----
166 mac_address: "fa:16:3e:9b:89:52"
167 pci_address: "0000:00:04.0"
170 mac_address: "fa:16:3e:7a:33:60"
171 pci_address: "0000:00:05.0"
174 mac_address: "fa:16:3e:29:b7:ae"
175 pci_address: "0000:00:06.0"
178 mac_address: "fa:16:3e:76:8d:ff"
179 pci_address: "0000:00:07.0"
193 -----BEGIN RSA PRIVATE KEY-----
194 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
195 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
196 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
197 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
198 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
199 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
200 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
201 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
202 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
203 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
204 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
205 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
206 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
207 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
208 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
209 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
210 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
211 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
212 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
213 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
214 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
215 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
216 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
217 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
218 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
219 -----END RSA PRIVATE KEY-----
223 mac_address: "fa:16:3e:ad:6c:7d"
224 pci_address: "0000:00:04.0"
227 mac_address: "fa:16:3e:94:a4:99"
228 pci_address: "0000:00:05.0"
231 mac_address: "fa:16:3e:75:92:da"
232 pci_address: "0000:00:06.0"
235 mac_address: "fa:16:3e:2c:b1:2a"
236 pci_address: "0000:00:07.0"
244 VPP Installed Packages - Ubuntu
245 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
250 ii libvppinfra 19.04-release amd64 Vector Packet Processing--runtime libraries
251 ii libvppinfra-dev 19.04-release amd64 Vector Packet Processing--runtime libraries
252 ii python3-vpp-api 19.04-release amd64 VPP Python3 API bindings
253 ii vpp 19.04-release amd64 Vector Packet Processing--executables
254 ii vpp-api-python 19.04-release amd64 VPP Python API bindings
255 ii vpp-dbg 19.04-release amd64 Vector Packet Processing--debug symbols
256 ii vpp-dev 19.04-release amd64 Vector Packet Processing--development support
257 ii vpp-plugin-core 19.04-release amd64 Vector Packet Processing--runtime core plugins
258 ii vpp-plugin-dpdk 19.04-release amd64 Vector Packet Processing--runtime dpdk plugin
260 VPP Installed Packages - Centos
261 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
270 Install Date: Thu 25 Apr 2019 04:14:51 AM EDT
271 Group : System Environment/Libraries
275 Source RPM : vpp-19.04-release.src.rpm
276 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
277 Build Host : 940fc1a9327e
278 Relocations : (not relocatable)
279 Summary : VPP libraries
281 This package contains the VPP shared libraries, including:
282 vppinfra - foundation library supporting vectors, hashes, bitmaps, pools, and string formatting.
284 vlib - vector processing library
285 vlib-api - binary API library
286 vnet - network stack library
291 Install Date: Thu 25 Apr 2019 04:14:52 AM EDT
292 Group : Development/Libraries
296 Source RPM : vpp-19.04-release.src.rpm
297 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
298 Build Host : 940fc1a9327e
299 Relocations : (not relocatable)
300 Summary : VPP header files, static libraries
302 This package contains the header files for VPP.
303 Install this package if you want to write a
304 program for compilation and linking with vpp lib.
307 vnet - devices, classify, dhcp, ethernet flow, gre, ip, etc.
310 Name : vpp-selinux-policy
314 Install Date: Thu 25 Apr 2019 04:14:49 AM EDT
315 Group : System Environment/Base
319 Source RPM : vpp-19.04-release.src.rpm
320 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
321 Build Host : 940fc1a9327e
322 Relocations : (not relocatable)
323 Summary : VPP Security-Enhanced Linux (SELinux) policy
325 This package contains a tailored VPP SELinux policy
330 Install Date: Thu 25 Apr 2019 04:14:51 AM EDT
331 Group : System Environment/Libraries
335 Source RPM : vpp-19.04-release.src.rpm
336 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
337 Build Host : 940fc1a9327e
338 Relocations : (not relocatable)
339 Summary : Vector Packet Processing--runtime plugins
341 This package contains VPP plugins
342 Name : vpp-api-python
346 Install Date: Thu 25 Apr 2019 04:14:51 AM EDT
347 Group : Development/Libraries
351 Source RPM : vpp-19.04-release.src.rpm
352 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
353 Build Host : 940fc1a9327e
354 Relocations : (not relocatable)
355 Summary : VPP api python bindings
357 This package contains the python bindings for the vpp api
362 Install Date: Thu 25 Apr 2019 04:14:51 AM EDT
367 Source RPM : vpp-19.04-release.src.rpm
368 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
369 Build Host : 940fc1a9327e
370 Relocations : (not relocatable)
371 Summary : Vector Packet Processing
373 This package provides VPP executables: vpp, vpp_api_test, vpp_json_test
374 vpp - the vector packet engine
375 vpp_api_test - vector packet engine API test tool
376 vpp_json_test - vector packet engine JSON test tool
378 VPP Startup Configuration
379 ~~~~~~~~~~~~~~~~~~~~~~~~~
381 VPP startup configuration is common for all test cases except test cases related
384 **Common Configuration**
386 There is used the default startup configuration as defined in `VPP startup.conf`_
388 **SW Crypto Device Configuration**
392 $ cat /etc/vpp/startup.conf
395 cli-listen /run/vpp/cli.sock
407 vdev cryptodev_aesni_gcm_pmd,socket_id=0
408 vdev cryptodev_aesni_mb_pmd,socket_id=0
414 Traffic Generator node is VM running the same OS Linux as SUTs. Ports of this
415 VM are used as source (Tx) and destination (Rx) ports for the traffic.
417 Traffic scripts of test cases are executed on this VM.
422 Configuration of the TG VMs is defined in `VIRL topologies directory`_.
424 /csit/resources/tools/virl/topologies/double-ring-nested.xenial.virl
426 - List of TG VM interfaces:::
428 <interface id="0" name="eth1"/>
429 <interface id="1" name="eth2"/>
430 <interface id="2" name="eth3"/>
431 <interface id="3" name="eth4"/>
432 <interface id="4" name="eth5"/>
433 <interface id="5" name="eth6"/>
435 TG Port Configuration
436 ~~~~~~~~~~~~~~~~~~~~~
438 Port configuration of TG is defined in topology file that is generated per VIRL
439 simulation based on the definition stored in `VIRL topologies directory`_.
441 Example of TG node configuration:::
450 -----BEGIN RSA PRIVATE KEY-----
451 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
452 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
453 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
454 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
455 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
456 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
457 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
458 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
459 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
460 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
461 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
462 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
463 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
464 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
465 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
466 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
467 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
468 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
469 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
470 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
471 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
472 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
473 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
474 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
475 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
476 -----END RSA PRIVATE KEY-----
480 mac_address: "fa:16:3e:b9:e1:27"
481 pci_address: "0000:00:06.0"
485 mac_address: "fa:16:3e:e9:c8:68"
486 pci_address: "0000:00:07.0"
490 mac_address: "fa:16:3e:e8:d3:47"
491 pci_address: "0000:00:08.0"
495 mac_address: "fa:16:3e:cf:ca:58"
496 pci_address: "0000:00:09.0"
503 Functional tests utilize Scapy as a traffic generator. Scapy v2.3.1 is
504 used for |vpp-release| tests.