4 CSIT VPP functional tests are executed in FD.io VIRL testbeds. The
5 physical VIRL testbed infrastructure consists of three VIRL servers:
11 - VIRL STD server version: 0.10.32.16
12 - VIRL UWM server version: 0.10.32.16
18 - VIRL STD server version: 0.10.32.16
19 - VIRL UWM server version: 0.10.32.16
25 - VIRL STD server version: 0.10.32.19
26 - VIRL UWM server version: 0.10.32.19
28 - VIRL hosts: Cisco UCS C240-M4, each with 2x Intel Xeon E5-2699
29 v3 (2.30 GHz, 18c), 512GB RAM.
31 Whenever a patch is submitted to gerrit for review, parallel VIRL
32 simulations are started to reduce the time of execution of all
33 functional tests. The number of parallel VIRL simulations is equal to a
34 number of test groups defined by TEST_GROUPS variable in
35 :file:`csit/bootstrap.sh` file. VIRL host to run VIRL simulation is
36 selected based on least load algorithm per VIRL simulation.
38 Every VIRL simulation uses the same three-node logical ring topology -
39 Traffic Generator (TG node) and two Systems Under Test (SUT1 and SUT2).
40 The appropriate pre-built VPP packages built by Jenkins for the patch
41 under review are then installed on the two SUTs, along with their
42 :file:`/etc/vpp/startup.conf` file, in all VIRL simulations.
44 SUT Settings - VIRL Guest VM
45 ----------------------------
47 SUT VMs' settings are defined in `VIRL topologies directory`_
49 - List of SUT VM interfaces:
51 <interface id="0" name="GigabitEthernet0/4/0"/>
52 <interface id="1" name="GigabitEthernet0/5/0"/>
53 <interface id="2" name="GigabitEthernet0/6/0"/>
54 <interface id="3" name="GigabitEthernet0/7/0"/>
56 - Number of 2MB hugepages: 1024.
58 - Maximum number of memory map areas: 20000.
60 - Kernel Shared Memory Max: 2147483648 (vm.nr_hugepages * 2 * 1024 * 1024).
62 SUT Settings - VIRL Guest OS Linux
63 ----------------------------------
65 In CSIT terminology, the VM operating system for both SUTs that |vpp-release|
66 has been tested with, is the following:
70 This image implies Ubuntu 16.04.1 LTS, current as of yyyy-mm-dd (that is,
71 package versions are those that would have been installed by a
72 :command:`apt-get update`, :command:`apt-get upgrade` on that day), produced
73 by CSIT disk image build scripts.
75 The exact list of installed packages and their versions (including the Linux
76 kernel package version) are included in `VIRL ubuntu images lists`_.
78 A replica of this VM image can be built by running the :command:`build.sh`
79 script in CSIT repository.
83 This image implies Centos 7.4-1711, current as of yyyy-mm-dd (that is,
84 package versions are those that would have been installed by a
85 :command:`yum update`, :command:`yum upgrade` on that day), produced
86 by CSIT disk image build scripts.
88 The exact list of installed packages and their versions (including the Linux
89 kernel package version) are included in `VIRL centos images lists`_.
91 A replica of this VM image can be built by running the :command:`build.sh`
92 script in CSIT repository.
96 In addition to the "main" VM image, tests which require VPP to communicate to
97 a VM over a vhost-user interface, utilize a "nested" VM image.
99 This "nested" VM is dynamically created and destroyed as part of a test case,
100 and therefore the "nested" VM image is optimized to be small, lightweight and
101 have a short boot time. The "nested" VM image is not built around any
102 established Linux distribution, but is based on `BuildRoot
103 <https://buildroot.org/>`_, a tool for building embedded Linux systems. Just
104 as for the "main" image, scripts to produce an identical replica of the
105 "nested" image are included in CSIT GIT repository, and the image can be
106 rebuilt using the "build.sh" script at `VIRL nested`_.
111 Every System Under Test runs VPP SW application in Linux user-mode as a Device
112 Under Test (DUT) node.
114 DUT Port Configuration
115 ~~~~~~~~~~~~~~~~~~~~~~
117 Port configuration of DUTs is defined in topology file that is generated per
118 VIRL simulation based on the definition stored in `VIRL topologies directory`_.
120 Example of DUT nodes configuration:
136 -----BEGIN RSA PRIVATE KEY-----
137 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
138 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
139 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
140 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
141 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
142 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
143 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
144 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
145 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
146 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
147 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
148 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
149 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
150 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
151 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
152 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
153 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
154 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
155 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
156 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
157 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
158 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
159 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
160 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
161 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
162 -----END RSA PRIVATE KEY-----
166 mac_address: "fa:16:3e:9b:89:52"
167 pci_address: "0000:00:04.0"
170 mac_address: "fa:16:3e:7a:33:60"
171 pci_address: "0000:00:05.0"
174 mac_address: "fa:16:3e:29:b7:ae"
175 pci_address: "0000:00:06.0"
178 mac_address: "fa:16:3e:76:8d:ff"
179 pci_address: "0000:00:07.0"
193 -----BEGIN RSA PRIVATE KEY-----
194 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
195 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
196 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
197 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
198 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
199 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
200 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
201 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
202 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
203 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
204 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
205 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
206 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
207 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
208 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
209 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
210 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
211 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
212 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
213 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
214 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
215 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
216 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
217 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
218 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
219 -----END RSA PRIVATE KEY-----
223 mac_address: "fa:16:3e:ad:6c:7d"
224 pci_address: "0000:00:04.0"
227 mac_address: "fa:16:3e:94:a4:99"
228 pci_address: "0000:00:05.0"
231 mac_address: "fa:16:3e:75:92:da"
232 pci_address: "0000:00:06.0"
235 mac_address: "fa:16:3e:2c:b1:2a"
236 pci_address: "0000:00:07.0"
244 VPP Installed Packages - Ubuntu
245 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
250 ii libvppinfra 19.04-release amd64 Vector Packet Processing--runtime libraries
251 ii libvppinfra-dev 19.04-release amd64 Vector Packet Processing--runtime libraries
252 ii python3-vpp-api 19.04-release amd64 VPP Python3 API bindings
253 ii vpp 19.04-release amd64 Vector Packet Processing--executables
254 ii vpp-api-python 19.04-release amd64 VPP Python API bindings
255 ii vpp-dbg 19.04-release amd64 Vector Packet Processing--debug symbols
256 ii vpp-dev 19.04-release amd64 Vector Packet Processing--development support
257 ii vpp-plugin-core 19.04-release amd64 Vector Packet Processing--runtime core plugins
258 ii vpp-plugin-dpdk 19.04-release amd64 Vector Packet Processing--runtime dpdk plugin
260 VPP Installed Packages - Centos
261 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
270 Install Date: Thu 25 Apr 2019 04:14:51 AM EDT
271 Group : System Environment/Libraries
275 Source RPM : vpp-19.04-release.src.rpm
276 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
277 Build Host : 940fc1a9327e
278 Relocations : (not relocatable)
279 Summary : VPP libraries
281 This package contains the VPP shared libraries, including:
282 vppinfra - foundation library supporting vectors, hashes, bitmaps, pools, and string formatting.
284 vlib - vector processing library
285 vlib-api - binary API library
286 vnet - network stack library
291 Install Date: Thu 25 Apr 2019 04:14:52 AM EDT
292 Group : Development/Libraries
296 Source RPM : vpp-19.04-release.src.rpm
297 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
298 Build Host : 940fc1a9327e
299 Relocations : (not relocatable)
300 Summary : VPP header files, static libraries
302 This package contains the header files for VPP.
303 Install this package if you want to write a
304 program for compilation and linking with vpp lib.
307 vnet - devices, classify, dhcp, ethernet flow, gre, ip, etc.
310 Name : vpp-selinux-policy
314 Install Date: Thu 25 Apr 2019 04:14:49 AM EDT
315 Group : System Environment/Base
319 Source RPM : vpp-19.04-release.src.rpm
320 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
321 Build Host : 940fc1a9327e
322 Relocations : (not relocatable)
323 Summary : VPP Security-Enhanced Linux (SELinux) policy
325 This package contains a tailored VPP SELinux policy
330 Install Date: Thu 25 Apr 2019 04:14:51 AM EDT
331 Group : System Environment/Libraries
335 Source RPM : vpp-19.04-release.src.rpm
336 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
337 Build Host : 940fc1a9327e
338 Relocations : (not relocatable)
339 Summary : Vector Packet Processing--runtime plugins
341 This package contains VPP plugins
342 Name : vpp-api-python
346 Install Date: Thu 25 Apr 2019 04:14:51 AM EDT
347 Group : Development/Libraries
351 Source RPM : vpp-19.04-release.src.rpm
352 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
353 Build Host : 940fc1a9327e
354 Relocations : (not relocatable)
355 Summary : VPP api python bindings
357 This package contains the python bindings for the vpp api
362 Install Date: Thu 25 Apr 2019 04:14:51 AM EDT
367 Source RPM : vpp-19.04-release.src.rpm
368 Build Date : Tue 23 Apr 2019 08:46:26 PM EDT
369 Build Host : 940fc1a9327e
370 Relocations : (not relocatable)
371 Summary : Vector Packet Processing
373 This package provides VPP executables: vpp, vpp_api_test, vpp_json_test
374 vpp - the vector packet engine
375 vpp_api_test - vector packet engine API test tool
376 vpp_json_test - vector packet engine JSON test tool
378 VPP Startup Configuration
379 ~~~~~~~~~~~~~~~~~~~~~~~~~
381 VPP startup configuration is common for all test cases except test cases related
384 **Common Configuration**
386 There is used the default startup configuration as defined in `VPP startup.conf`_
388 **SW Crypto Device Configuration**
392 $ cat /etc/vpp/startup.conf
395 cli-listen /run/vpp/cli.sock
410 vdev cryptodev_aesni_gcm_pmd,socket_id=0
411 vdev cryptodev_aesni_mb_pmd,socket_id=0
417 Traffic Generator node is VM running the same OS Linux as SUTs. Ports of this
418 VM are used as source (Tx) and destination (Rx) ports for the traffic.
420 Traffic scripts of test cases are executed on this VM.
425 Configuration of the TG VMs is defined in `VIRL topologies directory`_.
427 /csit/resources/tools/virl/topologies/double-ring-nested.xenial.virl
429 - List of TG VM interfaces:::
431 <interface id="0" name="eth1"/>
432 <interface id="1" name="eth2"/>
433 <interface id="2" name="eth3"/>
434 <interface id="3" name="eth4"/>
435 <interface id="4" name="eth5"/>
436 <interface id="5" name="eth6"/>
438 TG Port Configuration
439 ~~~~~~~~~~~~~~~~~~~~~
441 Port configuration of TG is defined in topology file that is generated per VIRL
442 simulation based on the definition stored in `VIRL topologies directory`_.
444 Example of TG node configuration:::
453 -----BEGIN RSA PRIVATE KEY-----
454 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
455 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
456 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
457 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
458 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
459 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
460 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
461 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
462 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
463 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
464 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
465 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
466 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
467 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
468 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
469 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
470 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
471 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
472 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
473 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
474 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
475 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
476 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
477 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
478 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
479 -----END RSA PRIVATE KEY-----
483 mac_address: "fa:16:3e:b9:e1:27"
484 pci_address: "0000:00:06.0"
488 mac_address: "fa:16:3e:e9:c8:68"
489 pci_address: "0000:00:07.0"
493 mac_address: "fa:16:3e:e8:d3:47"
494 pci_address: "0000:00:08.0"
498 mac_address: "fa:16:3e:cf:ca:58"
499 pci_address: "0000:00:09.0"
506 Functional tests utilize Scapy as a traffic generator. Scapy v2.3.1 is
507 used for |vpp-release| tests.