4 CSIT functional tests are currently executed in FD.IO VIRL testbed. The physical
\r
5 VIRL testbed infrastructure consists of three identical VIRL hosts:
\r
7 - All hosts are Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @2.30GHz,
\r
11 - Status: Production
\r
12 - OS: Ubuntu 16.04.2
\r
13 - STD server version 0.10.32.16
\r
14 - UWM server version 0.10.32.16
\r
18 - Status: Production
\r
19 - OS: Ubuntu 16.04.2
\r
20 - STD server version 0.10.32.16
\r
21 - UWM server version 0.10.32.16
\r
25 - Status: Production
\r
26 - OS: Ubuntu 14.04.3
\r
27 - STD server version 0.10.24.7
\r
28 - UWM server version 0.10.24.7
\r
30 Whenever a patch is submitted to gerrit for review, parallel VIRL simulations
\r
31 are started to reduce the time of execution of all functional tests. The number
\r
32 of parallel VIRL simulations is equal to number of test groups defined by
\r
33 TEST_GROUPS variable in csit/bootstrap.sh file. The VIRL host to run VIRL
\r
34 simulation is selected based on least load algorithm per VIRL simulation.
\r
36 Every VIRL simulation uses the same three-node - Traffic Generator (TG node) and
\r
37 two Systems Under Test (SUT1 and SUT2) - "double-ring" topology. The appropriate
\r
38 pre-built VPP packages built by Jenkins for the patch under review are then
\r
39 installed on the two SUTs, along with their /etc/vpp/startup.conf file, in all
\r
42 SUT Configuration - VIRL Guest VM
\r
43 ---------------------------------
\r
45 Configuration of the SUT VMs is defined in file
\r
47 /csit/resources/tools/virl/topologies/double-ring-nested.xenial.virl
\r
49 - List of SUT VM interfaces:::
\r
51 <interface id="0" name="GigabitEthernet0/4/0"/>
\r
52 <interface id="1" name="GigabitEthernet0/5/0"/>
\r
53 <interface id="2" name="GigabitEthernet0/6/0"/>
\r
54 <interface id="3" name="GigabitEthernet0/7/0"/>
\r
56 - Number of 2MB hugepages: 1024
\r
58 - Maximum number of memory map areas: 20000
\r
60 - Kernel Shared Memory Max: 2147483648 (vm.nr_hugepages * 2 * 1024 * 1024)
\r
62 SUT Configuration - VIRL Guest OS Linux
\r
63 ---------------------------------------
\r
65 In CSIT terminology, the VM operating system for both SUTs that |vpp-release| has
\r
66 been tested with, is the following:
\r
68 **#. Ubuntu VIRL image**
\r
70 This image implies Ubuntu 16.04.1 LTS, current as of yyyy-mm-dd (that is,
\r
71 package versions are those that would have been installed by a "apt-get update",
\r
72 "apt-get upgrade" on that day), produced by CSIT disk image build scripts.
\r
74 The exact list of installed packages and their versions (including the Linux
\r
75 kernel package version) are included in CSIT source repository:
\r
77 resources/tools/disk-image-builder/ubuntu/lists/|virl-image-ubuntu|
\r
79 A replica of this VM image can be built by running the "build.sh" script in CSIT
\r
80 repository resources/tools/disk-image-builder/ubuntu.
\r
82 **#. CentOS VIRL image**
\r
84 The Centos7.3 image is ready to be used but no tests running on it now.
\r
85 Corresponding Jenkins jobs are under preparation.
\r
87 The exact list of installed packages and their versions (including the Linux
\r
88 kernel package version) are included in CSIT source repository:
\r
90 resources/tools/disk-image-builder/ubuntu/lists/|virl-image-centos|
\r
92 A replica of this VM image can be built by running the "build.sh" script in CSIT
\r
93 repository resources/tools/disk-image-builder/centos.
\r
95 **#. Nested VM image**
\r
97 In addition to the "main" VM image, tests which require VPP to communicate to a
\r
98 VM over a vhost-user interface, utilize a "nested" VM image.
\r
100 This "nested" VM is dynamically created and destroyed as part of a test case,
\r
101 and therefore the "nested" VM image is optimized to be small, lightweight and
\r
102 have a short boot time. The "nested" VM image is not built around any
\r
103 established Linux distribution, but is based on `BuildRoot
\r
104 <https://buildroot.org/>`_, a tool for building embedded Linux systems. Just as
\r
105 for the "main" image, scripts to produce an identical replica of the "nested"
\r
106 image are included in CSIT GIT repository, and the image can be rebuilt using
\r
107 the "build.sh" script at:
\r
109 resources/tools/disk-image-builder/ubuntu/lists/nested
\r
111 DUT Configuration - VPP
\r
112 -----------------------
\r
114 Every System Under Test runs VPP SW application in Linux user-mode as a Device
\r
115 Under Test (DUT) node.
\r
117 **DUT port configuration**
\r
119 Port configuration of DUTs is defined in topology file that is generated per
\r
120 VIRL simulation based on the definition stored in file
\r
122 /csit/resources/tools/virl/topologies/double-ring-nested.xenial.yaml
\r
124 Example of DUT nodes configuration:::
\r
128 host: "10.30.51.157"
\r
137 -----BEGIN RSA PRIVATE KEY-----
\r
138 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
\r
139 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
\r
140 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
\r
141 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
\r
142 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
\r
143 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
\r
144 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
\r
145 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
\r
146 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
\r
147 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
\r
148 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
\r
149 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
\r
150 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
\r
151 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
\r
152 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
\r
153 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
\r
154 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
\r
155 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
\r
156 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
\r
157 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
\r
158 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
\r
159 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
\r
160 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
\r
161 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
\r
162 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
\r
163 -----END RSA PRIVATE KEY-----
\r
167 mac_address: "fa:16:3e:9b:89:52"
\r
168 pci_address: "0000:00:04.0"
\r
171 mac_address: "fa:16:3e:7a:33:60"
\r
172 pci_address: "0000:00:05.0"
\r
175 mac_address: "fa:16:3e:29:b7:ae"
\r
176 pci_address: "0000:00:06.0"
\r
179 mac_address: "fa:16:3e:76:8d:ff"
\r
180 pci_address: "0000:00:07.0"
\r
184 host: "10.30.51.156"
\r
193 -----BEGIN RSA PRIVATE KEY-----
\r
194 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
\r
195 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
\r
196 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
\r
197 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
\r
198 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
\r
199 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
\r
200 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
\r
201 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
\r
202 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
\r
203 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
\r
204 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
\r
205 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
\r
206 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
\r
207 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
\r
208 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
\r
209 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
\r
210 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
\r
211 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
\r
212 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
\r
213 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
\r
214 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
\r
215 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
\r
216 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
\r
217 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
\r
218 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
\r
219 -----END RSA PRIVATE KEY-----
\r
223 mac_address: "fa:16:3e:ad:6c:7d"
\r
224 pci_address: "0000:00:04.0"
\r
227 mac_address: "fa:16:3e:94:a4:99"
\r
228 pci_address: "0000:00:05.0"
\r
231 mac_address: "fa:16:3e:75:92:da"
\r
232 pci_address: "0000:00:06.0"
\r
235 mac_address: "fa:16:3e:2c:b1:2a"
\r
236 pci_address: "0000:00:07.0"
\r
243 **VPP Installed Packages**
\r
247 Desired=Unknown/Install/Remove/Purge/Hold
\r
248 | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
\r
249 |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
\r
250 ||/ Name Version Architecture Description
\r
251 +++-==============-=============-============-=============================================
\r
252 ii vpp 17.07-release amd64 Vector Packet Processing--executables
\r
253 ii vpp-dbg 17.07-release amd64 Vector Packet Processing--debug symbols
\r
254 ii vpp-dev 17.07-release amd64 Vector Packet Processing--development support
\r
255 ii vpp-dpdk-dev 17.07-release amd64 Vector Packet Processing--development support
\r
256 ii vpp-dpdk-dkms 17.07-release amd64 DPDK 2.1 igb_uio_driver
\r
257 ii vpp-lib 17.07-release amd64 Vector Packet Processing--runtime libraries
\r
258 ii vpp-plugins 17.07-release amd64 Vector Packet Processing--runtime plugins
\r
260 **VPP Startup Configuration**
\r
262 VPP startup configuration is common for all test cases.
\r
266 $ cat /etc/vpp/startup.conf
\r
282 ## In the VPP there is one main thread and optionally the user can create worker(s)
\r
283 ## The main thread and worker thread(s) can be pinned to CPU core(s) manually or automatically
\r
285 ## Manual pinning of thread(s) to CPU core(s)
\r
287 ## Set logical CPU core where main thread runs
\r
290 ## Set logical CPU core(s) where worker threads are running
\r
291 # corelist-workers 2-3,18-19
\r
293 ## Automatic pinning of thread(s) to CPU core(s)
\r
295 ## Sets number of CPU core(s) to be skipped (1 ... N-1)
\r
296 ## Skipped CPU core(s) are not used for pinning main thread and working thread(s).
\r
297 ## The main thread is automatically pinned to the first available CPU core and worker(s)
\r
298 ## are pinned to next free CPU core(s) after core assigned to main thread
\r
301 ## Specify a number of workers to be created
\r
302 ## Workers are pinned to N consecutive CPU cores while skipping "skip-cores" CPU core(s)
\r
303 ## and main thread's CPU core
\r
306 ## Set scheduling policy and priority of main and worker threads
\r
308 ## Scheduling policy options are: other (SCHED_OTHER), batch (SCHED_BATCH)
\r
309 ## idle (SCHED_IDLE), fifo (SCHED_FIFO), rr (SCHED_RR)
\r
310 # scheduler-policy fifo
\r
312 ## Scheduling priority is used only for "real-time policies (fifo and rr),
\r
313 ## and has to be in the range of priorities supported for a particular policy
\r
314 # scheduler-priority 50
\r
318 ## Change default settings for all intefaces
\r
320 ## Number of receive queues, enables RSS
\r
324 ## Number of transmit queues, Default is equal
\r
325 ## to number of worker threads or 1 if no workers treads
\r
328 ## Number of descriptors in transmit and receive rings
\r
329 ## increasing or reducing number can impact performance
\r
330 ## Default is 1024 for both rx and tx
\r
334 ## VLAN strip offload mode for interface
\r
336 # vlan-strip-offload on
\r
339 ## Whitelist specific interface by specifying PCI address
\r
342 ## Whitelist specific interface by specifying PCI address and in
\r
343 ## addition specify custom parameters for this interface
\r
344 # dev 0000:02:00.1 {
\r
348 ## Change UIO driver used by VPP, Options are: uio_pci_generic, vfio-pci
\r
349 ## and igb_uio (default)
\r
350 # uio-driver uio_pci_generic
\r
352 ## Disable mutli-segment buffers, improves performance but
\r
353 ## disables Jumbo MTU support
\r
356 ## Increase number of buffers allocated, needed only in scenarios with
\r
357 ## large number of interfaces and worker threads. Value is per CPU socket.
\r
358 ## Default is 32768
\r
361 ## Change hugepages allocation per-socket, needed only if there is need for
\r
362 ## larger number of mbufs. Default is 256M on each detected CPU socket
\r
363 # socket-mem 2048,2048
\r
369 Traffic Generator node is VM running the same OS Linux as SUTs. Ports of this
\r
370 VM are used as source (Tx) and destination (Rx) ports for the traffic.
\r
372 Traffic scripts of test cases are executed on this VM.
\r
374 **TG VM configuration**
\r
376 Configuration of the TG VMs is defined in file
\r
378 /csit/resources/tools/virl/topologies/double-ring-nested.xenial.virl
\r
380 - List of TG VM interfaces:::
\r
382 <interface id="0" name="eth1"/>
\r
383 <interface id="1" name="eth2"/>
\r
384 <interface id="2" name="eth3"/>
\r
385 <interface id="3" name="eth4"/>
\r
386 <interface id="4" name="eth5"/>
\r
387 <interface id="5" name="eth6"/>
\r
389 **TG node port configuration**
\r
391 Port configuration of TG is defined in topology file that is generated per VIRL
\r
392 simulation based on the definition stored in file
\r
394 /csit/resources/tools/virl/topologies/double-ring-nested.xenial.yaml
\r
396 Example of TG node configuration:::
\r
400 host: "10.30.51.155"
\r
404 -----BEGIN RSA PRIVATE KEY-----
\r
405 MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb
\r
406 v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd
\r
407 vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2
\r
408 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc
\r
409 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs
\r
410 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7
\r
411 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN
\r
412 EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN
\r
413 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU
\r
414 MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD
\r
415 p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW
\r
416 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG
\r
417 OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD
\r
418 GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef
\r
419 bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg
\r
420 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu
\r
421 f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u
\r
422 UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/
\r
423 +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT
\r
424 D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/
\r
425 sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn
\r
426 g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY
\r
427 qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8
\r
428 EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD
\r
429 BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN
\r
430 -----END RSA PRIVATE KEY-----
\r
434 mac_address: "fa:16:3e:b9:e1:27"
\r
435 pci_address: "0000:00:06.0"
\r
439 mac_address: "fa:16:3e:e9:c8:68"
\r
440 pci_address: "0000:00:07.0"
\r
444 mac_address: "fa:16:3e:e8:d3:47"
\r
445 pci_address: "0000:00:08.0"
\r
449 mac_address: "fa:16:3e:cf:ca:58"
\r
450 pci_address: "0000:00:09.0"
\r
454 **Traffic generator**
\r
456 Functional tests utilize Scapy as a traffic generator. There was used Scapy
\r
457 v2.3.1 for |vpp-release| tests.
\r