X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Freport%2Fvpp_performance_tests%2Ftest_environment.rst;h=aa64d1b5bcf35b37762a6849829bd850f8a71651;hp=c8ea96972831255572267db2feb4ed6d7064861a;hb=94f675829cc82bd511b0b4eabeca5f8f27e2e411;hpb=5d294b7d4c501d971d67c4e3fe192a9dc504d76e diff --git a/docs/report/vpp_performance_tests/test_environment.rst b/docs/report/vpp_performance_tests/test_environment.rst index c8ea969728..aa64d1b5bc 100644 --- a/docs/report/vpp_performance_tests/test_environment.rst +++ b/docs/report/vpp_performance_tests/test_environment.rst @@ -1,3637 +1,109 @@ -Test Environment -================ - -To execute performance tests, there are three identical testbeds, each testbed -consists of two SUTs and one TG. - -SUT Configuration - Host HW ---------------------------- -Host hardware details (CPU, memory, NIC layout) and physical topology are -described in detail in -`LF FDio CSIT testbed wiki page `_. - -**Host configuration** - -- All hosts are Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz, - 18c, 512GB RAM), -- BIOS settings are default except the following: - - - Hyperthreading disabled. - - SpeedStep disabled. - - TurboBoost disabled. - - Power Technology: Performance. - -- Hosts run Ubuntu 16.04.1, kernel 4.4.0-42-generic. -- Linux kernel boot command line option "intel_pstate=disable" is applied to - both SUTs and TG. In addition, on SUTs, only cores 0 and 18 (the first core on - each socket) are available to the Linux operating system and generic tasks, - all other CPU cores are isolated and reserved for VPP. -- In addition to CIMC and Management, each TG has 4x Intel X710 10GB NIC - (=8 ports) and 2x Intel XL710 40GB NIC (=4 ports), whereas each SUT has: - - - 1x Intel X520 NIC (10GB, 2 ports), - - 1x Cisco VIC 1385 (40GB, 2 ports), - - 1x Intel XL710 NIC (40GB, 2 ports), - - 1x Intel X710 NIC (10GB, 2 ports), - - 1x Cisco VIC 1227 (10GB, 2 ports). - - This allows for a total of five ring topologies, each using ports on - specific NIC model, enabling per NIC model benchmarking. - -**NIC models and placement** - -- 0a:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ - Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server - Adapter X520-2 -- 0a:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ - Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server - Adapter X520-2 -- 06:00.0 Ethernet controller: Cisco Systems Inc VIC Ethernet NIC (rev a2) - Subsystem: Cisco Systems Inc VIC 1227 PCIe Ethernet NIC -- 07:00.0 Ethernet controller: Cisco Systems Inc VIC Ethernet NIC (rev a2) - Subsystem: Cisco Systems Inc VIC 1227 PCIe Ethernet NIC -- 13:00.0 Ethernet controller: Cisco Systems Inc VIC Ethernet NIC (rev a2) - Subsystem: Cisco Systems Inc VIC 1385 PCIe Ethernet NIC -- 15:00.0 Ethernet controller: Cisco Systems Inc VIC Ethernet NIC (rev a2) - Subsystem: Cisco Systems Inc VIC 1385 PCIe Ethernet NIC -- 85:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 - for 40GbE QSFP+ (rev 01) Subsystem: Intel Corporation Ethernet Converged - Network Adapter XL710-Q2 -- 85:00.1 Ethernet controller: Intel Corporation Ethernet Controller XL710 - for 40GbE QSFP+ (rev 01) Subsystem: Intel Corporation Ethernet Converged - Network Adapter XL710-Q2 -- 87:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for - 10GbE SFP+ (rev 01) Subsystem: Intel Corporation Ethernet Converged Network - Adapter X710-2 -- 87:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for - 10GbE SFP+ (rev 01) Subsystem: Intel Corporation Ethernet Converged Network - Adapter X710-2 - -SUT Configuration - Host OS Linux ---------------------------------- - -Software details (OS, configuration) are described in -`CSIT/CSIT_LF_testbed `_. - -Below a subset of the configuration: - -.. code-block:: xml - - -
- - - 1 - - - - - 1 - - - - - 0 - - - - - 0 - - - - - 0 - - - - - 0 - - - - - 0 - - - - - 0 - - - - - 0 - - -
-
- - - 127 - - - - - 0 - - - - - 0 - - - - - 0 - - - - - 0 - - - - - 0 - - - - - 1 - - - - - 0 - - - - - - - 0 - - - - - 127 - - - - - 0 - - - - - 0 - - - - - 0 - - - - - 1 - - - - - 127 - - - - - 0 - /dev/null || echo Debian` - GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8" - GRUB_CMDLINE_LINUX="isolcpus=1-17,19-35 nohz_full=1-17,19-35 rcu_nocbs=1-17,19-35 intel_pstate=disable" - - # Uncomment to enable BadRAM filtering, modify to suit your needs - # This works with Linux (no patch required) and with any kernel that obtains - # the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...) - #GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef" - - # Uncomment to disable graphical terminal (grub-pc only) - #GRUB_TERMINAL=console - GRUB_TERMINAL=serial - GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1" - - # The resolution used on graphical terminal - # note that you can use only modes which your graphic card supports via VBE - # you can see them in real GRUB with the command `vbeinfo' - #GRUB_GFXMODE=640x480 - - # Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux - #GRUB_DISABLE_LINUX_UUID=true - - # Uncomment to disable generation of recovery mode menu entries - #GRUB_DISABLE_RECOVERY="true" - - # Uncomment to get a beep at grub start - #GRUB_INIT_TUNE="480 440 1" - ]]> - - - - 1 - - - - - 0 - mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1 - link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 - 6: enp23s0f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 - link/ether 00:fe:c8:e5:68:32 brd ff:ff:ff:ff:ff:ff - ]]> - - - - 0 - - - - - 0 - - -
-
- - - 127 - - - - - 0 - - -
-
- - - 0 - - - - -
-
- - - 1 - - - - - 127 - - - - - 1 - - -
- - -DUT Configuration - VPP ------------------------ - -**VPP Version** - -17.01-release_amd64 - -**VPP Compile Parameters** - -VPP Compile Job: https://jenkins.fd.io/view/vpp/job/vpp-merge-1701-ubuntu1604/ - -**VPP Install Parameters** - -:: - - $ dpkg -i --force-all - -**VPP Startup Configuration** - -VPP startup configuration changes per test case with different settings for CPU -cores, rx-queues and no-multi-seg parameter. Startup config is aligned with -applied test case tag: - -Tagged by **1T1C**:: - - $ cat /etc/vpp/startup.conf - unix { - nodaemon - log /tmp/vpe.log - cli-listen localhost:5002 - full-coredump - } - api-trace { - on - } - cpu { - main-core 0 corelist-workers 1 - } - dpdk { - socket-mem 1024,1024 - dev default { - num-rx-queues 1 - } - dev 0000:0a:00.1 - dev 0000:0a:00.0 - no-multi-seg - } - ip6 { - hash-buckets 2000000 - heap-size 3G - } - -Tagged by **2T1C**:: - - $ cat /etc/vpp/startup.conf - unix { - nodaemon - log /tmp/vpe.log - cli-listen localhost:5002 - full-coredump - } - api-trace { - on - } - cpu { - main-core 0 corelist-workers 1,2 - } - dpdk { - socket-mem 1024,1024 - dev default { - num-rx-queues 1 - } - dev 0000:0a:00.1 - dev 0000:0a:00.0 - no-multi-seg - } - ip6 { - hash-buckets 2000000 - heap-size 3G - } - -Tagged by **4T4C**:: - - $ cat /etc/vpp/startup.conf - unix { - nodaemon - log /tmp/vpe.log - cli-listen localhost:5002 - full-coredump - } - api-trace { - on - } - cpu { - main-core 0 corelist-workers 1,2,3,4 - } - dpdk { - socket-mem 1024,1024 - dev default { - num-rx-queues 1 - } - dev 0000:0a:00.1 - dev 0000:0a:00.0 - no-multi-seg - } - ip6 { - hash-buckets 2000000 - heap-size 3G - } - - -TG Configuration - TRex ------------------------ - -**TG Version** - -TRex v2.09 - -**DPDK version** - -DPDK v16.07 (20e2b6eba13d9eb61b23ea75f09f2aa966fa6325 - in DPDK repo) - -**TG Build Script used** - -https://gerrit.fd.io/r/gitweb?p=csit.git;a=blob;f=resources/tools/t-rex/t-rex-installer.sh;h=e89b06f9b12499996df18e5e3399fcd660ebc017;hb=refs/heads/rls1701 - -**TG Startup Configuration** - -:: - - $ cat /etc/trex_cfg.yaml - - port_limit : 2 - version : 2 - interfaces : ["0000:0d:00.0","0000:0d:00.1"] - port_info : - - dest_mac : [0x3c,0xfd,0xfe,0x9c,0xee,0xf5] - src_mac : [0x3c,0xfd,0xfe,0x9c,0xee,0xf4] - - dest_mac : [0x3c,0xfd,0xfe,0x9c,0xee,0xf4] - src_mac : [0x3c,0xfd,0xfe,0x9c,0xee,0xf5] - -**TG common API - pointer to driver** - -https://gerrit.fd.io/r/gitweb?p=csit.git;a=blob;f=resources/tools/t-rex/t-rex-stateless.py;h=24f4a997389ba3f10ad42e1f9564ef915fd58b44;hb=refs/heads/rls1701 + +.. raw:: latex + + \clearpage + +.. _vpp_test_environment: + +.. include:: ../introduction/test_environment_intro.rst + +.. include:: ../introduction/test_environment_sut_calib_hsw.rst + +.. include:: ../introduction/test_environment_sut_calib_skx.rst + +.. + .. include:: ../introduction/test_environment_sut_calib_dnv.rst + +.. include:: ../introduction/test_environment_sut_conf_1.rst + +.. include:: ../introduction/test_environment_sut_conf_2.rst + +.. include:: ../introduction/test_environment_sut_conf_3.rst + + +DUT Settings - VPP +------------------ + +VPP Version +~~~~~~~~~~~ + +|vpp-release| + +VPP Compile Parameters +~~~~~~~~~~~~~~~~~~~~~~ + +`FD.io VPP compile job`_ + +VPP Install Parameters +~~~~~~~~~~~~~~~~~~~~~~ + +:: + + $ dpkg -i --force-all vpp* + +VPP Startup Configuration +~~~~~~~~~~~~~~~~~~~~~~~~~ + +VPP startup configuration vary per test case, with different settings +for `$$CORELIST_WORKERS`, `$$NUM_RX_QUEUES`, `$$UIO_DRIVER`, `$$NUM- +MBUFS` and `$$NO_MULTI_SEG` parameter. Default template is provided +below: + +:: + + ip + { + heap-size 4G + } + statseg + { + size 4G + } + unix + { + cli-listen /run/vpp/cli.sock + log /tmp/vpe.log + nodaemon + } + ip6 + { + heap-size 4G + hash-buckets 2000000 + } + heapsize 4G + plugins + { + plugin default + { + disable + } + plugin dpdk_plugin.so + { + enable + } + } + cpu + { + corelist-workers $$CORELIST_WORKERS + main-core 1 + } + dpdk + { + num-mbufs $$NUM-MBUFS + uio-driver $$UIO_DRIVER + $$NO_MULTI_SEG + log-level debug + dev default + { + num-rx-queues $$NUM_RX_QUEUES + } + socket-mem 1024,1024 + no-tx-checksum-offload + dev $$DEV_1 + dev $$DEV_2 + } + +Description of VPP startup settings used in CSIT is provided in +:ref:`test_methodology`. + +.. include:: ../introduction/test_environment_tg.rst