X-Git-Url: https://gerrit.fd.io/r/gitweb?a=blobdiff_plain;f=docs%2Freport%2Fvpp_performance_tests%2Foverview.rst;h=ebc14789e6a351d8d803c818d5762462ddef94ad;hb=912acb80b6607a6b904242a0ec695c874580bd02;hp=3ee9b2b9556006253cf594599ae46773aaec7802;hpb=fc30d9f7d402c82b69ba3c93d87bf5d7536a13a8;p=csit.git diff --git a/docs/report/vpp_performance_tests/overview.rst b/docs/report/vpp_performance_tests/overview.rst index 3ee9b2b955..ebc14789e6 100644 --- a/docs/report/vpp_performance_tests/overview.rst +++ b/docs/report/vpp_performance_tests/overview.rst @@ -52,20 +52,21 @@ Going forward CSIT project will be looking to add more hardware into FD.io performance labs to address larger scale multi-interface and multi-NIC performance testing scenarios. -For test cases that require DUT (VPP) to communicate with VM(s) over vhost-user -interfaces, N of VM instances are created on SUT1 and SUT2. For N=1 DUT (VPP) -forwards packets between vhostuser and physical interfaces. For N>1 DUT (VPP) a -logical service chain forwarding topology is created on DUT (VPP) by applying L2 -or IPv4/IPv6 configuration depending on the test suite. -DUT (VPP) test topology with N VM instances -is shown in the figure below including applicable packet flow thru the DUTs and -VMs (marked in the figure with ``***``). +For test cases that require DUT (VPP) to communicate with +VirtualMachines(VMs)/LinuxContainers(LXCs) over vhost-user/memif +interfaces, N of VM/LXC instances are created on SUT1 and SUT2. For N=1 +DUT forwards packets between vhost/memif and physical interfaces. For +N>1 DUT a logical service chain forwarding topology is created on DUT by +applying L2 or IPv4/IPv6 configuration depending on the test suite. DUT +test topology with N VM/LXC instances is shown in the figure below +including applicable packet flow thru the DUTs and VMs/LXCs (marked in +the figure with ``***``). :: +-------------------------+ +-------------------------+ | +---------+ +---------+ | | +---------+ +---------+ | - | | VM[1] | | VM[N] | | | | VM[1] | | VM[N] | | + | |VM/LXC[1]| |VM/LXC[N]| | | |VM/LXC[1]| |VM/LXC[N]| | | | ***** | | ***** | | | | ***** | | ***** | | | +--^---^--+ +--^---^--+ | | +--^---^--+ +--^---^--+ | | *| |* *| |* | | *| |* *| |* | @@ -85,26 +86,28 @@ VMs (marked in the figure with ``***``). **********************| |********************** +-----------+ -For VM tests, packets are switched by DUT (VPP) multiple times: twice for a -single VM, three times for two VMs, N+1 times for N VMs. -Hence the external -throughput rates measured by TG and listed in this report must be multiplied -by (N+1) to represent the actual DUT aggregate packet forwarding rate. - -Note that reported VPP performance results are specific to the SUTs tested. -Current LF FD.io SUTs are based on Intel XEON E5-2699v3 2.3GHz CPUs. SUTs with -other CPUs are likely to yield different results. A good rule of thumb, that -can be applied to estimate VPP packet thoughput for Phy-to-Phy (NIC-to-NIC, -PCI-to-PCI) topology, is to expect the forwarding performance to be -proportional to CPU core frequency, assuming CPU is the only limiting factor -and all other SUT parameters equivalent to FD.io CSIT environment. The same rule -of thumb can be also applied for Phy-to-VM-to-Phy (NIC-to-VM-to-NIC) topology, -but due to much higher dependency on intensive memory operations and -sensitivity to Linux kernel scheduler settings and behaviour, this estimation -may not always yield good enough accuracy. - -For detailed LF FD.io test bed specification and physical topology please refer -to `LF FDio CSIT testbed wiki page `_. +For VM/LXC tests, packets are switched by DUT multiple times: twice for +a single VM/LXC, three times for two VMs/LXCs, N+1 times for N VMs/LXCs. +Hence the external throughput rates measured by TG and listed in this +report must be multiplied by (N+1) to represent the actual DUT aggregate +packet forwarding rate. + +Note that reported DUT (VPP) performance results are specific to the +SUTs tested. Current LF FD.io SUTs are based on Intel XEON E5-2699v3 +2.3GHz CPUs. SUTs with other CPUs are likely to yield different results. +A good rule of thumb, that can be applied to estimate VPP packet +thoughput for Phy-to-Phy (NIC-to-NIC, PCI-to-PCI) topology, is to expect +the forwarding performance to be proportional to CPU core frequency, +assuming CPU is the only limiting factor and all other SUT parameters +equivalent to FD.io CSIT environment. The same rule of thumb can be also +applied for Phy-to-VM/LXC-to-Phy (NIC-to-VM/LXC-to-NIC) topology, but +due to much higher dependency on intensive memory operations and +sensitivity to Linux kernel scheduler settings and behaviour, this +estimation may not always yield good enough accuracy. + +For detailed LF FD.io test bed specification and physical topology +please refer to +`LF FD.io CSIT testbed wiki page `_. Performance Tests Coverage -------------------------- @@ -194,7 +197,7 @@ Performance Tests Naming ------------------------ CSIT |release| follows a common structured naming convention for all -performance and system functional tests, introduced in CSIT rls1701. +performance and system functional tests, introduced in CSIT |release-1|. The naming should be intuitive for majority of the tests. Complete description of CSIT test naming convention is provided on `CSIT test naming wiki @@ -205,38 +208,38 @@ suites: #. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P** - - *PortNICConfig-WireEncapsulation-PacketForwardingFunction- - PacketProcessingFunction1-...-PacketProcessingFunctionN-TestType* - - *10ge2p1x520-dot1q-l2bdbasemaclrn-ndrdisc.robot* => 2 ports of 10GE on - Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain baseline switching - with MAC learning, NDR throughput discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrchk.robot* => 2 ports of 10GE - on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain baseline - switching with MAC learning, NDR throughput discovery. - - *10ge2p1x520-ethip4-ip4base-ndrdisc.robot* => 2 ports of 10GE on Intel - x520 NIC, IPv4 baseline routed forwarding, NDR throughput discovery. - - *10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot* => 2 ports of 10GE on - Intel x520 NIC, IPv6 scaled up routed forwarding, NDR throughput - discovery. + - *PortNICConfig-WireEncapsulation-PacketForwardingFunction- + PacketProcessingFunction1-...-PacketProcessingFunctionN-TestType* + - *10ge2p1x520-dot1q-l2bdbasemaclrn-ndrdisc.robot* => 2 ports of 10GE on + Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain baseline switching + with MAC learning, NDR throughput discovery. + - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrchk.robot* => 2 ports of 10GE + on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain baseline + switching with MAC learning, NDR throughput discovery. + - *10ge2p1x520-ethip4-ip4base-ndrdisc.robot* => 2 ports of 10GE on Intel + x520 NIC, IPv4 baseline routed forwarding, NDR throughput discovery. + - *10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot* => 2 ports of 10GE on + Intel x520 NIC, IPv6 scaled up routed forwarding, NDR throughput + discovery. #. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC, P2V2P, NIC2VMchain2NIC, P2V2V2P** - - *PortNICConfig-WireEncapsulation-PacketForwardingFunction- - PacketProcessingFunction1-...-PacketProcessingFunctionN-VirtEncapsulation- - VirtPortConfig-VMconfig-TestType* - - *10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 ports - of 10GE on Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain - switching to/from two vhost interfaces and one VM, NDR throughput - discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 - ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain - switching to/from two vhost interfaces and one VM, NDR throughput - discovery. - - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc.robot* => 2 - ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain - switching to/from four vhost interfaces and two VMs, NDR throughput - discovery. + - *PortNICConfig-WireEncapsulation-PacketForwardingFunction- + PacketProcessingFunction1-...-PacketProcessingFunctionN-VirtEncapsulation- + VirtPortConfig-VMconfig-TestType* + - *10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 ports + of 10GE on Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain + switching to/from two vhost interfaces and one VM, NDR throughput + discovery. + - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 + ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain + switching to/from two vhost interfaces and one VM, NDR throughput + discovery. + - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc.robot* => 2 + ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain + switching to/from four vhost interfaces and two VMs, NDR throughput + discovery. Methodology: Multi-Thread and Multi-Core ---------------------------------------- @@ -313,8 +316,8 @@ Methodology: KVM VM vhost ------------------------- CSIT |release| introduced environment configuration changes to KVM Qemu vhost- -user tests in order to more representatively measure VPP-17.04 performance in -configurations with vhost-user interfaces and VMs. +user tests in order to more representatively measure |vpp-release| performance +in configurations with vhost-user interfaces and VMs. Current setup of CSIT FD.io performance lab is using tuned settings for more optimal performance of KVM Qemu: @@ -356,29 +359,29 @@ specific configuration. TRex is installed and run on the TG compute node. The typical procedure is: - - If the TRex is not already installed on TG, it is installed in the - suite setup phase - see `TRex intallation `_. - - TRex configuration is set in its configuration file - :: +- If the TRex is not already installed on TG, it is installed in the + suite setup phase - see `TRex intallation`_. +- TRex configuration is set in its configuration file + :: - /etc/trex_cfg.yaml + /etc/trex_cfg.yaml - - TRex is started in the background mode - :: +- TRex is started in the background mode + :: - sh -c 'cd /opt/trex-core-2.22/scripts/ && sudo nohup ./t-rex-64 -i -c 7 --iom 0 > /dev/null 2>&1 &' > /dev/null + $ sh -c 'cd /opt/trex-core-2.25/scripts/ && sudo nohup ./t-rex-64 -i -c 7 --iom 0 > /dev/null 2>&1 &' > /dev/null - - There are traffic streams dynamically prepared for each test. The traffic - is sent and the statistics obtained using trex_stl_lib.api.STLClient. +- There are traffic streams dynamically prepared for each test. The traffic + is sent and the statistics obtained using trex_stl_lib.api.STLClient. **Measuring packet loss** - - Create an instance of STLClient - - Connect to the client - - Add all streams - - Clear statistics - - Send the traffic for defined time - - Get the statistics +- Create an instance of STLClient +- Connect to the client +- Add all streams +- Clear statistics +- Send the traffic for defined time +- Get the statistics If there is a warm-up phase required, the traffic is sent also before test and the statistics are ignored.