performance labs to address larger scale multi-interface and multi-NIC\r
performance testing scenarios.\r
\r
-For test cases that require DUT (VPP) to communicate with VM over vhost-user\r
-interfaces, a VM is created on SUT1 and SUT2. DUT (VPP) test topology with VM\r
-is shown in the figure below including applicable packet flow thru the VM\r
+For test cases that require DUT (VPP) to communicate with VM(s) over vhost-user\r
+interfaces, N of VM instances are created on SUT1 and SUT2. For N=1 DUT (VPP) forwards packets between vhostuser and physical interfaces. For N>1 DUT (VPP) a logical service chain forwarding topology is created on DUT (VPP) by applying L2 or IPv4/IPv6 configuration depending on the test suite.\r
+DUT (VPP) test topology with N VM instances\r
+is shown in the figure below including applicable packet flow thru the DUTs and VMs\r
(marked in the figure with ``***``).\r
\r
::\r
\r
- +------------------------+ +------------------------+\r
- | +----------+ | | +----------+ |\r
- | | VM | | | | VM | |\r
- | | ****** | | | | ****** | |\r
- | +--^----^--+ | | +--^----^--+ |\r
- | *| |* | | *| |* |\r
- | +------v----v------+ | | +------v----v------+ |\r
- | | * * |**|***********|**| * * | |\r
- | | ***** *******<----------------->******* ***** | |\r
- | | * DUT1 | | | | DUT2 * | |\r
- | +--^---------------+ | | +---------------^--+ |\r
- | *| | | |* |\r
- | *| SUT1 | | SUT2 |* |\r
- +------------------------+ +------------------^-----+\r
- *| |*\r
- *| |*\r
- *| +-----------+ |*\r
- *| | | |*\r
- *+------------------> TG <------------------+*\r
- ******************* | |********************\r
- +-----------+\r
-\r
-For VM tests, packets are switched by DUT (VPP) twice, hence the\r
-throughput rates measured by TG (and listed in this report) must be multiplied\r
-by two to represent the actual DUT aggregate packet forwarding rate.\r
-\r
-Note that reported VPP performance results are specific to the SUT tested.\r
+ +-------------------------+ +-------------------------+\r
+ | +---------+ +---------+ | | +---------+ +---------+ |\r
+ | | VM[1] | | VM[N] | | | | VM[1] | | VM[N] | |\r
+ | | ***** | | ***** | | | | ***** | | ***** | |\r
+ | +--^---^--+ +--^---^--+ | | +--^---^--+ +--^---^--+ |\r
+ | *| |* *| |* | | *| |* *| |* |\r
+ | +--v---v-------v---v--+ | | +--v---v-------v---v--+ |\r
+ | | * * * * |*|***********|*| * * * * | |\r
+ | | * ********* ***<-|-----------|->*** ********* * | |\r
+ | | * DUT1 | | | | DUT2 * | |\r
+ | +--^------------------+ | | +------------------^--+ |\r
+ | *| | | |* |\r
+ | *| SUT1 | | SUT2 |* |\r
+ +-------------------------+ +-------------------------+\r
+ *| |*\r
+ *| |*\r
+ *| +-----------+ |*\r
+ *| | | |*\r
+ *+--------------------> TG <--------------------+*\r
+ **********************| |**********************\r
+ +-----------+\r
+\r
+For VM tests, packets are switched by DUT (VPP) multiple times: twice for a single VM, three times for two VMs, N+1 times for N VMs.\r
+Hence the external\r
+throughput rates measured by TG and listed in this report must be multiplied\r
+by (N+1) to represent the actual DUT aggregate packet forwarding rate.\r
+\r
+CSIT |release|\r
+\r
+Note that reported VPP performance results are specific to the SUTs tested.\r
Current LF FD.io SUTs are based on Intel XEON E5-2699v3 2.3GHz CPUs. SUTs with\r
other CPUs are likely to yield different results. A good rule of thumb, that\r
can be applied to estimate VPP packet thoughput for Phy-to-Phy (NIC-to-NIC,\r
PCI-to-PCI) topology, is to expect the forwarding performance to be\r
proportional to CPU core frequency, assuming CPU is the only limiting factor\r
-and all other SUT aspects equal to FD.io CSIT environment. The same rule of\r
+and all other SUT parameters equivalent to FD.io CSIT environment. The same rule of\r
thumb can be also applied for Phy-to-VM-to-Phy (NIC-to-VM-to-NIC) topology,\r
-but due to much higher dependency on very high frequency memory operations and\r
+but due to much higher dependency on intensive memory operations and\r
sensitivity to Linux kernel scheduler settings and behaviour, this estimation\r
may not always yield good enough accuracy.\r
\r
-Detailed LF FD.io test bed specification and physical topology are described\r
-in `wiki CSIT LF FDio testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.\r
+For detailed LF FD.io test bed specification and physical topology please refer to `LF FDio CSIT testbed wiki page <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.\r
\r
Performance Tests Coverage\r
--------------------------\r
in accordance to RFC2544.\r
\r
- NDR - discovery of Non Drop Rate packet throughput, at zero packet loss;\r
- followed by packet one-way latency measurements at 10%, 50% and 100% of\r
+ followed by one-way packet latency measurements at 10%, 50% and 100% of\r
discovered NDR throughput.\r
- PDR - discovery of Partial Drop Rate, with specified non-zero packet loss\r
- currently set to 0.5%; followed by packet one-way latency measurements at\r
+ currently set to 0.5%; followed by one-way packet latency measurements at\r
100% of discovered PDR throughput.\r
\r
- Throughput verification - verification of packet forwarding rate against\r
Performance Tests Naming\r
------------------------\r
\r
-CSIT |release| introduced a common structured naming convention for all\r
-performance and functional tests. This change was driven by substantially\r
-growing number and type of CSIT test cases. Firstly, the original practice did\r
-not always follow any strict naming convention. Secondly test names did not\r
-always clearly capture tested packet encapsulations, and the actual type or\r
-content of the tests. Thirdly HW configurations in terms of NICs, ports and\r
-their locality were not captured either. These were but few reasons that drove\r
-the decision to change and define a new more complete and stricter test naming\r
-convention, and to apply this to all existing and new test cases.\r
-\r
-The new naming should be intuitive for majority of the tests. The complete\r
+CSIT |release| follows a common structured naming convention for all\r
+performance and system functional tests, introduced in CSIT rls1701.\r
+\r
+The naming should be intuitive for majority of the tests. Complete\r
description of CSIT test naming convention is provided on `CSIT test naming wiki\r
<https://wiki.fd.io/view/CSIT/csit-test-naming>`_.\r
\r
\r
#. 1t1c - 1 VPP worker thread on 1 CPU physical core.\r
#. 2t2c - 2 VPP worker threads on 2 CPU physical cores.\r
-#. 4t4c - 4 VPP threads on 4 CPU physical cores.\r
\r
-Note that in quite a few test cases running VPP on 2 or 4 physical cores hits\r
+Note that in quite a few test cases running VPP on 2 physical cores hits\r
the tested NIC I/O bandwidth or packets-per-second limit.\r
\r
Methodology: Packet Throughput\r
Code Review / csit.git / blobdiff