Background\r
----------\r
\r
-CSIT |release| introduced a common structured naming convention for all\r
-performance and functional tests. This change was driven by substantially\r
-growing number and type of CSIT test cases. Firstly, the original practice did\r
-not always follow any strict naming convention. Secondly test names did not\r
-always clearly capture tested packet encapsulations, and the actual type or\r
-content of the tests. Thirdly HW configurations in terms of NICs, ports and\r
-their locality were not captured either. These were but few reasons that drove\r
-the decision to change and define a new more complete and stricter test naming\r
-convention. And to apply it to all existing and new test suites and test\r
-cases.\r
-\r
-The new naming should be intuitive for majority of the tests. Complete\r
-description of the newly adopted CSIT test naming convention is provided on\r
-`CSIT test naming page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_. Here\r
-few illustrative examples of the new naming usage for test suites across CSIT\r
+CSIT |release| follows a common structured naming convention for all\r
+performance and system functional tests, introduced in CSIT rls1701.\r
+\r
+The naming should be intuitive for majority of the tests. Complete\r
+description of CSIT test naming convention is provided on\r
+`CSIT test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_. Below\r
+few illustrative examples of the naming usage for test suites across CSIT\r
performance, functional and HoneyComb management test areas.\r
\r
Naming Convention\r
sub-interface events with HoneyComb NetConf Notifications, functional tests.\r
\r
For complete description of CSIT test naming convention please refer to `CSIT\r
-test naming page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.\r
+test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.\r
\r
All CSIT test results listed in this report are sourced and auto-generated\r
from output.xml Robot Framework (RF) files resulting from LF FD.io Jenkins\r
-jobs execution against VPP-17.01 release artifacts. References are provided\r
+jobs execution against VPP-17.04 release artifacts. References are provided\r
to the original LF FD.io Jenkins job results. However, as LF FD.io Jenkins\r
infrastructure does not automatically archive all jobs (history record is\r
provided for the last 30 days or 40 jobs only), additional references are\r
\r
FD.io CSIT project currently covers multiple FD.io system and sub-system\r
testing areas and this is reflected in this report, where each testing area\r
-is listed separately, as follows\r
+is listed separately, as follows:\r
\r
-#. **VPP Performance Tests** - VPP performance are tests executed in physical\r
- FD.io testbeds, focusing on VPP network data plane performance at this stage,\r
+#. **VPP Performance Tests** - VPP performance tests are executed in physical\r
+ FD.io testbeds, focusing on VPP network data plane performance at this stage,\r
both for Phy-to-Phy (NIC-to-NIC) and Phy-to-VM-to-Phy (NIC-to-VM-to-NIC)\r
forwarding topologies. Tested across a range of NICs, 10GE and 40GE\r
interfaces, range of multi-thread and multi-core configurations. VPP\r
functionality of VPP. Tests cover a range of CRUD operations executed\r
against VPP.\r
\r
-CSIT |release| report does also include VPP unit test results. These tests\r
-have been developed within the FD.io VPP project, and not in CSIT, but they do\r
-complement tests done by CSIT. They are provided mainly as a reference, to\r
-give the reader a more complete view of automated testing executed against\r
-VPP-17.01 release.\r
+In addition to above, CSIT |release| report does also include VPP unit test results. VPP unit tests are developed within the FD.io VPP project and as they complement CSIT system functional tests, they are provided mainly as a reference and to\r
+provide a more complete view of automated testing executed against\r
+VPP-17.04 release.\r
\r
FD.io CSIT system is developed using two main coding platforms: Robot\r
Framework (RF) and Python. CSIT |release| source code for the executed test\r
\r
This is the **F**\ast **D**\ata **I**/**O** Project (**FD.io**) **C**\ontinuous\r
**S**\ystem **I**\ntegration and **T**\esting (**CSIT**) project report for CSIT\r
-|release| system testing of VPP-17.01 release.\r
+|release| system testing of VPP-17.04 release.\r
\r
The report describes CSIT functional and performance tests and their\r
continuous execution delivered in CSIT |release|. A high-level overview is\r
provided for each CSIT test environment running in Linux Foundation (LF) FD.io\r
Continuous Performance Labs. This is followed by summary of all executed tests\r
-against the VPP-17.01 release and associated sub-systems (HoneyComb, DPDK),\r
-CSIT |release| release notes, result highlights and known issues. More\r
+against the VPP-17.04 release and associated FD.io projects and sub-systems (HoneyComb, DPDK),\r
+CSIT |release| release notes, result highlights and known issues discovered in CSIT. More\r
detailed description of each environment, pointers to CSIT test code\r
documentation and detailed test resuls with links to the source data files are\r
also provided.\r
specifics, methodology for multi-core, packet throughput and latency, and\r
KVM VM vhost tests; *CSIT Release Notes* - changes in CSIT |release|, added\r
tests, performance changes, environment or methodology changes, known CSIT\r
- issues, tests to be added; *Packet Throughput Graphs* and *Packet Latency\r
+ issues; *Packet Throughput Graphs* and *Packet Latency\r
Graphs* - plotted NDR, PDR throughput and latency results from multiple\r
- test job executions; *Test Environment* - environment description ;\r
- *Documentation* - source code documentation for VPP performance tests.\r
+ test job executions; *Test Environment* - environment description;\r
+ *Documentation* - CSIT source code documentation for VPP performance tests.\r
\r
#. **Testpmd Performance Tests** - Testpmd performance tests executed in\r
physical FD.io testbeds; *Overview* - tested topologies, test coverage;\r
*CSIT Release Notes* - changes in CSIT |release|, any known CSIT issues;\r
- *Tests to Be Added* - performance tests to be added in the next revision of\r
- CSIT |release| report; *Packet Throughput Graphs* and *Packet Latency Graphs*\r
+ *Packet Throughput Graphs* and *Packet Latency Graphs*\r
- plotted NDR, PDR throughput and latency results from multiple test job\r
executions; *Test Environment* - environment description; *Documentation* -\r
- source code documentation for Testpmd performance tests.\r
+ CSIT source code documentation for Testpmd performance tests.\r
\r
#. **VPP Functional Tests** - VPP functional tests executed in virtual\r
FD.io testbeds; *Overview* - tested virtual topologies, test coverage and\r
#. **HoneyComb Functional Tests** - HoneyComb functional tests executed in\r
virtual FD.io testbeds; *Overview* - tested virtual topologies, test\r
coverage and naming specifics; *CSIT Release Notes* - changes in CSIT\r
- |release|, added tests, environment or methodology changes, known CSIT issues,\r
- tests to be added; *Test Environment* - environment description ;\r
+ |release|, added tests, environment or methodology changes, known CSIT issues;\r
+ *Test Environment* - environment description ;\r
*Documentation* - source code documentation for Honeycomb functional tests.\r
\r
#. **VPP Unit Tests** - refers to VPP functional unit tests executed as\r
#. Added Testpmd tests
- - new NICs - Intel x520
+ - new NICs - Intel xl710
-Tests to be Added - Additional NICs
------------------------------------
-
-Adding test cases with more NIC types. Once the results become available, they
-will be published as an addendum to the current version of CSIT |release|
-report.
=============\r
\r
`CSIT Testpmd Performance Tests Documentation\r
-<https://docs.fd.io/csit/master/doc/tests.perf.html>`_ contains detailed\r
+<https://docs.fd.io/csit/rls1704/doc/tests.perf.html>`_ contains detailed\r
functional description and input parameters for each test case.\r
#. 2port40GE VIC1385 Cisco.
#. 2port40GE XL710 Intel.
-Detailed LF FD.io test bed specification and topology is described in
-`wiki CSIT LF testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
+For detailed LF FD.io test bed specification and physical topology please refer to `LF FDio CSIT testbed wiki page <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
Performance Tests Coverage
--------------------------
- **L2IntLoop** - L2 Interface Loop forwarding any Ethernet frames between
two Interfaces.
+- 2port40GE XL710 Intel
+
+ - **L2IntLoop** - L2 Interface Loop forwarding any Ethernet frames between
+ two Interfaces.
+
Execution of performance tests takes time, especially the throughput discovery
tests. Due to limited HW testbed resources available within FD.io labs hosted
by Linux Foundation, the number of tests for NICs other than X520 (a.k.a.
#. 1t1c - 1 Testpmd pmd thread on 1 CPU physical core.
#. 2t2c - 2 Testpmd pmd threads on 2 CPU physical cores.
-#. 4t4c - 4 Testpmd pmd threads on 4 CPU physical cores.
Note that in many tests running Testpmd reaches tested NIC I/O bandwidth
or packets-per-second limit.
(s), and their physical CPU core(s) placement.\r
\r
*Title of each graph* is a regex (regular expression) matching all plotted\r
-latency test cases, *X-axis labels* are indeces of csit-dpdk-perf-1701 jobs\r
+latency test cases, *X-axis labels* are indeces of csit-dpdk-perf-1704 jobs\r
that created result output files used as data sources for the graph,\r
*Y-axis labels* are measured packet Latency [uSec] values, and the *graph\r
legend* identifes the plotted test suites.\r
.. note::\r
\r
Data sources for reported test results: i) FD.io test executor jobs\r
- `csit-dpdk-perf-1701-all <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1701-all/>`_\r
- and\r
- `csit-dpdk-perf-1701-long <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1701-long/>`_\r
- , ii) archived FD.io jobs test result output files\r
- `csit-dpdk-perf-1701-all#job-number <https://nexus.fd.io/link>`_.\r
+ `csit-dpdk-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_ ,\r
+ ii) archived FD.io jobs test result output files\r
+ `csit-dpdk-perf-1704-all#job-number <https://nexus.fd.io/link>`_.\r
\r
Testpmd packet latency - running in configuration of **one worker thread (1t) on one\r
physical core (1c)** - is presented in the figure below.\r
\r
10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc\r
\r
-Testpmd packet latency - running in configuration of **four worker threads (4t)\r
-on four physical cores (4c)** - is presented in the figure below.\r
-\r
-.. raw:: html\r
-\r
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/testpmd/64B-4t4c-l2-ndrdisc-lat50.html"></iframe>\r
-\r
-*Figure 3. Testpmd 4thread 4core - packet latency for Phy-to-Phy L2 Ethernet Looping.*\r
-\r
-CSIT test cases used to generate results presented above can be found in CSIT\r
-git repository by filtering with specified regex as follows:\r
-\r
-.. code-block:: bash\r
-\r
- $ csit/tests/perf\r
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *\r
-\r
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc\r
-\r
configurations.\r
\r
*Title of each graph* is a regex (regular expression) matching all plotted\r
-throughput test cases, *X-axis labels* are indeces of csit-dpdk-perf-1701 jobs\r
+throughput test cases, *X-axis labels* are indeces of csit-dpdk-perf-1704 jobs\r
that created result output files used as data sources for the graph,\r
*Y-axis labels* are measured Packets Per Second [pps] values, and the *graph\r
legend* identifes the plotted test suites.\r
\r
.. note::\r
\r
- Sources of test results data: i) FD.io executor jobs `csit-dpdk-perf-1701-all\r
- <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1701-all/>`_ and\r
- `csit-dpdk-perf-1701-long\r
- <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1701-long/>`_, ii)\r
- FD.io jobs archieved result files.\r
+ Sources of test results data: i) FD.io executor jobs `csit-dpdk-perf-1704-all\r
+ <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_ ,\r
+ ii) FD.io jobs archieved result files.\r
\r
NDR Throughput\r
~~~~~~~~~~~~~~\r
\r
10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc\r
\r
-NDR throughput of Testpmd - running in configuration of **four worker threads\r
-(4t) on four physical cores (4c)** - is presented in the figure below.\r
-\r
-.. raw:: html\r
-\r
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/testpmd/64B-4t4c-l2-ndrdisc.html"></iframe>\r
-\r
-*Figure 3. Testpmd 4threads 4cores - NDR Throughput for Phy-to-Phy L2 Ethernet\r
-Looping.*\r
-\r
-CSIT test cases used to generate results presented above can be found in CSIT\r
-git repository by filtering with specified regex as follows:\r
-\r
-.. code-block:: bash\r
-\r
- $ csit/dpdk-tests/perf\r
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *\r
-\r
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc\r
-\r
PDR Throughput\r
~~~~~~~~~~~~~~\r
\r
\r
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/testpmd/64B-1t1c-l2-pdrdisc.html"></iframe>\r
\r
-*Figure 4. Testpmd 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet\r
+*Figure 3. Testpmd 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet\r
Looping.*\r
\r
CSIT test cases used to generate results presented above can be found in CSIT\r
\r
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/testpmd/64B-2t2c-l2-pdrdisc.html"></iframe>\r
\r
-*Figure 5. Testpmd 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet\r
+*Figure 4. Testpmd 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet\r
Looping.*\r
\r
CSIT test cases used to generate results presented above can be found in CSIT\r
\r
10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-pdrdisc\r
\r
-PDR throughput of Testpmd - running in configuration of **four worker threads\r
-(4t) on four physical cores (4c)** - is presented in the figure below.\r
-\r
-.. raw:: html\r
-\r
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/testpmd/64B-4t4c-l2-pdrdisc.html"></iframe>\r
-\r
-*Figure 6. Testpmd 4thread 4core - PDR Throughput for Phy-to-Phy L2 Ethernet\r
-Looping.*\r
-\r
-CSIT test cases used to generate results presented above can be found in CSIT\r
-git repository by filtering with specified regex as follows:\r
-\r
-.. code-block:: bash\r
-\r
- $ csit/tests/perf\r
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" *\r
-\r
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-pdrdisc\r
\r
SUT Configuration - Host HW\r
---------------------------\r
-Hardware details (CPU, memory, NIC layout) are described in\r
-`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_ in\r
-summary:\r
+Host hardware details (CPU, memory, NIC layout) and physical topology are\r
+described in detail in\r
+`LF FDio CSIT testbed wiki page <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.\r
+\r
+**Host configuration**\r
\r
- All hosts are Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz,\r
18c, 512GB RAM),\r
- This allows for a total of five ring topologies, each using ports on\r
specific NIC model, enabling per NIC model benchmarking.\r
\r
-**NIC types**\r
+**NIC models and placement**\r
\r
- 0a:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+\r
Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server\r
SUT Configuration - Host OS Linux\r
---------------------------------\r
\r
-Software details (OS, configuration) is described in\r
-`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_ in\r
-summary:\r
+Software details (OS, configuration) are described in\r
+`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.\r
+\r
+Below a subset of the configuration:\r
\r
.. code-block:: xml\r
\r
<exec_command><![CDATA[lsblk -l]]></exec_command>\r
<exec_return_code>0</exec_return_code>\r
<exec_output><![CDATA[NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT\r
- sda 8:0 0 1.8T 0 disk \r
+ sda 8:0 0 1.8T 0 disk\r
sda1 8:1 0 243M 0 part /boot\r
sda2 8:2 0 1.8T 0 part /\r
- sda3 8:3 0 1K 0 part \r
+ sda3 8:3 0 1K 0 part\r
sda5 8:5 0 976M 0 part [SWAP]\r
]]></exec_output>\r
</function>\r
dev.cdrom.check_media = 0\r
dev.cdrom.debug = 0\r
dev.cdrom.info = CD-ROM information, Id: cdrom.c 3.20 2003/12/17\r
- dev.cdrom.info = \r
- dev.cdrom.info = drive name: \r
- dev.cdrom.info = drive speed: \r
+ dev.cdrom.info =\r
+ dev.cdrom.info = drive name:\r
+ dev.cdrom.info = drive speed:\r
dev.cdrom.info = drive # of slots:\r
- dev.cdrom.info = Can close tray: \r
- dev.cdrom.info = Can open tray: \r
- dev.cdrom.info = Can lock tray: \r
+ dev.cdrom.info = Can close tray:\r
+ dev.cdrom.info = Can open tray:\r
+ dev.cdrom.info = Can lock tray:\r
dev.cdrom.info = Can change speed:\r
dev.cdrom.info = Can select disk:\r
dev.cdrom.info = Can read multisession:\r
- dev.cdrom.info = Can read MCN: \r
+ dev.cdrom.info = Can read MCN:\r
dev.cdrom.info = Reports media changed:\r
- dev.cdrom.info = Can play audio: \r
- dev.cdrom.info = Can write CD-R: \r
+ dev.cdrom.info = Can play audio:\r
+ dev.cdrom.info = Can write CD-R:\r
dev.cdrom.info = Can write CD-RW:\r
- dev.cdrom.info = Can read DVD: \r
+ dev.cdrom.info = Can read DVD:\r
dev.cdrom.info = Can write DVD-R:\r
dev.cdrom.info = Can write DVD-RAM:\r
- dev.cdrom.info = Can read MRW: \r
- dev.cdrom.info = Can write MRW: \r
- dev.cdrom.info = Can write RAM: \r
- dev.cdrom.info = \r
- dev.cdrom.info = \r
+ dev.cdrom.info = Can read MRW:\r
+ dev.cdrom.info = Can write MRW:\r
+ dev.cdrom.info = Can write RAM:\r
+ dev.cdrom.info =\r
+ dev.cdrom.info =\r
dev.cdrom.lock = 0\r
dev.hpet.max-user-freq = 64\r
dev.mac_hid.mouse_button2_keycode = 97\r
kernel.hardlockup_all_cpu_backtrace = 0\r
kernel.hardlockup_panic = 0\r
kernel.hostname = t1-sut1\r
- kernel.hotplug = \r
+ kernel.hotplug =\r
kernel.hung_task_check_count = 4194304\r
kernel.hung_task_panic = 0\r
kernel.hung_task_timeout_secs = 120\r
net.ipv4.ip_forward = 0\r
net.ipv4.ip_forward_use_pmtu = 0\r
net.ipv4.ip_local_port_range = 32768 60999\r
- net.ipv4.ip_local_reserved_ports = \r
+ net.ipv4.ip_local_reserved_ports =\r
net.ipv4.ip_no_pmtu_disc = 0\r
net.ipv4.ip_nonlocal_bind = 0\r
net.ipv4.ipfrag_high_thresh = 4194304\r
\r
**DPDK Version**\r
\r
-16.09\r
+17.02\r
\r
**DPDK Compile Parameters**\r
\r
-
-.. |br| raw:: html
-
- <br />
-
CSIT Release Notes
==================
Changes in CSIT |release|
-------------------------
-#. Naming change for all VPP functional test suites in VIRL
-
- - VPP functional test case names stayed unchanged
-
-#. VPP functional test environment changes
-
- - upgrade to Ubuntu 16.04
- - VM and vhost-user test environment optimizations
-
-#. Introduction of Centos tests
+#. Introduction of Centos7 tests
#. Added VPP functional tests
| | | The (Rx) traffic is mirrored to correct destination port. |
+---+-------------------------------------------------+-----------------------------------------------------------------+
-Tests to be Added - Centos
---------------------------
-
-CSIT |release| added Centos functional test execution environment in FD.io
-VIRL testbeds.However due to the factors beyond CSIT project control execution
-of those tests within the LF FD.io test environment still need to complete.
-Once the results become available, they will be published as an addendum to
-the current version of CSIT |release| report.
=============\r
\r
`CSIT VPP Functional Tests Documentation\r
-<https://docs.fd.io/csit/master/doc/tests.func.html>`_ contains detailed\r
+<https://docs.fd.io/csit/rls1704/doc/tests.func.html>`_ contains detailed\r
functional description and input parameters for each test case.\r
\r
+---------------> <--------------+\r
| | | | | |\r
| |------------> DUT1 <-----------+ |\r
- | | | +--^--^------------+ | | |\r
+ | | | +------------------+ | | |\r
| | | | | |\r
| | | SUT1 | | |\r
| | +------------------------+ | |\r
Functional Tests Naming\r
-----------------------\r
\r
-CSIT |release| introduced a common structured naming convention for all\r
-performance and functional tests. This change was driven by substantially\r
-growing number and type of CSIT test cases. Firstly, the original practice did\r
-not always follow any strict naming convention. Secondly test names did not\r
-always clearly capture tested packet encapsulations, and the actual type or\r
-content of the tests. Thirdly HW configurations in terms of NICs, ports and\r
-their locality were not captured either. These were but few reasons that drove\r
-the decision to change and define a new more complete and stricter test naming\r
-convention, and to apply this to all existing and new test cases.\r
-\r
-The new naming should be intuitive for majority of the tests. The complete\r
+CSIT |release| follows a common structured naming convention for all\r
+performance and system functional tests, introduced in CSIT rls1701.\r
+\r
+The naming should be intuitive for majority of the tests. Complete\r
description of CSIT test naming convention is provided on `CSIT test naming\r
page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.\r
\r
Changes in CSIT |release|\r
-------------------------\r
\r
-#. Naming change for all VPP performance test suites and test cases.\r
#. VPP performance test environment changes\r
\r
- - upgrade to Ubuntu 16.04\r
- - VM and vhost-user test environment optimizations\r
- - addition of HW cryptodev devices in LFD FD.io physical testbed\r
+ - further VM and vhost-user test environment optimizations - Qemu virtio queue size increased from default value of 256 to 1024\r
+ - addition of HW cryptodev devices in all three LF FD.io physical testbeds\r
\r
#. Added tests\r
\r
Substantial improvements in measured packet throughput, with relative increase\r
of double-digit percentage points, have been observed in a number of CSIT\r
|release| tests listed below. Relative improvements are calculated against the\r
-test results in CSIT rls1609 report.\r
+test results in CSIT rls1701 report.\r
\r
NDR Throughput\r
~~~~~~~~~~~~~~\r
Other changes in measured packet throughput, with either minor relative\r
increase or decrease, have been observed in a number of CSIT |release| tests\r
listed below. Relative changes are calculated against the test results in CSIT\r
-rls1609 report.\r
+rls1701 report.\r
\r
NDR Throughput\r
~~~~~~~~~~~~~~\r
| | - 2t2c, 4t4c - for some tested NICs. | |\r
+---+-------------------------------------------------+-----------------------------------------------------------------+\r
\r
-Tests to be Added - Cryptodev\r
------------------------------\r
-\r
-DPDK Cryptodev functionality support for both SW and HW crypto devices has\r
-been introduced in VPP-17.01 release. CSIT functional and performance tests\r
-have been also developed and merged. However due to the factors beyond CSIT\r
-project control execution of those tests within the LF FD.io test environment\r
-still need to complete. Once the results become available, they will be\r
-published as an addendum to the current version of CSIT |release| report.\r
-\r
-Tests to be Added - SNAT44\r
---------------------------\r
-\r
-VPP SNAT44 functionality has been introduced in VPP-17.01 release. CSIT\r
-performance tests are still in development and integration into LFD FD.io test\r
-environment. Once the tests are fully integrated and results become available,\r
-they will be published as an addendum to the current version of CSIT |release|\r
-report.\r
=============\r
\r
`CSIT VPP Performance Tests Documentation\r
-<https://docs.fd.io/csit/master/doc/tests.perf.html>`_ contains detailed\r
+<https://docs.fd.io/csit/rls1704/doc/tests.perf.html>`_ contains detailed\r
functional description and input parameters for each test case.\r
performance labs to address larger scale multi-interface and multi-NIC\r
performance testing scenarios.\r
\r
-For test cases that require DUT (VPP) to communicate with VM over vhost-user\r
-interfaces, a VM is created on SUT1 and SUT2. DUT (VPP) test topology with VM\r
-is shown in the figure below including applicable packet flow thru the VM\r
+For test cases that require DUT (VPP) to communicate with VM(s) over vhost-user\r
+interfaces, N of VM instances are created on SUT1 and SUT2. For N=1 DUT (VPP) forwards packets between vhostuser and physical interfaces. For N>1 DUT (VPP) a logical service chain forwarding topology is created on DUT (VPP) by applying L2 or IPv4/IPv6 configuration depending on the test suite.\r
+DUT (VPP) test topology with N VM instances\r
+is shown in the figure below including applicable packet flow thru the DUTs and VMs\r
(marked in the figure with ``***``).\r
\r
::\r
\r
- +------------------------+ +------------------------+\r
- | +----------+ | | +----------+ |\r
- | | VM | | | | VM | |\r
- | | ****** | | | | ****** | |\r
- | +--^----^--+ | | +--^----^--+ |\r
- | *| |* | | *| |* |\r
- | +------v----v------+ | | +------v----v------+ |\r
- | | * * |**|***********|**| * * | |\r
- | | ***** *******<----------------->******* ***** | |\r
- | | * DUT1 | | | | DUT2 * | |\r
- | +--^---------------+ | | +---------------^--+ |\r
- | *| | | |* |\r
- | *| SUT1 | | SUT2 |* |\r
- +------------------------+ +------------------^-----+\r
- *| |*\r
- *| |*\r
- *| +-----------+ |*\r
- *| | | |*\r
- *+------------------> TG <------------------+*\r
- ******************* | |********************\r
- +-----------+\r
-\r
-For VM tests, packets are switched by DUT (VPP) twice, hence the\r
-throughput rates measured by TG (and listed in this report) must be multiplied\r
-by two to represent the actual DUT aggregate packet forwarding rate.\r
-\r
-Note that reported VPP performance results are specific to the SUT tested.\r
+ +-------------------------+ +-------------------------+\r
+ | +---------+ +---------+ | | +---------+ +---------+ |\r
+ | | VM[1] | | VM[N] | | | | VM[1] | | VM[N] | |\r
+ | | ***** | | ***** | | | | ***** | | ***** | |\r
+ | +--^---^--+ +--^---^--+ | | +--^---^--+ +--^---^--+ |\r
+ | *| |* *| |* | | *| |* *| |* |\r
+ | +--v---v-------v---v--+ | | +--v---v-------v---v--+ |\r
+ | | * * * * |*|***********|*| * * * * | |\r
+ | | * ********* ***<-|-----------|->*** ********* * | |\r
+ | | * DUT1 | | | | DUT2 * | |\r
+ | +--^------------------+ | | +------------------^--+ |\r
+ | *| | | |* |\r
+ | *| SUT1 | | SUT2 |* |\r
+ +-------------------------+ +-------------------------+\r
+ *| |*\r
+ *| |*\r
+ *| +-----------+ |*\r
+ *| | | |*\r
+ *+--------------------> TG <--------------------+*\r
+ **********************| |**********************\r
+ +-----------+\r
+\r
+For VM tests, packets are switched by DUT (VPP) multiple times: twice for a single VM, three times for two VMs, N+1 times for N VMs.\r
+Hence the external\r
+throughput rates measured by TG and listed in this report must be multiplied\r
+by (N+1) to represent the actual DUT aggregate packet forwarding rate.\r
+\r
+CSIT |release|\r
+\r
+Note that reported VPP performance results are specific to the SUTs tested.\r
Current LF FD.io SUTs are based on Intel XEON E5-2699v3 2.3GHz CPUs. SUTs with\r
other CPUs are likely to yield different results. A good rule of thumb, that\r
can be applied to estimate VPP packet thoughput for Phy-to-Phy (NIC-to-NIC,\r
PCI-to-PCI) topology, is to expect the forwarding performance to be\r
proportional to CPU core frequency, assuming CPU is the only limiting factor\r
-and all other SUT aspects equal to FD.io CSIT environment. The same rule of\r
+and all other SUT parameters equivalent to FD.io CSIT environment. The same rule of\r
thumb can be also applied for Phy-to-VM-to-Phy (NIC-to-VM-to-NIC) topology,\r
-but due to much higher dependency on very high frequency memory operations and\r
+but due to much higher dependency on intensive memory operations and\r
sensitivity to Linux kernel scheduler settings and behaviour, this estimation\r
may not always yield good enough accuracy.\r
\r
-Detailed LF FD.io test bed specification and physical topology are described\r
-in `wiki CSIT LF FDio testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.\r
+For detailed LF FD.io test bed specification and physical topology please refer to `LF FDio CSIT testbed wiki page <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.\r
\r
Performance Tests Coverage\r
--------------------------\r
in accordance to RFC2544.\r
\r
- NDR - discovery of Non Drop Rate packet throughput, at zero packet loss;\r
- followed by packet one-way latency measurements at 10%, 50% and 100% of\r
+ followed by one-way packet latency measurements at 10%, 50% and 100% of\r
discovered NDR throughput.\r
- PDR - discovery of Partial Drop Rate, with specified non-zero packet loss\r
- currently set to 0.5%; followed by packet one-way latency measurements at\r
+ currently set to 0.5%; followed by one-way packet latency measurements at\r
100% of discovered PDR throughput.\r
\r
- Throughput verification - verification of packet forwarding rate against\r
Performance Tests Naming\r
------------------------\r
\r
-CSIT |release| introduced a common structured naming convention for all\r
-performance and functional tests. This change was driven by substantially\r
-growing number and type of CSIT test cases. Firstly, the original practice did\r
-not always follow any strict naming convention. Secondly test names did not\r
-always clearly capture tested packet encapsulations, and the actual type or\r
-content of the tests. Thirdly HW configurations in terms of NICs, ports and\r
-their locality were not captured either. These were but few reasons that drove\r
-the decision to change and define a new more complete and stricter test naming\r
-convention, and to apply this to all existing and new test cases.\r
-\r
-The new naming should be intuitive for majority of the tests. The complete\r
+CSIT |release| follows a common structured naming convention for all\r
+performance and system functional tests, introduced in CSIT rls1701.\r
+\r
+The naming should be intuitive for majority of the tests. Complete\r
description of CSIT test naming convention is provided on `CSIT test naming wiki\r
<https://wiki.fd.io/view/CSIT/csit-test-naming>`_.\r
\r
\r
#. 1t1c - 1 VPP worker thread on 1 CPU physical core.\r
#. 2t2c - 2 VPP worker threads on 2 CPU physical cores.\r
-#. 4t4c - 4 VPP threads on 4 CPU physical cores.\r
\r
-Note that in quite a few test cases running VPP on 2 or 4 physical cores hits\r
+Note that in quite a few test cases running VPP on 2 physical cores hits\r
the tested NIC I/O bandwidth or packets-per-second limit.\r
\r
Methodology: Packet Throughput\r
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured packet Latency [uSec] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
VPP packet latency - running in configuration of **one worker thread (1t) on one
10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale2m-ndrdisc
40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale200k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale20k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale2m-ndrdisc
- 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc
-
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured packet Latency [uSec] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
VPP packet latency - running in configuration of **one worker thread (1t) on one
10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-ndrdisc
10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2xcbase-ndrdisc
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv4 Overlay Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip6-ip4base-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2xcbase-ndrdisc
-
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured packet Latency [uSec] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
VPP packet latency - running in configuration of **one worker thread (1t) on one
10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale2m-ndrdisc
40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv6 Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale200k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale20k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale2m-ndrdisc
- 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc
-
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured packet Latency [uSec] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
VPP packet latency - running in configuration of **one worker thread (1t) on one
10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip4-ip6base-ndrdisc
10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip6-ip6base-ndrdisc
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv6 Overlay Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip4-ip6base-ndrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip6-ip6base-ndrdisc
-
-
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured packet Latency [uSec] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
VPP packet latency - running in configuration of **one worker thread (1t) on one
40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy L2 Ethernet Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1ad-l2xcbase-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2xcbase-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc
-
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured packet Latency [uSec] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
VPP packet latency - running in configuration of **one worker thread (1t) on one
10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-VPP packet latency - running in configuration of **four worker threads (4t) on four
-physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-VM-to-Phy VM vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-.*vhost.*-ndrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured Packets Per Second [pps] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
NDR Throughput
10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale2m-ndrdisc
40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc
-VPP NDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-ndrdisc.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv4
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale200k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale20k-ndrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale2m-ndrdisc
- 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc
-
PDR Throughput
~~~~~~~~~~~~~~
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ip4-pdrdisc.html"></iframe>
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4
+*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4
Routed-Forwarding.*
CSIT test cases used to generate results presented above can be found in CSIT
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ip4-pdrdisc.html"></iframe>
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4
+*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4
Routed-Forwarding.*
CSIT test cases used to generate results presented above can be found in CSIT
10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale20k-pdrdisc
10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale2m-pdrdisc
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv4
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" *
-
- 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-copwhtlistbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-iacldstbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-ipolicemarkbase-pdrdisc
- 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-pdrdisc
- 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale200k-pdrdisc
- 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale20k-pdrdisc
- 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale2m-pdrdisc
-
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured Packets Per Second [pps] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
NDR Throughput
10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-ndrdisc
10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2xcbase-ndrdisc
-VPP NDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ndrdisc.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv4 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip4-ip4base-ndrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip6-ip4base-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2xcbase-ndrdisc
-
PDR Throughput
~~~~~~~~~~~~~~
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-pdrdisc.html"></iframe>
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Overlay
+*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Overlay
Tunnels.*
CSIT test cases used to generate results presented above can be found in CSIT
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-pdrdisc.html"></iframe>
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.*
+*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.*
CSIT test cases used to generate results presented above can be found in CSIT
git repository by filtering with specified regex as follows:
10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-pdrdisc
10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2xcbase-pdrdisc
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv4 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-pdrdisc" *
-
- 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4lispip4-ip4base-pdrdisc
- 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4lispip6-ip4base-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2xcbase-pdrdisc
-
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured Packets Per Second [pps] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
NDR Throughput
10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale2m-ndrdisc
40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc
-VPP NDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-ndrdisc.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv6
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-copwhtlistbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-iacldstbase-ndrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale200k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale20k-ndrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale2m-ndrdisc
- 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc
-
PDR Throughput
~~~~~~~~~~~~~~
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ip6-pdrdisc.html"></iframe>
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6
+*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6
Routed-Forwarding.*
CSIT test cases used to generate results presented above can be found in CSIT
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ip6-pdrdisc.html"></iframe>
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6
+*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6
Routed-Forwarding.*
CSIT test cases used to generate results presented above can be found in CSIT
10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale20k-pdrdisc
10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale2m-pdrdisc
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv6
-Routed-Forwarding.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" *
-
- 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-copwhtlistbase-pdrdisc
- 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-iacldstbase-pdrdisc
- 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-pdrdisc
- 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale200k-pdrdisc
- 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale20k-pdrdisc
- 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale2m-pdrdisc
-
-
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured Packets Per Second [pps] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
NDR Throughput
10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip4-ip6base-ndrdisc
10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip6-ip6base-ndrdisc
-VPP NDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ndrdisc.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv6 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip4-ip6base-ndrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip6-ip6base-ndrdisc
-
PDR Throughput
~~~~~~~~~~~~~~
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-pdrdisc.html"></iframe>
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 Overlay
+*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 Overlay
Tunnels.*
CSIT test cases used to generate results presented above can be found in CSIT
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-pdrdisc.html"></iframe>
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 Overlay
+*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 Overlay
Tunnels.*
CSIT test cases used to generate results presented above can be found in CSIT
10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6lispip4-ip6base-pdrdisc
10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6lispip6-ip6base-pdrdisc
-
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv6 Overlay
-Tunnels.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-pdrdisc" *
-
- 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6lispip4-ip6base-pdrdisc
- 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6lispip6-ip6base-pdrdisc
-
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured Packets Per Second [pps] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
NDR Throughput
40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc
40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc
-VPP NDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-ndrdisc.html"></iframe>
-
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy L2 Ethernet
-Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1ad-l2xcbase-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2xcbase-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc
- 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc
-
PDR Throughput
~~~~~~~~~~~~~~
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-l2-pdrdisc.html"></iframe>
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet
+*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet
Switching.*
CSIT test cases used to generate results presented above can be found in CSIT
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-l2-pdrdisc.html"></iframe>
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet
+*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet
Switching.*
CSIT test cases used to generate results presented above can be found in CSIT
10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc
40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy L2 Ethernet
-Switching.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" *
-
- 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1ad-l2xcbase-pdrdisc
- 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1q-l2xcbase-pdrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-pdrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc
-
their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs
+throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured Packets Per Second [pps] values, and the *graph
legend* identifes the plotted test suites.
.. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-vpp-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and
- `csit-vpp-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_
- , ii) archived FD.io jobs test result `output files
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result `output files
<../../_static/archive/>`_.
NDR Throughput
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-ndrdisc.html"></iframe>
-*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-VM-to-Phy VM
-vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-.*vhost.*-ndrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc
-
PDR Throughput
~~~~~~~~~~~~~~
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-vhost-pdrdisc.html"></iframe>
-*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-VM-to-Phy VM
+*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-VM-to-Phy VM
vhost-user.*
CSIT test cases used to generate results presented above can be found in CSIT
<iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-vhost-pdrdisc.html"></iframe>
-*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-VM-to-Phy VM
+*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-VM-to-Phy VM
vhost-user.*
CSIT test cases used to generate results presented above can be found in CSIT
10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
-VPP PDR Throughput - running in configuration of **four worker threads (4t) on
-four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-pdrdisc.html"></iframe>
-
-*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-VM-to-Phy VM
-vhost-user.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-.*vhost.*-pdrdisc" *
-
- 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc
- 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
- 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc
-
\r
SUT Configuration - Host HW\r
---------------------------\r
-Hardware details (CPU, memory, NIC layout) are described in\r
-`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_ in\r
-summary:\r
+Host hardware details (CPU, memory, NIC layout) and physical topology are\r
+described in detail in\r
+`LF FDio CSIT testbed wiki page <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.\r
+\r
+**Host configuration**\r
\r
- All hosts are Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz,\r
18c, 512GB RAM),\r
-- BIOS settings are default except for the following:\r
+- BIOS settings are default except the following:\r
\r
- - Hyperthreading disabled,\r
- - SpeedStep disabled\r
- - TurboBoost disabled\r
- - Power Technology: Performance\r
+ - Hyperthreading disabled.\r
+ - SpeedStep disabled.\r
+ - TurboBoost disabled.\r
+ - Power Technology: Performance.\r
\r
-- Hosts run Ubuntu 16.04.1, kernel 4.4.0-42-generic\r
+- Hosts run Ubuntu 16.04.1, kernel 4.4.0-42-generic.\r
- Linux kernel boot command line option "intel_pstate=disable" is applied to\r
both SUTs and TG. In addition, on SUTs, only cores 0 and 18 (the first core on\r
each socket) are available to the Linux operating system and generic tasks,\r
- This allows for a total of five ring topologies, each using ports on\r
specific NIC model, enabling per NIC model benchmarking.\r
\r
-**NIC types**\r
+**NIC models and placement**\r
\r
- 0a:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+\r
Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server\r
SUT Configuration - Host OS Linux\r
---------------------------------\r
\r
-Software details (OS, configuration) is described in\r
-`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_ in\r
-summary:\r
+Software details (OS, configuration) are described in\r
+`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.\r
+\r
+Below a subset of the configuration:\r
\r
.. code-block:: xml\r
\r
<exec_command><![CDATA[lsblk -l]]></exec_command>\r
<exec_return_code>0</exec_return_code>\r
<exec_output><![CDATA[NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT\r
- sda 8:0 0 1.8T 0 disk \r
+ sda 8:0 0 1.8T 0 disk\r
sda1 8:1 0 243M 0 part /boot\r
sda2 8:2 0 1.8T 0 part /\r
- sda3 8:3 0 1K 0 part \r
+ sda3 8:3 0 1K 0 part\r
sda5 8:5 0 976M 0 part [SWAP]\r
]]></exec_output>\r
</function>\r
dev.cdrom.check_media = 0\r
dev.cdrom.debug = 0\r
dev.cdrom.info = CD-ROM information, Id: cdrom.c 3.20 2003/12/17\r
- dev.cdrom.info = \r
- dev.cdrom.info = drive name: \r
- dev.cdrom.info = drive speed: \r
+ dev.cdrom.info =\r
+ dev.cdrom.info = drive name:\r
+ dev.cdrom.info = drive speed:\r
dev.cdrom.info = drive # of slots:\r
- dev.cdrom.info = Can close tray: \r
- dev.cdrom.info = Can open tray: \r
- dev.cdrom.info = Can lock tray: \r
+ dev.cdrom.info = Can close tray:\r
+ dev.cdrom.info = Can open tray:\r
+ dev.cdrom.info = Can lock tray:\r
dev.cdrom.info = Can change speed:\r
dev.cdrom.info = Can select disk:\r
dev.cdrom.info = Can read multisession:\r
- dev.cdrom.info = Can read MCN: \r
+ dev.cdrom.info = Can read MCN:\r
dev.cdrom.info = Reports media changed:\r
- dev.cdrom.info = Can play audio: \r
- dev.cdrom.info = Can write CD-R: \r
+ dev.cdrom.info = Can play audio:\r
+ dev.cdrom.info = Can write CD-R:\r
dev.cdrom.info = Can write CD-RW:\r
- dev.cdrom.info = Can read DVD: \r
+ dev.cdrom.info = Can read DVD:\r
dev.cdrom.info = Can write DVD-R:\r
dev.cdrom.info = Can write DVD-RAM:\r
- dev.cdrom.info = Can read MRW: \r
- dev.cdrom.info = Can write MRW: \r
- dev.cdrom.info = Can write RAM: \r
- dev.cdrom.info = \r
- dev.cdrom.info = \r
+ dev.cdrom.info = Can read MRW:\r
+ dev.cdrom.info = Can write MRW:\r
+ dev.cdrom.info = Can write RAM:\r
+ dev.cdrom.info =\r
+ dev.cdrom.info =\r
dev.cdrom.lock = 0\r
dev.hpet.max-user-freq = 64\r
dev.mac_hid.mouse_button2_keycode = 97\r
kernel.hardlockup_all_cpu_backtrace = 0\r
kernel.hardlockup_panic = 0\r
kernel.hostname = t1-sut1\r
- kernel.hotplug = \r
+ kernel.hotplug =\r
kernel.hung_task_check_count = 4194304\r
kernel.hung_task_panic = 0\r
kernel.hung_task_timeout_secs = 120\r
net.ipv4.ip_forward = 0\r
net.ipv4.ip_forward_use_pmtu = 0\r
net.ipv4.ip_local_port_range = 32768 60999\r
- net.ipv4.ip_local_reserved_ports = \r
+ net.ipv4.ip_local_reserved_ports =\r
net.ipv4.ip_no_pmtu_disc = 0\r
net.ipv4.ip_nonlocal_bind = 0\r
net.ipv4.ipfrag_high_thresh = 4194304\r
Code Review / csit.git / commitdiff