#. **VPP performance tests**
- - *MRR tests* - New Maximum Receive Rate tests measure the packet
- forwarding rate under the maximum load offered by traffic
- generator over a set trial duration, regardless of packet loss.
- MRR tests are used for continuous performance trending and for
- comparison between releases.
+ - *MRR tests* - Maximum Receive Rate tests measure the packet forwarding rate
+ under the maximum load offered by traffic generator over a set trial
+ duration, regardless of packet loss. MRR tests are used for continuous
+ performance trending and for comparison between releases.
- - *Service Chaining with SRv6* - New SRv6 (Segment Routing IPv6) proxy
- tests measure performance of SRv6 Endpoint fronting SR-unaware
- appliance via masquerading (End.AM), dynamic proxy (End.AD) or
- static proxy (End.AS) SR functions.
+ - *MLR tests* - NDR and PDR tests measure the packet forwarding rate using
+ MLRsearch library by traffic generator. All tests that previously used
+ binary search were converted to MLRsearch.
+
+ - *L2patch tests* - Tests measure performance of L2patch, cross linking
+ RX and TX of two physical interfaces.
+
+ - *2-node tests* - Baseline set of 2-node tests covering base ip4, ip4,
+ l2patch, l2bd, l2xc.
+
+ - *Generated tests* - Simplified and unified test structure,
+ semi-autogenerated by generator script. Test generator is currently able
+ to create test combinations with various frame size and cores combinations.
+ All existing test cases were converted to new format.
+
+ - *Simultaneous multithreading* - SMT-aware automatic detection and
+ resource configuration including thread affinity, number of rx queues and
+ DPDK I/O mbufs. Tests are automatically tagged during execution indicating
+ configuration.
+
+ - *Intel Skylake* - Topologies consisting of 2-node and 3-node using
+ SuperMirco servers each equipped with two Intel Xeon Skylake Platinum
+ processors.
#. **Presentation and Analytics Layer**
.. note::
Test results have been generated by
- `FD.io test executor vpp mrr jobs <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master/>`_
- with Robot Framework result files csit-vpp-perf-mrr-daily-master__*__output.xml.gz
- `archived here <https://docs.fd.io/csit/master/trending/_static/archive/>`_.
+ `FD.io test executor vpp performance job 3n-hsw`_,
+ with Robot Framework result
+ files csit-vpp-perf-|srelease|-\*.zip
+ `archived here <../_static/archive/>`_.
Comparison Across Testbeds
--------------------------
files csit-vpp-perf-|srelease|-\*.zip
`archived here <../_static/archive/>`_.
+MRR Changes
+~~~~~~~~~~~
+
+MRR throughput changes between testbeds are available in a
+CSV and pretty ASCII formats:
+
+ - `csv format for mrr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
+ - `pretty ASCII format for mrr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
+
+.. note::
+
+ Test results have been generated by
+ `FD.io test executor vpp performance job 3n-hsw`_ and
+ `FD.io test executor vpp performance job 3n-skx`_
+ with Robot Framework result
+ files csit-vpp-perf-|srelease|-\*.zip
+ `archived here <../_static/archive/>`_.
+
Throughput Trending
-------------------
| | on x520. | | with HW combination of X710-X520 in LF testbeds. Not observed |
| | | | outside of LF testbeds. |
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+
-| 2 | Lower than expected NDR throughput of DPDK | CSIT-571 | Suspected NIC firmware or DPDK driver issue affecting NDR and |
-| | testpmd and VPP L2 path NDR throughput with | | PDR throughput on XL710 and X710 NICs. |
-| | xl710 and x710 NICs, compared to x520 NICs. | | |
-+---+-------------------------------------------------+------------+-----------------------------------------------------------------+
-| 3 | Tagged Ethernet dot1q and dot1ad L2 path | CSIT-1066 | Tagged Ethernet dot1q and dot1ad L2 path throughput regression: |
-| | throughput regression. | | NDR -2%..-5%, PDR -2%..-6%, MRR. Affects l2xc and l2bd |
-| | | | performance tests. |
-+---+-------------------------------------------------+------------+-----------------------------------------------------------------+
-| 4 | IPSec (software, no QAT HW) throughput | CSIT-1064 | IPSec throughput regression: NDR -3%..-8%, PDR -2%..-8%, MRR |
-| | regression. | | -3%..-7%. Affects IPSec SW tests, QAT HW tests not affected. |
+| 2 | High failure rate of api call | VPP-1361 | Failure rate: 30-40% of tests failing due to interfaces not |
+| | sw_interface_set_flags [admin-up|link-up] | | in link-up state after API call sw_interface_set_flags. |
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+
-| 5 | High failure rate of creating working container | CSIT-1065 | About 20% of orchestrated container topology tests failing data |
-| | topologies with K8s/Ligato orchestration. | | plane verification indicating configuration issue. Suspected |
-| | | | issue with Ligato vpp-agent. |
+| 3 | Scale IPSecHW Interface mode throughput | CSIT-1234 | IPSec throughput regression: NDR -28%..-31%, PDR -28%..-31%, |
+| | regression. | | MRR -40%. Affects IPSec HW Scale 1000tnl tests with Int mode. |
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+
Code Review / csit.git / blobdiff