X-Git-Url: https://gerrit.fd.io/r/gitweb?a=blobdiff_plain;ds=sidebyside;f=docs%2Freport%2Fvpp_performance_tests%2Fcsit_release_notes.rst;h=6ac7fcdfc7b967bd688439978af14a77699f985e;hb=e253cf226e1595acecf42db1262e1cb179aae5e7;hp=99cd9e85401ac43e50a285a690efe7ef693f9cc9;hpb=ee1321d27bc39baab5bc2615834fd7392b79a160;p=csit.git
diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 99cd9e8540..6ac7fcdfc7 100644
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -18,7 +18,7 @@ Changes in |csit-release|
- *L2patch tests* - Tests measure performance of L2patch, cross linking
RX and TX of two physical interfaces.
- - *2-node tests* - Baseline set of 2-node tests covering base ip4, ip4,
+ - *2-node tests* - Baseline set of 2-node tests covering base ip4, ip6,
l2patch, l2bd, l2xc.
- *Generated tests* - Simplified and unified test structure,
@@ -31,9 +31,10 @@ Changes in |csit-release|
DPDK I/O mbufs. Tests are automatically tagged during execution indicating
configuration.
- - *Intel Skylake* - Topologies consisting of 2-node and 3-node using
- SuperMirco servers each equipped with two Intel Xeon Skylake Platinum
- processors.
+ - *Intel Skylake* - Topologies consisting of 2-node and 3-node using
+ SuperMirco servers each equipped with two Intel Xeon Skylake Platinum
+ processors. Full Ansible playbooks refactor for quick server
+ (re)installation and reference pointers of configuration.
#. **Presentation and Analytics Layer**
@@ -45,11 +46,6 @@ Changes in |csit-release|
#. **Test Framework Optimizations**
- - *Performance tests efficiency* - Qemu build/install optimizations,
- warmup phase handling, vpp restart handling. Resulted in improved
- stability and reduced total execution time by 30% for single pkt
- size e.g. 64B/78B.
-
- *General code housekeeping* - ongoing RF keywords optimizations,
removal of redundant RF keywords.
@@ -121,9 +117,10 @@ CSV and pretty ASCII formats:
.. note::
Test results have been generated by
- `FD.io test executor vpp mrr jobs `_
- with Robot Framework result files csit-vpp-perf-mrr-daily-master__*__output.xml.gz
- `archived here `_.
+ `FD.io test executor vpp performance job 3n-hsw`_,
+ with Robot Framework result
+ files csit-vpp-perf-|srelease|-\*.zip
+ `archived here <../_static/archive/>`_.
Comparison Across Testbeds
--------------------------
@@ -167,6 +164,24 @@ CSV and pretty ASCII formats:
files csit-vpp-perf-|srelease|-\*.zip
`archived here <../_static/archive/>`_.
+MRR Changes
+~~~~~~~~~~~
+
+MRR throughput changes between testbeds are available in a
+CSV and pretty ASCII formats:
+
+ - `csv format for mrr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
+ - `pretty ASCII format for mrr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
+
+.. note::
+
+ Test results have been generated by
+ `FD.io test executor vpp performance job 3n-hsw`_ and
+ `FD.io test executor vpp performance job 3n-skx`_
+ with Robot Framework result
+ files csit-vpp-perf-|srelease|-\*.zip
+ `archived here <../_static/archive/>`_.
+
Throughput Trending
-------------------
@@ -197,14 +212,9 @@ List of known issues in |csit-release| for VPP performance tests:
| | on x520. | | with HW combination of X710-X520 in LF testbeds. Not observed |
| | | | outside of LF testbeds. |
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+
-| 2 | Lower than expected NDR throughput of DPDK | CSIT-571 | Suspected NIC firmware or DPDK driver issue affecting NDR and |
-| | testpmd and VPP L2 path NDR throughput with | | PDR throughput on XL710 and X710 NICs. |
-| | xl710 and x710 NICs, compared to x520 NICs. | | |
-+---+-------------------------------------------------+------------+-----------------------------------------------------------------+
-| 3 | Tagged Ethernet dot1q and dot1ad L2 path | CSIT-1066 | Tagged Ethernet dot1q and dot1ad L2 path throughput regression: |
-| | throughput regression. | | NDR -2%..-5%, PDR -2%..-6%, MRR. Affects l2xc and l2bd |
-| | | | performance tests. |
+| 2 | High failure rate of api call | VPP-1361 | Failure rate: 30-40% of tests failing due to interfaces not |
+| | sw_interface_set_flags [admin-up|link-up] | | in link-up state after API call sw_interface_set_flags. |
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+
-| 4 | IPSec (software, no QAT HW) throughput | CSIT-1064 | IPSec throughput regression: NDR -3%..-8%, PDR -2%..-8%, MRR |
-| | regression. | | -3%..-7%. Affects IPSec SW tests, QAT HW tests not affected. |
+| 3 | Scale IPSecHW Interface mode throughput | CSIT-1234 | IPSec throughput regression: NDR -28%..-31%, PDR -28%..-31%, |
+| | regression. | | MRR -40%. Affects IPSec HW Scale 1000tnl tests with Int mode. |
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+