#. VPP PERFORMANCE TESTS
- - CSIT test environment is versioned, see
- :ref:`test_environment_versioning`.
-
- - **Upgrade to Ubuntu 20.04 LTS**: Reinstall base operating system to
- Ubuntu 20.04.2 LTS. Upgrade includes also baseline Docker
- containers used for spawning topology.
+ - **Intel Xeon Ice Lake**: Performance test data for these platforms
+ continues to be provided by external Intel benchmarking labs
+ executing |csit-release| tests. For details about the physical
+ setup see :ref:`tested_physical_topologies`.
- - Initial test data for Intel Xeon Ice Lake platforms. Current
- CSIT-2106 report data for Intel Xeon Ice Lake comes from an
- external source (Intel labs running CSIT code on 8360Y D Stepping
- and 6338N processors). For detail about the physical setup
- see :ref:`tested_physical_topologies`. Tested VPP and CSIT
- versions are pre-release, VPP 21.06-rc0~779-gd640ae52f.
+ - **AWS EC2 c5n**: Performance test data for AWS c5n instances has
+ been added to report for ip4, ip6 and ipsec tests.
- - **AF_XDP**: Added af_xdp driver testing for all testcases.
+ - **Reduction of tests**: Removed certain test variations executed
+ iteratively for the report (as well as in daily and weekly
+ trending) due to physical testbeds overload.
- - **GTPU tunnel**: Added GTPU HW Offload IPv4 routing tests.
+#. TEST FRAMEWORK
- - **Telemetry retouch**: Redesign telemetry retrieval from DUT.
- Include VPP perfmon plugin telemetry.
+ - **CSIT test environment** version has been updated to ver. 8, see
+ :ref:`test_environment_versioning`.
-#. TEST FRAMEWORK
+ - **CSIT in AWS environment**: Completed CSIT automation support for
+ AWS c5n instances environment, including weekly performance
+ trending tests execution.
- - **CSIT PAPI support**: Due to issues with PAPI performance, VAT is
- still used in CSIT for all VPP scale tests. See known issues below.
+ - **CSIT PAPI support**: Due to issues with PAPI performance, and
+ deprecation of VAT, CLI is used in CSIT for many VPP scale tests.
+ See known issues below.
- - **General Code Housekeeping**: Ongoing code optimizations,
- speed ups and bug fixes.
+ - **General Code Housekeeping**: Ongoing code optimizations and bug
+ fixes.
#. PRESENTATION AND ANALYTICS LAYER
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
| # | JiraID | Issue Description |
+====+=========================================+===========================================================================================================+
-| 1 | `CSIT-1782 | Multicore AVF tests are failing when trying to create interface. |
+| 1 | `CSIT-1782 | Multicore AVF tests are failing when trying to create interface. The issue persists after FVL FW upgrade. |
| | <https://jira.fd.io/browse/CSIT-1782>`_ | |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
| 2 | `CSIT-1763 | Adapt ramp-up phase of nat44 tests for different frame sizes. |
| | `VPP-1763 | commands in flight over socket), the VPP configuration for large scale tests (millions of messages) takes |
| | <https://jira.fd.io/browse/VPP-1763>`_ | too long. |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
+| 4 | `CSIT-1789 | AVF driver does not perform RSS in a deterministic way. |
+| | <https://jira.fd.io/browse/CSIT-1789>`_ | This increases standard deviation of tests with small number of flows (mainly ipsec). |
++----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
+| 5 | `CSIT-1780 | IPSEC SW async scheduler MRR tests failing with no traffic forwarded. |
+| | <https://jira.fd.io/browse/CSIT-1780>`_ | |
++----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
+| 6 | `CSIT-1786 | IP4 and IP6 scale tests failing with no traffic forwarded. |
+| | <https://jira.fd.io/browse/CSIT-1786>`_ | Issue reported to VPP devs. |
++----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
+| 7 | `CSIT-1785 | NAT44ED tests failing to establish all TCP sessions. |
+| | <https://jira.fd.io/browse/CSIT-1785>`_ | |
++----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
+| 8 | `CSIT-1791 | Performance regression in RDMA tests, due to CSIT environment changes. |
+| | <https://jira.fd.io/browse/CSIT-1791>`_ | |
++----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
Root Cause Analysis for Performance Changes
-------------------------------------------
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
| # | JiraID | Issue Description |
+====+=========================================+===========================================================================================================+
-| 1 | `VPP-1972 | One VPP change has decreased performance of NAT44ed processing, both slow path and fast path. |
+| 1 | `VPP-1972 | One VPP change has decreased performance of NAT44ed processing, mostly slow path. |
| | <https://jira.fd.io/browse/VPP-1972>`_ | |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
Code Review / csit.git / blobdiff