iteratively for the report (as well as in daily and weekly
trending) due to physical testbeds overload.
+ - **RC2 coverage test data is used for 2n-icx test bed**: There is only
+ one 2n-icx test bed and the amount of tests is large (same as 2n-skx with 4
+ test beds), so we decided to use test data already available from RC2
+ testing.
+
#. TEST FRAMEWORK
- **CSIT test environment** version has been updated to ver. 9, see
| 9 | `CSIT-1805 | 3n-skx testbeds (Skylake): all hoststack quic vppecho scale tests are failing. |
| | <https://jira.fd.io/browse/CSIT-1805>`_ | |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 10 | `CSIT-1806 | 3n-skx testbeds (Skylake): NDRPDR LISP ip6-ip6 tests are failing due to 100% drops in one direction. |
-| | <https://jira.fd.io/browse/CSIT-1806>`_ | |
-+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 11 | `CSIT-1810 | DPDK - performance regression with DPDK driver when Max Frame Size is set to less than 2023. |
+| 10 | `CSIT-1810 | DPDK - performance regression with DPDK driver when Max Frame Size is set to less than 2023. |
| | <https://jira.fd.io/browse/CSIT-1810>`_ | |
| +-----------------------------------------+ |
| | `VPP-1876 | Worse performance with DPDK driver when MTU is set to 2022 or less. |
| | <https://jira.fd.io/browse/VPP-1876>`_ | |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 12 | `CSIT-1811 | All 9000B NAT44DET 64k 1m scale tests fail due to bps rate set to high on TRex. |
+| 11 | `CSIT-1811 | All 9000B NAT44DET 64k 1m scale tests fail due to bps rate set to high on TRex. |
| | <https://jira.fd.io/browse/CSIT-1811>`_ | |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 13 | `CSIT-1812 | All IMIX NAT44DET 4m 16m scale tests fail due to not creating required session count. |
+| 12 | `CSIT-1812 | All IMIX NAT44DET 4m 16m scale tests fail due to not creating required session count. |
| | <https://jira.fd.io/browse/CSIT-1812>`_ | |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
| | `VPP-1763 | Still, we needed to reduce the number of scale tests run to keep overall duration reasonable. |
| | <https://jira.fd.io/browse/VPP-1763>`_ | More improvements needed to achieve sufficient configuration speed. |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 3 | `CSIT-1789 | IPSEC SW async scheduler tests show bad behavior. |
-| | <https://jira.fd.io/browse/CSIT-1789>`_ | VPP code is not behaving correctly when crypto workers are the bottleneck. |
-| +-----------------------------------------+ |
-| | `VPP-1998 | |
-| | <https://jira.fd.io/browse/VPP-1998>`_ | |
-+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 4 | `CSIT-1785 | NAT44ED tests failing to establish all TCP sessions. |
+| 3 | `CSIT-1785 | NAT44ED tests failing to establish all TCP sessions. |
| | <https://jira.fd.io/browse/CSIT-1785>`_ | At least for max scale, in allotted time (limited by session 500s timeout) due to worse |
| +-----------------------------------------+ slow path performance than previously measured and calibrated for. |
| | `VPP-1972 | CSIT removed the max scale NAT tests to avoid this issue. |
| | <https://jira.fd.io/browse/VPP-1972>`_ | |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 5 | `CSIT-1791 | Performance regression in RDMA tests, due to CSIT environment changes. |
+| 4 | `CSIT-1791 | Performance regression in RDMA tests, due to CSIT environment changes. |
| | <https://jira.fd.io/browse/CSIT-1791>`_ | Two symptoms: 1. 10-20% regression across most tests. 2. DUT performance cap just below 38 Mpps. |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
| # | JiraID | Issue Description |
+====+=========================================+===========================================================================================================+
-| 1 | `CSIT-1789 | AVF driver does not perform RSS in a deterministic way. |
-| | <https://jira.fd.io/browse/CSIT-1789>`_ | VPP now uses the same RSS key with AVF driver as with DPDK driver. |
-+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
-| 2 | `CSIT-1786 | IP4 and IP6 scale tests failing with no traffic forwarded. |
-| | <https://jira.fd.io/browse/CSIT-1786>`_ | CSIT replaced the old single VAT command by file full of "exec" CLI commands executed by VAT. |
+| 1 | `CSIT-1789 | IPSEC SW async scheduler tests show bad behavior. |
+| | <https://jira.fd.io/browse/CSIT-1789>`_ | VPP code is not behaving correctly when crypto workers are the bottleneck. |
+| +-----------------------------------------+ |
+| | `VPP-1998 | |
+| | <https://jira.fd.io/browse/VPP-1998>`_ | |
+----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
Root Cause Analysis for Performance Changes
List of RCAs in |csit-release| for VPP performance changes:
-+----+-----------------------------------------+------------------------------------------------------------+
-| # | JiraID | Issue Description |
-+====+=========================================+============================================================+
-| 0 | | To be updated when descriptions in Jira tickets are ready. |
-+----+-----------------------------------------+------------------------------------------------------------+
++----+-----------------------------------------+--------------------------------------------------------------+
+| # | JiraID | Issue Description |
++====+=========================================+==============================================================+
+| 1 | `CSIT-1814 | rls2202 regression: NAT44ED CPS larger scales. |
+| | <https://jira.fd.io/browse/CSIT-1814>`_ | |
++----+-----------------------------------------+--------------------------------------------------------------+
+| 2 | `CSIT-1815 | rls2202 regression: l2bdbasemaclrn around 2021-11-11. |
+| | <https://jira.fd.io/browse/CSIT-1815>`_ | |
++----+-----------------------------------------+--------------------------------------------------------------+
+| 3 | `CSIT-1816 | rls2202 regression: l2-input. |
+| | <https://jira.fd.io/browse/CSIT-1816>`_ | |
++----+-----------------------------------------+--------------------------------------------------------------+
+| 4 | `CSIT-1820 | rls2202 regression: tx2 ip6base at dpdk upgrade. |
+| | <https://jira.fd.io/browse/CSIT-1820>`_ | |
++----+-----------------------------------------+--------------------------------------------------------------+
+| 5 | `CSIT-1821 | rls2202 regression: tx2 ip6base before dpdk upgrade. |
+| | <https://jira.fd.io/browse/CSIT-1821>`_ | |
++----+-----------------------------------------+--------------------------------------------------------------+
+| 6 | `CSIT-1822 | rls2202 regression: avf-ip4base around 2022-01-06. |
+| | <https://jira.fd.io/browse/CSIT-1822>`_ | |
++----+-----------------------------------------+--------------------------------------------------------------+
+| 7 | `CSIT-1823 | rls2202 regression: vxlan-l2xcbase on tsh around 2022-01-07. |
+| | <https://jira.fd.io/browse/CSIT-1823>`_ | |
++----+-----------------------------------------+--------------------------------------------------------------+
+| 8 | `CSIT-1824 | rls2202 regression: SRv6. |
+| | <https://jira.fd.io/browse/CSIT-1824>`_ | |
++----+-----------------------------------------+--------------------------------------------------------------+