rls1801 report: edits to static content for vpp and dpdk perf sections.
[csit.git] / docs / report / vpp_performance_tests / csit_release_notes.rst
index 7c17e0f..17003bc 100644 (file)
@@ -4,199 +4,102 @@ CSIT Release Notes
 Changes in CSIT |release|\r
 -------------------------\r
 \r
-#. VPP performance test environment changes\r
+#. Added VPP performance tests\r
 \r
-    - Further optimizations of VM and vhost-user test environment - Qemu virtio\r
-      queue size increased from default value of 256 to 1024.\r
-    - Addition of HW cryptodev devices - Intel QAT 8950 50G - in all three\r
-      LF FD.io physical testbeds.\r
+   - **Container Service Chain Topologies Orchestrated by K8s with VPP Memif**\r
 \r
-#. VPP performance test framework changes\r
+     - Added tests with VPP vswitch in container connecting a number of VPP-\r
+       in-container service chain topologies with L2 Cross-Connect and L2\r
+       Bridge-Domain configurations, orchestrated by Kubernetes. Added\r
+       following forwarding topologies: i) "Parallel" with packets flowing from\r
+       NIC via VPP to container and back to VPP and NIC; ii) "Chained" (a.k.a.\r
+       "Snake") with packets flowing via VPP to container, back to VPP, to next\r
+       container, back to VPP and so on until the last container in a chain,\r
+       then back to VPP and NIC; iii) "Horizontal" with packets flowing via VPP\r
+       to container, then via "horizontal" memif to next container, and so on\r
+       until the last container, then back to VPP and NIC;\r
 \r
-    - Added VAT command history collection for every test case as part of teardown.\r
+   - **VPP TCP/IP stack**\r
 \r
-#. Added VPP performance tests\r
+     - Added tests for VPP TCP/IP stack using VPP built-in HTTP server.\r
+       WRK traffic generator is used as a client-side;\r
 \r
-    - **CGNAT**\r
-\r
-      - Carrier Grade Network Address Translation tests with varying number\r
-        of users and ports per user: 1u-15p, 10u-15p, 100u-15p, 1000u-15p,\r
-        2000u-15p, 4000u-15p - with Intel x520 NIC.\r
-\r
-    - **vhost-user tests with one VM**\r
-\r
-      - L2 Bridge Domain switched-forwarding with Intel x710 NIC, Intel x520 NIC,\r
-        Intel xl710 NIC.\r
-      - VXLAN and L2 Bridge Domain switched-forwarding with Intel x520 NIC.\r
-\r
-    - **vhost-user tests with two VMs service chain**\r
-\r
-      - L2 cross-connect switched-forwarding with Intel x520 NIC, Intel xl710 NIC.\r
-      - L2 Bridge Domain switched-forwarding with Intel x520 NIC, Intel xl710 NIC.\r
-      - IPv4 routed-forwarding with Intel x520 NIC, Intel xl710 NIC.\r
-\r
-    - **IPSec encryption with**\r
-\r
-      - AES-GCM, CBC-SHA1 ciphers, in combination with IPv4 routed-forwarding\r
-        with Intel xl710 NIC.\r
-      - CBC-SHA1 ciphers, in combination with LISP-GPE overlay tunneling for\r
-        IPv4-over-IPv4 with Intel xl710 NIC.\r
-\r
-Performance Improvements\r
-------------------------\r
-\r
-Some performance improvements in measured packet throughput have been observed\r
-in a number of CSIT |release| tests listed below. Relative improvements are\r
-calculated against the test results in CSIT rls1701 report. VPP-16.09 and\r
-VPP-17.01 numbers are provided for reference.\r
-\r
-NDR Throughput\r
-~~~~~~~~~~~~~~\r
-\r
-Non-Drop Rate Throughput discovery tests:\r
-\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| VPP Functionality | Test Name                                                       | VPP-16.09  | VPP-17.01 | VPP-17.04 | 17.01 to 17.04  |\r
-|                   |                                                                 |   [Mpps]   |  [Mpps]   |   [Mpps]  | Relative Change |\r
-+===================+=================================================================+============+===========+===========+=================+\r
-| L2XC              | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-ndrdisc                      | 9.4        | 12.7      | 13.4      | 6%              |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| L2XC              | 10ge2p1xl710: 64B-1t1c-eth-l2xcbase-ndrdisc                     | 9.5        | 12.2      | 12.4      | 2%              |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| L2XC dot1ad       | 10ge2p1x520: 64B-1t1c-dot1ad-l2xcbase-ndrdisc                   | 7.4        | 8.8       | 9.3       | 6%              |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| L2XC dot1q        | 10ge2p1x520: 64B-1t1c-dot1q-l2xcbase-ndrdisc                    | 7.5        | 8.8       | 9.2       | 5%              |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| L2XC VxLAN        | 10ge2p1x520: 64B-1t1c-ethip4vxlan-l2xcbase-ndrdisc              | 5.4        | 6.5       | 6.8       | 5%              |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| L2XC-vhost-VM     | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc       | 0.5        | 2.8       | 3.2       | 14%             |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| L2BD              | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-ndrdisc                | 7.8        | 10.4      | 10.8      | 4%              |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| L2BD-vhost-VM     | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc | 0.4        | 2.7       | 3.4       | 26%             |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| IPv4              | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ndrdisc                    | 8.7        | 9.7       | 10.6      | 9%              |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| IPv4 COP          | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc     | 7.1        | 8.3       | 9.0       | 8%              |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| IPv4 iAcl         | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc        | 6.9        | 7.6       | 8.3       | 9%              |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| IPv4 vhost        | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc     | 0.3        | 2.6       | 3.1       | 19%             |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-| IPv6              | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-ndrdisc                    | 3.0        | 7.3       | 8.1       | 11%             |\r
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+\r
-\r
-PDR Throughput\r
-~~~~~~~~~~~~~~\r
-\r
-Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%:\r
-\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| VPP Functionality | Test Name                                                       | VPP-16.09 | VPP-17.01 | VPP-17.04 | 17.01 to 17.04  |\r
-|                   |                                                                 |   [Mpps]  |  [Mpps]   |   [Mpps]  | Relative Change |\r
-+===================+=================================================================+===========+===========+===========+=================+\r
-| L2XC              | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-pdrdisc                      | 9.4       | 12.7      | 13.4      | 6%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| L2XC dot1ad       | 10ge2p1x520: 64B-1t1c-dot1ad-l2xcbase-pdrdisc                   | 7.4       | 8.8       | 9.3       | 6%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| L2XC dot1q        | 10ge2p1x520: 64B-1t1c-dot1q-l2xcbase-pdrdisc                    | 7.5       | 8.8       | 9.5       | 8%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| L2XC VxLAN        | 10ge2p1x520: 64B-1t1c-ethip4vxlan-l2xcbase-pdrdisc              | 5.4       | 6.5       | 6.8       | 5%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| L2XC-vhost-VM     | 10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc       | 2.6       | 3.2       | 3.2       | 0%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| L2BD              | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-pdrdisc                | 7.8       | 10.6      | 11.1      | 5%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| L2BD-vhost-VM     | 10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc | 2.1       | 2.9       | 3.2       | 10%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4              | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-pdrdisc                    | 8.7       | 9.7       | 10.6      | 9%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4 COP          | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-copwhtlistbase-pdrdisc     | 7.1       | 8.3       | 9.2       | 11%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4 iAcl         | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-iacldstbase-pdrdisc        | 7.1       | 7.6       | 8.3       | 9%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4 vhost        | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc     | 2.0       | 2.7       | 3.2       | 19%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv6              | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-pdrdisc                    | 7.7       | 7.3       | 8.1       | 11%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
+   - **SRv6**\r
 \r
-Measured improvements are in line with VPP code optimizations listed in\r
-`VPP-17.04 release notes\r
-<https://docs.fd.io/vpp/17.04/release_notes_1704.html>`_.\r
+     - Initial SRv6 (Segment Routing IPv6) tests verifying performance of\r
+       IPv6 and SRH (Segment Routing Header) encapsulation, decapsulation,\r
+       lookups and rewrites based on configured End and End.DX6 SRv6 egress\r
+       functions;\r
 \r
-Additionally, vhost-VM performance improvements are due to both VPP code\r
-optimizations as well as due to the FD.io CSIT Linux KVM test environment\r
-optimizations for vhost-VM tests - see section "2.1.7. Methodology: KVM VM\r
-vhost".\r
+   - **IPSecSW**\r
 \r
+     - SW computed IPSec encryption with AES-GCM, CBC-SHA1 ciphers, in\r
+       combination with IPv4 routed-forwarding;\r
 \r
-Other Performance Changes\r
--------------------------\r
+#. Presentation and Analytics Layer\r
+\r
+     - Added throughput speedup analysis for multi-core and multi-thread\r
+       VPP tests into Presentation and Analytics Layer (PAL) for automated\r
+       CSIT test results analysis;\r
+\r
+#. Other changes\r
+\r
+     - **Framework optimizations**\r
+\r
+       - Ability to run CSIT framework on ARM architecture;\r
+\r
+       - Overall stability improvements;\r
+\r
+     - **NDR and PDR throughput binary search change**\r
+\r
+       - Increased binary search resolution by reducing final step from\r
+         100kpps to 50kpps;\r
+\r
+     - **VPP plugin loaded as needed by tests**\r
+\r
+       - From this release only plugins required by tests are loaded at\r
+         VPP initialization time. Previously all plugins were loaded for\r
+         all tests;\r
 \r
-Other changes in measured packet throughput, with either minor relative\r
-increase or decrease, have been observed in a number of CSIT |release| tests\r
-listed below. Relative changes are calculated against the test results in CSIT\r
-rls1701 report.\r
-\r
-NDR Throughput\r
-~~~~~~~~~~~~~~\r
-\r
-Non-Drop Rate Throughput discovery tests:\r
-\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| VPP Functionality | Test Name                                                       | VPP-16.09 | VPP-17.01 | VPP-17.04 | 17.01 to 17.04  |\r
-|                   |                                                                 |   [Mpps]  |  [Mpps]   |   [Mpps]  | Relative Change |\r
-+===================+=================================================================+===========+===========+===========+=================+\r
-| IPv4 FIB 200k     | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale200k-ndrdisc               | 8.5       | 9.0       | 9.7       | 8%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4 FIB 20k      | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale20k-ndrdisc                | 8.5       | 9.0       | 9.4       | 4%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4 FIB 2M       | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale2m-ndrdisc                 | 8.5       | 7.8       | 8.1       | 4%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4 Policer      | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc    | 6.9       | 7.4       | 8.1       | 9%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4 LISP         | 10ge2p1x520: 64B-1t1c-ethip4lispip4-ip4base-ndrdisc             | 4.4       | 4.8       | 5.5       | 15%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv6 COP          | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-copwhtlistbase-ndrdisc     | 6.1       | 6.1       | 6.9       | 13%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv6 FIB 200k     | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale200k-ndrdisc               | 6.5       | 5.3       | 5.3       | 0%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv6 FIB 20k      | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-ndrdisc                | 6.9       | 6.5       | 6.9       | 6%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv6 FIB 2M       | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-ndrdisc                 | 5.3       | 4.2       | 4.6       | 10%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv6 iAcl         | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-ndrdisc        | 6.5       | 6.1       | 6.9       | 13%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-\r
-PDR Throughput\r
-~~~~~~~~~~~~~~\r
-\r
-Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%:\r
-\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| VPP Functionality | Test Name                                                       | VPP-16.09 | VPP-17.01 | VPP-17.04 | 17.01 to 17.04  |\r
-|                   |                                                                 |   [Mpps]  |  [Mpps]   |   [Mpps]  | Relative Change |\r
-+===================+=================================================================+===========+===========+===========+=================+\r
-| IPv4 FIB 200k     | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale200k-pdrdisc               | 8.5       | 9.0       | 9.7       | 8%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4 FIB 20k      | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale20k-pdrdisc                | 8.5       | 9.0       | 9.7       | 8%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4 FIB 2M       | 10ge2p1x520: 64B-1t1c-ethip4-ip4scale2m-pdrdisc                 | 8.3       | 8.1       | 8.3       | 2%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4 Policer      | 10ge2p1x520: 64B-1t1c-ethip4-ip4base-ipolicemarkbase-pdrdisc    | 7.1       | 7.4       | 8.1       | 9%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv4 LISP         | 10ge2p1x520: 64B-1t1c-ethip4lispip4-ip4base-pdrdisc             | 4.6       | 4.8       | 5.5       | 15%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv6 COP          | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-copwhtlistbase-pdrdisc     | 6.1       | 6.1       | 6.9       | 13%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv6 FIB 200k     | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale200k-pdrdisc               | 6.9       | 5.3       | 5.3       | 0%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv6 FIB 20k      | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-pdrdisc                | 6.9       | 6.5       | 6.9       | 6%              |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv6 FIB 2M       | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-pdrdisc                 | 5.3       | 4.2       | 4.6       | 10%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
-| IPv6 iAcl         | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-pdrdisc        | 6.5       | 6.1       | 6.9       | 13%             |\r
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+\r
+Performance Changes\r
+-------------------\r
+\r
+Relative performance changes in measured packet throughput in CSIT\r
+|release| are calculated against the results from CSIT |release-1|\r
+report. Listed mean and standard deviation values are computed based on\r
+a series of the same tests executed against respective VPP releases to\r
+verify test results repeatibility, with percentage change calculated for\r
+mean values. Note that the standard deviation is quite high for a small\r
+number of packet throughput tests, what indicates poor test results\r
+repeatability and makes the relative change of mean throughput value not\r
+fully representative for these tests. The root causes behind poor\r
+results repeatibility vary between the test cases.\r
+\r
+NDR Throughput Changes\r
+~~~~~~~~~~~~~~~~~~~~~~\r
+\r
+NDR small packet throughput changes between releases are available in a CSV and\r
+pretty ASCII formats:\r
+\r
+  - `csv format for 1t1c <../_static/vpp/performance-changes-ndr-1t1c-full.csv>`_,\r
+  - `csv format for 2t2c <../_static/vpp/performance-changes-ndr-2t2c-full.csv>`_,\r
+  - `pretty ASCII format for 1t1c <../_static/vpp/performance-changes-ndr-1t1c-full.txt>`_,\r
+  - `pretty ASCII format for 2t2c <../_static/vpp/performance-changes-ndr-2t2c-full.txt>`_.\r
+\r
+PDR Throughput Changes\r
+~~~~~~~~~~~~~~~~~~~~~~\r
+\r
+NDR small packet throughput changes between releases are available in a CSV and\r
+pretty ASCII formats:\r
+\r
+  - `csv format for 1t1c <../_static/vpp/performance-changes-pdr-1t1c-full.csv>`_,\r
+  - `csv format for 2t2c <../_static/vpp/performance-changes-pdr-2t2c-full.csv>`_,\r
+  - `pretty ASCII format for 1t1c <../_static/vpp/performance-changes-pdr-1t1c-full.txt>`_,\r
+  - `pretty ASCII format for 2t2c <../_static/vpp/performance-changes-pdr-2t2c-full.txt>`_.\r
+\r
+Measured improvements are in line with VPP code optimizations listed in\r
+`VPP-18.01 release notes\r
+<https://docs.fd.io/vpp/18.01/release_notes_1801.html>`_.\r
 \r
 Known Issues\r
 ------------\r
@@ -206,25 +109,17 @@ Here is the list of known issues in CSIT |release| for VPP performance tests:
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 | # | Issue                                           | Jira ID    | Description                                                     |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 1 | NDR discovery test failures 1518B frame size    | VPP-663    | VPP reporting errors: dpdk-input Rx ip checksum errors.         |\r
-|   | for ip4scale200k, ip4scale2m scale IPv4 routed- |            | Observed frequency: all test runs.                              |\r
-|   | forwarding tests. ip4scale20k tests are fine.   |            |                                                                 |\r
-+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 2 | VAT API timeouts during ip6scale2m scale IPv6   | VPP-712    | Needs fixing VPP VAT API timeouts for large volume of IPv6      |\r
-|   | routed-forwarding tests when volume adding IPv6 |            | routes.                                                         |\r
-|   | routes - 2M in this case. ip6scale2kk works.    |            |                                                                 |\r
+| 1 | Vic1385 and Vic1227 low performance.            | VPP-664    | Low NDR performance.                                            |\r
+|   |                                                 |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 3 | Vic1385 and Vic1227 low performance             | VPP-664    | Low NDR performance.                                            |\r
-|   |                                                 |            |                                       .                         |\r
+| 2 | Sporadic (1 in 200) NDR discovery test failures | CSIT-570   | DPDK reporting rx-errors, indicating L1 issue. Suspected issue  |\r
+|   | on x520.                                        |            | with HW combination of X710-X520 in LF testbeds. Not observed   |\r
+|   |                                                 |            | outside of LF testbeds.                                         |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 4 | Sporadic NDR discovery test failures on x520    | CSIT-750   | Suspected issue with HW settings (BIOS, FW) in LF               |\r
-|   |                                                 |            | infrastructure. Issue can't be replicated outside LF.           |\r
-+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 5 | VPP in 2t2c setups - large variation            | CSIT-568   | Suspected NIC firmware or DPDK driver issue affecting NDR       |\r
-|   | of discovered NDR throughput values across      |            | throughput. Applies to XL710 and X710 NICs, x520 NICs are fine. |\r
-|   | multiple test runs with xl710 and x710 NICs.    |            |                                       .                         |\r
-+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 6 | Lower than expected NDR and PDR throughput with | CSIT-569   | Suspected NIC firmware or DPDK driver issue affecting NDR and   |\r
+| 3 | Lower than expected NDR throughput with         | CSIT-571   | Suspected NIC firmware or DPDK driver issue affecting NDR and   |\r
 |   | xl710 and x710 NICs, compared to x520 NICs.     |            | PDR throughput. Applies to XL710 and X710 NICs.                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
+| 4 | QAT IPSec scale with 1000 tunnels (interfaces)  | VPP-1121   | VPP crashes during configuration of 1000 IPsec tunnels.         |\r
+|   | in 2t2c config, all tests are failing.          |            | 1t1c tests are not affected                                     |\r
++---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 \r