Code Review / csit.git / blobdiff
? search:
summary | shortlog | log | commit | commitdiff | review | tree
raw | inline | side by side
CSIT rls1704 report - small updates
[csit.git] / docs / report / vpp_performance_tests / csit_release_notes.rst
diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 10e01ab..b4fc9de 100644 (file)
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -6,31 +6,36 @@ Changes in CSIT |release|
 \r
 #. VPP performance test environment changes\r
 \r
 \r
 #. VPP performance test environment changes\r
 \r
-    - Further VM and vhost-user test environment optimizations - Qemu virtio\r
+    - Further optimizations of VM and vhost-user test environment - Qemu virtio\r
       queue size increased from default value of 256 to 1024.\r
       queue size increased from default value of 256 to 1024.\r
-    - Addition of HW cryptodev devices in all three LF FD.io physical testbeds.\r
+    - Addition of HW cryptodev devices - Intel QAT 8950 50G - in all three\r
+      LF FD.io physical testbeds.\r
 \r
 \r
-#. Added tests\r
+#. VPP performance test framework changes\r
 \r
 \r
-    - CGNAT\r
+    - Added VAT command history collection for every test case as part of teardown.\r
+\r
+#. Added VPP performance tests\r
+\r
+    - **CGNAT**\r
 \r
       - Carrier Grade Network Address Translation tests with varying number\r
         of users and ports per user: 1u-15p, 10u-15p, 100u-15p, 1000u-15p,\r
         2000u-15p, 4000u-15p - with Intel x520 NIC.\r
 \r
 \r
       - Carrier Grade Network Address Translation tests with varying number\r
         of users and ports per user: 1u-15p, 10u-15p, 100u-15p, 1000u-15p,\r
         2000u-15p, 4000u-15p - with Intel x520 NIC.\r
 \r
-    - vhost-user tests with one VM\r
+    - **vhost-user tests with one VM**\r
 \r
       - L2 Bridge Domain switched-forwarding with Intel x710 NIC, Intel x520 NIC,\r
         Intel xl710 NIC.\r
       - VXLAN and L2 Bridge Domain switched-forwarding with Intel x520 NIC.\r
 \r
 \r
       - L2 Bridge Domain switched-forwarding with Intel x710 NIC, Intel x520 NIC,\r
         Intel xl710 NIC.\r
       - VXLAN and L2 Bridge Domain switched-forwarding with Intel x520 NIC.\r
 \r
-    - vhost-user tests with two VM service chain\r
+    - **vhost-user tests with two VMs service chain**\r
 \r
       - L2 cross-connect switched-forwarding with Intel x520 NIC, Intel xl710 NIC.\r
       - L2 Bridge Domain switched-forwarding with Intel x520 NIC, Intel xl710 NIC.\r
       - IPv4 routed-forwarding with Intel x520 NIC, Intel xl710 NIC.\r
 \r
 \r
       - L2 cross-connect switched-forwarding with Intel x520 NIC, Intel xl710 NIC.\r
       - L2 Bridge Domain switched-forwarding with Intel x520 NIC, Intel xl710 NIC.\r
       - IPv4 routed-forwarding with Intel x520 NIC, Intel xl710 NIC.\r
 \r
-    - IPSec encryption with\r
+    - **IPSec encryption with**\r
 \r
       - AES-GCM, CBC-SHA1 ciphers, in combination with IPv4 routed-forwarding\r
         with Intel xl710 NIC.\r
 \r
       - AES-GCM, CBC-SHA1 ciphers, in combination with IPv4 routed-forwarding\r
         with Intel xl710 NIC.\r
@@ -205,22 +210,21 @@ Here is the list of known issues in CSIT |release| for VPP performance tests:
 |   | for ip4scale200k, ip4scale2m scale IPv4 routed- |            | Observed frequency: all test runs.                              |\r
 |   | forwarding tests. ip4scale20k tests are fine.   |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 |   | for ip4scale200k, ip4scale2m scale IPv4 routed- |            | Observed frequency: all test runs.                              |\r
 |   | forwarding tests. ip4scale20k tests are fine.   |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 2 | VAT API timeouts during ip6scale2m scale IPv6   |            | Needs fixing VPP VAT API timeouts for large volume of IPv6      |\r
-|   | routed-forwarding tests when volume adding IPv6 | VPP-?      | routes.                                                         |\r
+| 2 | VAT API timeouts during ip6scale2m scale IPv6   | VPP-712    | Needs fixing VPP VAT API timeouts for large volume of IPv6      |\r
+|   | routed-forwarding tests when volume adding IPv6 |            | routes.                                                         |\r
 |   | routes - 2M in this case. ip6scale2kk works.    |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 |   | routes - 2M in this case. ip6scale2kk works.    |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 3 | Vic1385 and Vic1227 low performance             | CSIT-?     | Low NDR performance.                                            |\r
-|   |                                                 |            |                                       .                         |\r
+| 3 | Vic1385 and Vic1227 low performance             | VPP-664    | Low NDR performance.                                            |\r
+|   |                                                 |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 4 | Sporadic NDR discovery test failures on x520    | CSIT-?     | Suspected issue with HW settings (BIOS, FW) in LF               |\r
+| 4 | Sporadic NDR discovery test failures on x520    | CSIT-750   | Suspected issue with HW settings (BIOS, FW) in LF               |\r
 |   |                                                 |            | infrastructure. Issue can't be replicated outside LF.           |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 |   |                                                 |            | infrastructure. Issue can't be replicated outside LF.           |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 5 | Testpmd - Non-repeatible zig-zagging NDR        | CSIT-?     | Suspected NIC firmware or driver issue affecting NDR            |\r
-|   | throughput in multi-thread/-core tests          |            | in multi-thread/-core operation. Need to update to latest       |\r
-|   | - 2t2c - for some tested NICs.                  |            | firmware in NICs. Applies to XL710 and X710 NICs.               |\r
+| 5 | VPP in 2t2c setups - large variation            | CSIT-568   | Suspected NIC firmware or DPDK driver issue affecting NDR       |\r
+|   | of discovered NDR throughput values across      |            | throughput. Applies to XL710 and X710 NICs, x520 NICs are fine. |\r
+|   | multiple test runs with xl710 and x710 NICs.    |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 6 | VPP - Non-repeatible zig-zagging NDR            | CSIT-?     | Suspected NIC firmware or driver issue affecting NDR            |\r
-|   | throughput in multi-thread/-core tests          |            | in multi-thread/-core operation. Need to update to latest       |\r
-|   | - 2t2c - for some tested NICs.                  |            | firmware in NICs. Applies to XL710 and X710 NICs.               |\r
+| 6 | Lower than expected NDR and PDR throughput with | CSIT-569   | Suspected NIC firmware or DPDK driver issue affecting NDR and   |\r
+|   | xl710 and x710 NICs, compared to x520 NICs.     |            | PDR throughput. Applies to XL710 and X710 NICs.                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 \r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 \r
Integration tests