docs/report/vpp_performance_tests/csit_release_notes.rst

   1 Release Notes
   2 =============
   3
   4 Changes in |csit-release|
   5 -------------------------
   6
   7 #. VPP PERFORMANCE TESTS
   8
   9    - CSIT test environment is versioned, see
  10      :ref:`test_environment_versioning`.
  11
  12    - **GENEVE tests**: Added VPP performance tests for GENEVE tunnels.
  13
  14      - See :ref:`geneve_methodology` for more details.
  15
  16    - **NAT44 tests**: Adapted existing and added new tests.
  17
  18      - Refactored NAT44 deterministic mode (nat44det) tests to use separate
  19        det44 vpp plugin and to use the same scheme of inside and outside
  20        addresses and ports, as used in new NAT44 endpoint-dependent mode tests.
  21
  22      - Added new NAT44 endpoint-depended mode uni-directional (nat44ed-udir)
  23        tests that measure packet throughput in one direction with usage of TRex
  24        in stateless mode.
  25
  26      - Added new NAT44 endpoint-dependent mode CPS tests that measure
  27        connections per second with usage of TRex in stateful mode.
  28        UPD packet size is 64 bytes. Size of TCP control packets
  29        is not configurable, please ignore the -64b- part of test names.
  30
  31      - Added new NAT44 endpoint-dependent mode PPS tests that measure
  32        packets per second (control and data together) with usage of TRex
  33        in stateful mode. UPD packet size is 64 bytes. Size of TCP
  34        data packets is governed by the default MSS value, so most data packets
  35        are 1460 bytes long, please ignore the -64b- part of test names.
  36
  37      - See :ref:`nat44_methodology` for more details.
  38
  39    - **IPSec async mode tests**: Added VPP performance tests for async crypto
  40      engine.
  41
  42    - **AMD 2n-tx2 testbed**: New physical testbed type installed in
  43      FD.io CSIT, with VPP and DPDK performance data added to CSIT
  44      trending and this report.
  45
  46 #. TEST FRAMEWORK
  47
  48    - **TRex ASTF**: Improved capability to run TRex in advanced stateful mode.
  49
  50    - **CSIT PAPI support**: Due to issues with PAPI performance, VAT is
  51      still used in CSIT for all VPP scale tests. See known issues below.
  52
  53    - **General Code Housekeeping**: Ongoing code optimizations,
  54      speed ups and bug fixes.
  55
  56 #. PRESENTATION AND ANALYTICS LAYER
  57
  58    - **Graphs improvements**: Updated Packet Latency graphs,
  59      see :ref:`latency_methodology`.
  60
  61 .. raw:: latex
  62
  63     \clearpage
  64
  65 .. _vpp_known_issues:
  66
  67 Known Issues
  68 ------------
  69
  70 List of known issues in |csit-release| for VPP performance tests:
  71
  72 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
  73 | #  | JiraID                                  | Issue Description                                                                                         |
  74 +====+=========================================+===========================================================================================================+
  75 |  1 | `CSIT-570                               | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue.   |
  76 |    | <https://jira.fd.io/browse/CSIT-570>`_  | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds.     |
  77 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
  78 |  2 | `VPP-662                                | 9000B packets not supported by NICs VIC1227 and VIC1387.                                                  |
  79 |    | <https://jira.fd.io/browse/VPP-662>`_   |                                                                                                           |
  80 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
  81 |  3 | `CSIT-1763                              | Adapt ramp-up phase of nat44 tests for different frame sizes.                                             |
  82 |    | <https://jira.fd.io/browse/CSIT-1763>`_ | Currently ramp-up phase rate and duration values are correctly set for tests with 64B frame size.         |
  83 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
  84 |  4 | `CSIT-1671                              | All CSIT scale tests can not use PAPI due to much slower performance compared to VAT/CLI (it takes much   |
  85 |    | <https://jira.fd.io/browse/CSIT-1671>`_ | longer to program VPP). This needs to be addressed on the PAPI side.                                      |
  86 |    +-----------------------------------------+ The usual PAPI library spends too much time parsing arguments, so even with async processing (hundreds of |
  87 |    | `VPP-1763                               | commands in flight over socket), the VPP configuration for large scale tests (millions of messages) takes |
  88 |    | <https://jira.fd.io/browse/VPP-1763>`_  | too long.                                                                                                 |
  89 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
  90 |  5 | `CSIT-1771                              | IPv4 IPSEC 9000B packet tests had been failing when chained buffers were not supported.                   |
  91 |    | <https://jira.fd.io/browse/CSIT-1771>`_ | This has been fixed on VPP side, but CSIT still needs to re-enable jumbo tests.                           |
  92 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
  93 |  6 | `VPP-1934                               | [i40e] Interfaces are not brought up from carrier-down.                                                   |
  94 |    | <https://jira.fd.io/browse/VPP-1934>`_  | In case of i40e -based interface (e.g Intel x700 series NIC) is bound to kernel driver (i40e) and is in   |
  95 |    |                                         | state "no-carrier" (<NO-CARRIER,BROADCAST,MULTICAST,UP>) because previously it was disabled via           |
  96 |    |                                         | "I40E_AQ_PHY_LINK_ENABLED" call, then VPP during initialization of AVF interface is not re-enabling       |
  97 |    |                                         | interface link via i40e driver to up.                                                                     |
  98 |    |                                         | CSIT implemented `workaround for AVF interface <https://gerrit.fd.io/r/c/csit/+/29086>`_ until fixed.     |
  99 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
 100
 101 Root Cause Analysis for Performance Changes
 102 -------------------------------------------
 103
 104 List of RCAs in |csit-release| for VPP performance changes:
 105
 106 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
 107 | #  | JiraID                                  | Issue Description                                                                                         |
 108 +====+=========================================+===========================================================================================================+
 109 |  1 |                                         |                                                                                                           |
 110 |    |                                         |                                                                                                           |
 111 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+