docs/report/vpp_performance_tests/csit_release_notes.rst

   1 Release Notes
   2 =============
   3
   4 Changes in |csit-release|
   5 -------------------------
   6
   7 #. VPP PERFORMANCE TESTS
   8
   9    - **Service density 2n-skx tests**: Added higher NF density tests with
  10      IPSec encencryption between DUTs.
  11
  12    - **AVF tests**: Full test coveraged based on code changes in CSIT core
  13      layer (driver/interface awareness) and generated by suite generator
  14      (Fortville NICs only).
  15
  16    - **Hoststack TCP/IP tests**: Major refactor of Hoststack TCP
  17      performance tests using WRK generator talking to the VPP HTTP
  18      static server plugin measuring connections per second and
  19      requests per second.
  20
  21 #. TEST FRAMEWORK
  22
  23    - **CSIT Python3 Support**: Full migration of CSIT from Python2.7 to
  24      Python3.6. This change includes library migration, PIP dependency upgrade,
  25      CSIT container images, infrastructure packages ugrade/installation.
  26
  27    - **CSIT PAPI Support**: Finished conversion of CSIT VAT L1 keywords
  28      to PAPI L1 KWs in CSIT using VPP Python bindings (VPP PAPI).
  29      Redesign of key components of PAPI Socket Executor and PAPI
  30      history. Due to issues with PAPI performance, VAT is still used
  31      in CSIT for all VPP scale tests. See known issues below.
  32
  33    - **General Code Housekeeping**: Ongoing RF keywords optimizations,
  34      removal of redundant RF keywords and aligning of suite/test
  35      setup/teardowns.
  36
  37
  38 #. PRESENTATION AND ANALYTICS LAYER
  39
  40    - **Graphs Layout Improvements**: Improved performance graphs layout
  41      for better readibility and maintenance: test grouping, axis
  42      labels, descriptions, other informative decoration.
  43
  44 .. raw:: latex
  45
  46     \clearpage
  47
  48 .. _vpp_known_issues:
  49
  50 Known Issues
  51 ------------
  52
  53 List of known issues in |csit-release| for VPP performance tests:
  54
  55 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
  56 | #  | JiraID                                  | Issue Description                                                                                        |
  57 +====+=========================================+==========================================================================================================+
  58 | 1  | `CSIT-570                               | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue.  |
  59 |    | <https://jira.fd.io/browse/CSIT-570>`_  | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds.    |
  60 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
  61 | 2  | `VPP-662                                | 9000B packets not supported by NICs VIC1227 and VIC1387.                                                 |
  62 |    | <https://jira.fd.io/browse/VPP-662>`_   |                                                                                                          |
  63 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
  64 | 3  | `CSIT-1498                              | Memif tests are sporadically failing on initialization of memif connection.                              |
  65 |    | <https://jira.fd.io/browse/CSIT-1498>`_ |                                                                                                          |
  66 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
  67 | 4  | `VPP-1677                               | 9000B ip4 nat44: VPP crash + coredump.                                                                   |
  68 |    | <https://jira.fd.io/browse/VPP-1677>`_  | VPP crashes very often in case that NAT44 is configured and it has to process IP4 jumbo frames (9000B).  |
  69 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
  70 | 5  | `CSIT-1591                              | All CSIT scale tests can not use PAPI due to much slower performance compared to VAT/CLI (it takes much  |
  71 |    | <https://jira.fd.io/browse/CSIT-1499>`_ | longer to program VPP). This needs to be addressed on the PAPI side.                                     |
  72 |    +-----------------------------------------+                                                                                                          |
  73 |    | `VPP-1763                               |                                                                                                          |
  74 |    | <https://jira.fd.io/browse/VPP-1763>`_  |                                                                                                          |
  75 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
  76 | 6  | `VPP-1675                               | IPv4 IPSEC 9000B packet tests are failing as no packet is forwarded.                                     |
  77 |    | <https://jira.fd.io/browse/VPP-1675>`_  | Reason: chained buffers are not supported.                                                               |
  78 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
  79 | 7  | `CSIT-1593                              | IPv4 AVF 9000B packet tests are failing on 3n-skx while passing on 2n-skx.                               |
  80 |    | <https://jira.fd.io/browse/CSIT-1593>`_ |                                                                                                          |
  81 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
  82