docs/report/dpdk_performance_tests/csit_release_notes.rst

   1 Release Notes
   2 =============
   3
   4 Changes in |csit-release|
   5 -------------------------
   6
   7 No code changes apart from bug fixes.
   8
   9 Performance Changes
  10 -------------------
  11
  12 Relative performance changes in measured packet throughput in |csit-release|
  13 are calculated against the results from |csit-release-1|
  14 report. Listed mean and standard deviation values are computed based on
  15 a series of the same tests executed against respective VPP releases to
  16 verify test results repeatability, with percentage change calculated for
  17 mean values. Note that the standard deviation is quite high for a small
  18 number of packet throughput tests, what indicates poor test results
  19 repeatability and makes the relative change of mean throughput value not
  20 fully representative for these tests. The root causes behind poor
  21 results repeatability vary between the test cases.
  22
  23 NDR Changes
  24 ~~~~~~~~~~~
  25
  26 NDR throughput changes between releases are available in a
  27 CSV and pretty ASCII formats:
  28
  29   - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
  30   - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
  31   - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
  32   - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
  33
  34 .. note::
  35
  36     Test results have been generated by
  37     `FD.io test executor dpdk performance job 3n-hsw`_
  38     with Robot Framework result
  39     files csit-vpp-perf-|srelease|-\*.zip
  40     `archived here <../_static/archive/>`_.
  41
  42 PDR Changes
  43 ~~~~~~~~~~~
  44
  45 PDR throughput changes between releases are available in a
  46 CSV and pretty ASCII formats:
  47
  48   - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
  49   - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
  50   - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
  51   - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
  52
  53 .. note::
  54
  55     Test results have been generated by
  56     `FD.io test executor dpdk performance job 3n-hsw`_
  57     with Robot Framework result
  58     files csit-vpp-perf-|srelease|-\*.zip
  59     `archived here <../_static/archive/>`_.
  60
  61 Comparison Across Testbeds
  62 --------------------------
  63
  64 Relative performance changes in measured packet throughputon 3-Node Skx testbed
  65 are calculated against the results measured on 3-Node Hsw testbed.
  66
  67 NDR Changes
  68 ~~~~~~~~~~~
  69
  70 NDR changes between testbeds are available in a
  71 CSV and pretty ASCII formats:
  72
  73   - `csv format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
  74   - `pretty ASCII format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
  75
  76 .. note::
  77
  78     Test results have been generated by
  79     `FD.io test executor dpdk performance job 3n-hsw`_ and
  80     `FD.io test executor dpdk performance job 3n-skx`_
  81     with Robot Framework result
  82     files csit-vpp-perf-|srelease|-\*.zip
  83     `archived here <../_static/archive/>`_.
  84
  85 PDR Changes
  86 ~~~~~~~~~~~
  87
  88 PDR throughput changes between testbeds are available in a
  89 CSV and pretty ASCII formats:
  90
  91   - `csv format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
  92   - `pretty ASCII format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
  93
  94 .. note::
  95
  96     Test results have been generated by
  97     `FD.io test executor dpdk performance job 3n-hsw`_ and
  98     `FD.io test executor dpdk performance job 3n-skx`_
  99     with Robot Framework result
 100     files csit-vpp-perf-|srelease|-\*.zip
 101     `archived here <../_static/archive/>`_.
 102
 103 Known Issues
 104 ------------
 105
 106 Here is the list of known issues in |csit-release| for Testpmd performance tests:
 107
 108 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
 109 | # | Issue                                             | Jira ID    | Description                                                     |
 110 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
 111 | 1 | Testpmd in 1t1c and 2t2c setups - large variation | CSIT-569   | Suspected NIC firmware or DPDK driver issue affecting NDR       |
 112 |   | of discovered NDR throughput values across        |            | throughput. Applies to XL710 and X710 NICs, no issues observed  |
 113 |   | multiple test runs with xl710 and x710 NICs.      |            | on x520 NICs.                                                   |
 114 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
 115 | 2 | Lower than expected NDR throughput with xl710     | CSIT-571   | Suspected NIC firmware or DPDK driver issue affecting NDR       |
 116 |   | and x710 NICs, compared to x520 NICs.             |            | throughput. Applies to XL710 and X710 NICs.                     |
 117 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+