docs/report/dpdk_performance_tests/csit_release_notes.rst

   1 Release Notes
   2 =============
   3
   4 Changes in |csit-release|
   5 -------------------------
   6
   7 #. **DPDK performance tests**
   8
   9    - *MRR tests* - Maximum Receive Rate tests measure the packet forwarding rate
  10      under the maximum load offered by traffic generator over a set trial
  11      duration, regardless of packet loss. MRR tests are used for continuous
  12      performance trending and for comparison between releases.
  13
  14    - *MLR tests* - NDR and PDR tests measure the packet forwarding rate using
  15      MLRsearch library by traffic generator. All tests that previously used
  16      binary search were converted to MLRsearch.
  17
  18    - *2-node tests* - Set of 2-node tests covering testpmd and l3fwd.
  19
  20    - Increased coverage of NIC specific tests (Intel-xxv710-da2, Intel-x710)
  21
  22    - *Generated tests* - Simplified and unified test structure,
  23      semi-autogenerated by generator script. Test generator is currently able
  24      to create test combinations with various frame size and cores combinations.
  25      All existing test cases were converted to new format.
  26
  27
  28 Performance Changes
  29 -------------------
  30
  31 Relative performance changes in measured packet throughput in |csit-release|
  32 are calculated against the results from |csit-release-1|
  33 report. Listed mean and standard deviation values are computed based on
  34 a series of the same tests executed against respective DPDK releases to
  35 verify test results repeatability, with percentage change calculated for
  36 mean values.
  37
  38 NDR Changes
  39 ~~~~~~~~~~~
  40
  41 NDR throughput changes between releases are available in a CSV and pretty ASCII
  42 formats:
  43
  44   - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
  45   - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
  46   - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
  47   - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
  48
  49 .. note::
  50
  51     Test results have been generated by
  52     `FD.io test executor dpdk performance job 3n-hsw`_
  53     with Robot Framework result
  54     files csit-dpdk-perf-|srelease|-\*.zip
  55     `archived here <../_static/archive/>`_.
  56
  57 PDR Changes
  58 ~~~~~~~~~~~
  59
  60 PDR throughput changes between releases are available in a CSV and pretty ASCII
  61 formats:
  62
  63   - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
  64   - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
  65   - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
  66   - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
  67
  68 .. note::
  69
  70     Test results have been generated by
  71     `FD.io test executor dpdk performance job 3n-hsw`_
  72     with Robot Framework result
  73     files csit-dpdk-perf-|srelease|-\*.zip
  74     `archived here <../_static/archive/>`_.
  75
  76 MRR Changes
  77 ~~~~~~~~~~~
  78
  79 MRR small packet throughput changes between releases are available in a
  80 CSV and pretty ASCII formats:
  81
  82   - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-mrr.csv>`_,
  83   - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-mrr.csv>`_,
  84   - `csv format for 4t4c <../_static/dpdk/performance-changes-4t4c-mrr.csv>`_,
  85   - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-mrr.txt>`_,
  86   - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-mrr.txt>`_,
  87   - `pretty ASCII format for 4t4c <../_static/dpdk/performance-changes-4t4c-mrr.txt>`_.
  88
  89 .. note::
  90
  91     Test results have been generated by
  92     `FD.io test executor dpdk performance job 3n-hsw`_,
  93     with Robot Framework result
  94     files csit-vpp-perf-|srelease|-\*.zip
  95     `archived here <../_static/archive/>`_.
  96
  97 Comparison Across Testbeds
  98 --------------------------
  99
 100 Relative performance changes in measured packet throughput on 3-Node Skx testbed
 101 are calculated against the results measured on 3-Node Hsw testbed.
 102
 103 NDR Changes
 104 ~~~~~~~~~~~
 105
 106 NDR throughput changes between testbeds are available in a CSV and pretty ASCII
 107 formats:
 108
 109   - `csv format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
 110   - `pretty ASCII format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
 111
 112 .. note::
 113
 114     Test results have been generated by
 115     `FD.io test executor dpdk performance job 3n-hsw`_ and
 116     `FD.io test executor dpdk performance job 3n-skx`_
 117     with Robot Framework result
 118     files csit-dpdk-perf-|srelease|-\*.zip
 119     `archived here <../_static/archive/>`_.
 120
 121 PDR Changes
 122 ~~~~~~~~~~~
 123
 124 PDR throughput changes between testbeds are available in a CSV and pretty ASCII
 125 formats:
 126
 127   - `csv format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
 128   - `pretty ASCII format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
 129
 130 .. note::
 131
 132     Test results have been generated by
 133     `FD.io test executor dpdk performance job 3n-hsw`_ and
 134     `FD.io test executor dpdk performance job 3n-skx`_
 135     with Robot Framework result
 136     files csit-dpdk-perf-|srelease|-\*.zip
 137     `archived here <../_static/archive/>`_.
 138
 139 MRR Changes
 140 ~~~~~~~~~~~
 141
 142 MRR throughput changes between testbeds are available in a
 143 CSV and pretty ASCII formats:
 144
 145   - `csv format for mrr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
 146   - `pretty ASCII format for mrr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
 147
 148 .. note::
 149
 150     Test results have been generated by
 151     `FD.io test executor dpdk performance job 3n-hsw`_ and
 152     `FD.io test executor dpdk performance job 3n-skx`_
 153     with Robot Framework result
 154     files csit-vpp-perf-|srelease|-\*.zip
 155     `archived here <../_static/archive/>`_.
 156
 157 Known Issues
 158 ------------
 159
 160 Here is the list of known issues in |csit-release| for Testpmd performance tests:
 161
 162 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
 163 | # | Issue                                             | Jira ID    | Description                                                     |
 164 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
 165 |   | No known issues                                   |            |                                                                 |
 166 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+