docs/report/dpdk_performance_tests/csit_release_notes.rst

   1 Release Notes
   2 =============
   3
   4 Changes in |csit-release|
   5 -------------------------
   6
   7 #. **DPDK release version change**
   8
   9    - |csit-release| tested DPDK-18.05, as used by |vpp-release|.
  10
  11 #. **DPDK performance tests**
  12
  13    - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
  14      configurable trial duration and number of consecutive executions.
  15      Coverage of MRR tests has been extended across more test
  16      scenarios. MRR tests are used for continuous performance trending
  17      and for comparison between DPDK releases.
  18
  19    - **MLRsearch Throughput**: MLRsearch algorithm has been introduced
  20      for all NDR and PDR throughput tests. All tests that previously
  21      used binary search got converted to MLRsearch. Coverage of NDR/PDR
  22      tests has been extended across more test scenarios.
  23
  24    - **2-Node Tests**: A new set of 2-node tests covering DPDK Testpmd
  25      and L3fwd, running on new Xeon Skylake testbeds.
  26
  27    - **More NICs**: Increased coverage of NIC models tested: Intel-
  28      xxv710-da2, Intel-x710-da4.
  29
  30    - **Generated tests**: Simplified and unified test structure, semi-
  31      autogenerated by generator script. Test generator is currently
  32      able to create test combinations with various frame size and
  33      cores combinations. All existing test cases were converted to new
  34      format.
  35
  36 #. **Presentation and Analytics Layer**
  37
  38    - **Performance trending**: Added continuous performance trending with
  39      anomaly detection and analysis for DPDK.
  40
  41 Performance Changes
  42 -------------------
  43
  44 Relative performance changes in measured NDR, PDR and MRR packet
  45 throughput in |csit-release| are calculated against the test results
  46 from |csit-release-1| report, for tests running on 3-Node Intel Xeon
  47 Haswell testbeds (3n-hsw) in 1-core, 2-core and 4-core (MRR only)
  48 configurations.
  49
  50 Listed mean and standard deviation values are computed based on a series
  51 of the same tests executed against respective VPP releases to verify
  52 test results repeatability, with percentage change calculated for mean
  53 values.
  54
  55 NDR Changes
  56 ~~~~~~~~~~~
  57
  58 NDR throughput changes between releases are available in CSV and pretty
  59 ASCII formats:
  60
  61   - `CSV 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
  62   - `CSV 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
  63   - `ASCII 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
  64   - `ASCII 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
  65
  66 .. note::
  67
  68     Test results have been generated by
  69     `FD.io test executor dpdk performance job 3n-hsw`_
  70     with RF result
  71     files csit-dpdk-perf-|srelease|-\*.zip
  72     `archived here <../_static/archive/>`_.
  73
  74 PDR Changes
  75 ~~~~~~~~~~~
  76
  77 PDR throughput changes between releases are available in CSV and pretty
  78 ASCII formats:
  79
  80   - `CSV 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
  81   - `CSV 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
  82   - `ASCII 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
  83   - `ASCII 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
  84
  85 .. note::
  86
  87     Test results have been generated by
  88     `FD.io test executor dpdk performance job 3n-hsw`_
  89     with RF result
  90     files csit-dpdk-perf-|srelease|-\*.zip
  91     `archived here <../_static/archive/>`_.
  92
  93 MRR Changes
  94 ~~~~~~~~~~~
  95
  96 MRR throughput changes between releases are available in CSV and pretty
  97 ASCII formats:
  98
  99   - `CSV 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.csv>`_,
 100   - `CSV 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.csv>`_,
 101   - `CSV 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.csv>`_,
 102   - `ASCII 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.txt>`_,
 103   - `ASCII 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.txt>`_,
 104   - `ASCII 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.txt>`_.
 105
 106 .. note::
 107
 108     Test results have been generated by
 109     `FD.io test executor dpdk performance job 3n-hsw`_,
 110     with RF result
 111     files csit-vpp-perf-|srelease|-\*.zip
 112     `archived here <../_static/archive/>`_.
 113
 114 Skx vs. Hsw Comparison
 115 ----------------------
 116
 117 Relative performance changes in measured packet throughput on 3-Node Skx testbed
 118 are calculated against the results measured on 3-Node Hsw testbed.
 119
 120 NDR Comparison
 121 ~~~~~~~~~~~~~~
 122
 123 NDR throughput changes between testbeds are available in a CSV and pretty ASCII
 124 formats:
 125
 126   - `CSV 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
 127   - `ASCII 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
 128
 129 .. note::
 130
 131     Test results have been generated by
 132     `FD.io test executor dpdk performance job 3n-hsw`_ and
 133     `FD.io test executor dpdk performance job 3n-skx`_
 134     with RF result
 135     files csit-dpdk-perf-|srelease|-\*.zip
 136     `archived here <../_static/archive/>`_.
 137
 138 PDR Comparison
 139 ~~~~~~~~~~~~~~
 140
 141 PDR throughput changes between testbeds are available in a CSV and pretty ASCII
 142 formats:
 143
 144   - `CSV 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
 145   - `ASCII 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
 146
 147 .. note::
 148
 149     Test results have been generated by
 150     `FD.io test executor dpdk performance job 3n-hsw`_ and
 151     `FD.io test executor dpdk performance job 3n-skx`_
 152     with RF result
 153     files csit-dpdk-perf-|srelease|-\*.zip
 154     `archived here <../_static/archive/>`_.
 155
 156 MRR Comparison
 157 ~~~~~~~~~~~~~~
 158
 159 MRR throughput changes between testbeds are available in a
 160 CSV and pretty ASCII formats:
 161
 162   - `CSV 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
 163   - `ASCII 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
 164
 165 .. note::
 166
 167     Test results have been generated by
 168     `FD.io test executor dpdk performance job 3n-hsw`_ and
 169     `FD.io test executor dpdk performance job 3n-skx`_
 170     with RF result
 171     files csit-vpp-perf-|srelease|-\*.zip
 172     `archived here <../_static/archive/>`_.
 173
 174 Known Issues
 175 ------------
 176
 177 Here is the list of known issues in |csit-release| for Testpmd performance tests:
 178
 179 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
 180 | # | Issue                                             | Jira ID    | Description                                                     |
 181 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
 182 |   | No known issues                                   |            |                                                                 |
 183 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+