docs/report/dpdk_performance_tests/csit_release_notes.rst

   1 Release Notes
   2 =============
   3
   4 Changes in |csit-release|
   5 -------------------------
   6
   7 #. **DPDK release version change**
   8
   9    - |csit-release| tested DPDK-18.05, as used by |vpp-release|.
  10
  11 #. **DPDK performance tests**
  12
  13    - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
  14      configurable trial duration and number of consecutive executions.
  15      Coverage of MRR tests has been extended across more test
  16      scenarios. MRR tests are used for continuous performance trending
  17      and for comparison between DPDK releases.
  18
  19    - **MLRsearch Throughput**: MLRsearch algorithm has been introduced
  20      for all NDR and PDR throughput tests. All tests that previously
  21      used binary search got converted to MLRsearch. Coverage of NDR/PDR
  22      tests has been extended across more test scenarios.
  23
  24    - **2-Node Tests**: A new set of 2-node tests covering DPDK Testpmd
  25      and L3fwd, running on new Xeon Skylake testbeds.
  26
  27    - **More NICs**: Increased coverage of NIC models tested: Intel-
  28      xxv710-da2, Intel-x710-da4.
  29
  30    - **Generated tests**: Simplified and unified test structure, semi-
  31      autogenerated by generator script. Test generator is currently
  32      able to create test combinations with various frame size and
  33      cores combinations. All existing test cases were converted to new
  34      format.
  35
  36 #. **Presentation and Analytics Layer**
  37
  38    - **Performance trending**: Added continuous performance trending with
  39      anomaly detection and analysis for DPDK.
  40
  41 Performance Changes
  42 -------------------
  43
  44 Relative performance changes in measured NDR, PDR and MRR packet
  45 throughput in |csit-release| are calculated against the test results
  46 from |csit-release-1| report, for tests running on 3-Node Intel Xeon
  47 Haswell testbeds (3n-hsw) in 1-core, 2-core and 4-core (MRR only)
  48 configurations.
  49
  50 Listed mean and standard deviation values are computed based on a series
  51 of the same tests executed against respective VPP releases to verify
  52 test results repeatability, with percentage change calculated for mean
  53 values.
  54
  55 NDR Changes
  56 ~~~~~~~~~~~
  57
  58 NDR throughput changes between releases are available in CSV and pretty
  59 ASCII formats:
  60
  61   - `CSV 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
  62   - `CSV 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
  63   - `ASCII 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
  64   - `ASCII 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
  65
  66 .. note::
  67
  68     Test results have been generated by
  69     `FD.io test executor dpdk performance job 3n-hsw`_
  70     with RF result
  71     files csit-dpdk-perf-|srelease|-\*.zip
  72     `archived here <../_static/archive/>`_.
  73
  74 PDR Changes
  75 ~~~~~~~~~~~
  76
  77 PDR throughput changes between releases are available in CSV and pretty
  78 ASCII formats:
  79
  80   - `CSV 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
  81   - `CSV 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
  82   - `ASCII 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
  83   - `ASCII 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
  84
  85 .. note::
  86
  87     Test results have been generated by
  88     `FD.io test executor dpdk performance job 3n-hsw`_
  89     with RF result
  90     files csit-dpdk-perf-|srelease|-\*.zip
  91     `archived here <../_static/archive/>`_.
  92
  93 Skx vs. Hsw Comparison
  94 ----------------------
  95
  96 Relative performance changes in measured packet throughput on 3-Node Skx testbed
  97 are calculated against the results measured on 3-Node Hsw testbed.
  98
  99 NDR Comparison
 100 ~~~~~~~~~~~~~~
 101
 102 NDR throughput changes between testbeds are available in a CSV and pretty ASCII
 103 formats:
 104
 105   - `CSV 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
 106   - `ASCII 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
 107
 108 .. note::
 109
 110     Test results have been generated by
 111     `FD.io test executor dpdk performance job 3n-hsw`_ and
 112     `FD.io test executor dpdk performance job 3n-skx`_
 113     with RF result
 114     files csit-dpdk-perf-|srelease|-\*.zip
 115     `archived here <../_static/archive/>`_.
 116
 117 PDR Comparison
 118 ~~~~~~~~~~~~~~
 119
 120 PDR throughput changes between testbeds are available in a CSV and pretty ASCII
 121 formats:
 122
 123   - `CSV 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
 124   - `ASCII 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
 125
 126 .. note::
 127
 128     Test results have been generated by
 129     `FD.io test executor dpdk performance job 3n-hsw`_ and
 130     `FD.io test executor dpdk performance job 3n-skx`_
 131     with RF result
 132     files csit-dpdk-perf-|srelease|-\*.zip
 133     `archived here <../_static/archive/>`_.
 134
 135 MRR Comparison
 136 ~~~~~~~~~~~~~~
 137
 138 MRR throughput changes between testbeds are available in a
 139 CSV and pretty ASCII formats:
 140
 141   - `CSV 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
 142   - `ASCII 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
 143
 144 .. note::
 145
 146     Test results have been generated by
 147     `FD.io test executor dpdk performance job 3n-hsw`_ and
 148     `FD.io test executor dpdk performance job 3n-skx`_
 149     with RF result
 150     files csit-vpp-perf-|srelease|-\*.zip
 151     `archived here <../_static/archive/>`_.
 152
 153 Known Issues
 154 ------------
 155
 156 Here is the list of known issues in |csit-release| for Testpmd performance tests:
 157
 158 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
 159 | # | Issue                                             | Jira ID    | Description                                                     |
 160 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
 161 |   | No known issues                                   |            |                                                                 |
 162 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+