docs/report/dpdk_performance_tests/csit_release_notes.rst

   1 Release Notes
   2 =============
   3
   4 Changes in |csit-release|
   5 -------------------------
   6
   7 #. **DPDK performance tests**
   8
   9   - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
  10     configurable trial duration and number of consecutive executions.
  11     Coverage of MRR tests has been extended across more test
  12     scenarios. MRR tests are used for continuous performance trending
  13     and for comparison between DPDK releases.
  14
  15   - **MLRsearch Throughput**: MLRsearch algorithm has been introduced
  16     for all NDR and PDR throughput tests. All tests that previously
  17     used binary search got converted to MLRsearch. Coverage of NDR/PDR
  18     tests has been extended across more test scenarios.
  19
  20   - **2-Node Tests**: A new set of 2-node tests covering DPDK Testpmd
  21     and L3fwd, running on new Xeon Skylake testbeds.
  22
  23   - **More NICs**: Increased coverage of NIC models tested: Intel-
  24     xxv710-da2, Intel-x710-da4.
  25
  26   - **Generated tests**: Simplified and unified test structure, semi-
  27     autogenerated by generator script. Test generator is currently
  28     able to create test combinations with various frame size and
  29     cores combinations. All existing test cases were converted to new
  30     format.
  31
  32 #. **Presentation and Analytics Layer**
  33
  34   - **Performance trending**: Added continuous performance trending with
  35     anomaly detection and analysis for DPDK.
  36
  37 Performance Changes
  38 -------------------
  39
  40 Relative performance changes in measured NDR, PDR and MRR packet
  41 throughput in |csit-release| are calculated against the test results
  42 from |csit-release-1| report, for tests running on 3-Node Intel Xeon
  43 Haswell testbeds (3n-hsw) in 1-core, 2-core and 4-core (MRR only)
  44 configurations.
  45
  46 Listed mean and standard deviation values are computed based on a series
  47 of the same tests executed against respective VPP releases to verify
  48 test results repeatability, with percentage change calculated for mean
  49 values.
  50
  51 NDR Changes
  52 ~~~~~~~~~~~
  53
  54 NDR throughput changes between releases are available in CSV and pretty
  55 ASCII formats:
  56
  57   - `CSV 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
  58   - `CSV 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
  59   - `ASCII 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
  60   - `ASCII 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
  61
  62 .. note::
  63
  64     Test results have been generated by
  65     `FD.io test executor dpdk performance job 3n-hsw`_
  66     with RF result
  67     files csit-dpdk-perf-|srelease|-\*.zip
  68     `archived here <../_static/archive/>`_.
  69
  70 PDR Changes
  71 ~~~~~~~~~~~
  72
  73 PDR throughput changes between releases are available in CSV and pretty
  74 ASCII formats:
  75
  76   - `CSV 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
  77   - `CSV 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
  78   - `ASCII 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
  79   - `ASCII 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
  80
  81 .. note::
  82
  83     Test results have been generated by
  84     `FD.io test executor dpdk performance job 3n-hsw`_
  85     with RF result
  86     files csit-dpdk-perf-|srelease|-\*.zip
  87     `archived here <../_static/archive/>`_.
  88
  89 MRR Changes
  90 ~~~~~~~~~~~
  91
  92 MRR throughput changes between releases are available in CSV and pretty
  93 ASCII formats:
  94
  95   - `CSV 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.csv>`_,
  96   - `CSV 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.csv>`_,
  97   - `CSV 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.csv>`_,
  98   - `ASCII 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.txt>`_,
  99   - `ASCII 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.txt>`_,
 100   - `ASCII 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.txt>`_.
 101
 102 .. note::
 103
 104     Test results have been generated by
 105     `FD.io test executor dpdk performance job 3n-hsw`_,
 106     with RF result
 107     files csit-vpp-perf-|srelease|-\*.zip
 108     `archived here <../_static/archive/>`_.
 109
 110 Skx vs. Hsw Comparison
 111 ----------------------
 112
 113 Relative performance changes in measured packet throughput on 3-Node Skx testbed
 114 are calculated against the results measured on 3-Node Hsw testbed.
 115
 116 NDR Comparison
 117 ~~~~~~~~~~~~~~
 118
 119 NDR throughput changes between testbeds are available in a CSV and pretty ASCII
 120 formats:
 121
 122   - `CSV 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
 123   - `ASCII 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
 124
 125 .. note::
 126
 127     Test results have been generated by
 128     `FD.io test executor dpdk performance job 3n-hsw`_ and
 129     `FD.io test executor dpdk performance job 3n-skx`_
 130     with RF result
 131     files csit-dpdk-perf-|srelease|-\*.zip
 132     `archived here <../_static/archive/>`_.
 133
 134 PDR Comparison
 135 ~~~~~~~~~~~~~~
 136
 137 PDR throughput changes between testbeds are available in a CSV and pretty ASCII
 138 formats:
 139
 140   - `CSV 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
 141   - `ASCII 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
 142
 143 .. note::
 144
 145     Test results have been generated by
 146     `FD.io test executor dpdk performance job 3n-hsw`_ and
 147     `FD.io test executor dpdk performance job 3n-skx`_
 148     with RF result
 149     files csit-dpdk-perf-|srelease|-\*.zip
 150     `archived here <../_static/archive/>`_.
 151
 152 MRR Comparison
 153 ~~~~~~~~~~~~~~
 154
 155 MRR throughput changes between testbeds are available in a
 156 CSV and pretty ASCII formats:
 157
 158   - `CSV 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
 159   - `ASCII 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
 160
 161 .. note::
 162
 163     Test results have been generated by
 164     `FD.io test executor dpdk performance job 3n-hsw`_ and
 165     `FD.io test executor dpdk performance job 3n-skx`_
 166     with RF result
 167     files csit-vpp-perf-|srelease|-\*.zip
 168     `archived here <../_static/archive/>`_.
 169
 170 Known Issues
 171 ------------
 172
 173 Here is the list of known issues in |csit-release| for Testpmd performance tests:
 174
 175 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
 176 | # | Issue                                             | Jira ID    | Description                                                     |
 177 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
 178 |   | No known issues                                   |            |                                                                 |
 179 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+