docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst

   1
   2 .. _vpp_compare_current_vs_previous_release:
   3
   4 Current vs. Previous Release
   5 ----------------------------
   6
   7 Relative comparison of VPP packet throughput (NDR, PDR and MRR) between
   8 |vpp-release| and |vpp-release-1| (measured for |csit-release| and
   9 |csit-release-1| respectively) is calculated from results of tests
  10 running on 3-Node Intel Xeon Haswell testbeds (3n-hsw) in 1-core, 2-core
  11 and 4-core (MRR only) configurations.
  12
  13 Listed mean and standard deviation values are computed based on a series
  14 of the same tests executed against respective VPP releases to verify
  15 test results repeatability, with percentage change calculated for mean
  16 values. Note that the standard deviation is quite high for a small
  17 number of packet throughput tests, what indicates poor test results
  18 repeatability and makes the relative change of mean throughput value not
  19 fully representative for these tests. The root causes behind poor
  20 results repeatability vary between the test cases.
  21
  22 .. note::
  23
  24     Test results have been generated by
  25
  26     - `FD.io test executor vpp performance job 3n-hsw`_,
  27     - `FD.io test executor vpp performance job 3n-skx`_,
  28     - `FD.io test executor vpp performance job 2n-skx`_
  29
  30     with RF result files csit-vpp-perf-|srelease|-\*.zip
  31     `archived here <../../_static/archive/>`_.
  32
  33 2n-skx
  34 ~~~~~~
  35
  36 NDR Comparison
  37 ``````````````
  38
  39 Comparison tables in HTML, ASCII and CSV formats:
  40
  41   - `HTML 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-ndr.html>`_
  42   - `HTML 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-ndr.html>`_
  43   - `ASCII 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-ndr.txt>`_
  44   - `ASCII 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-ndr.txt>`_
  45   - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-ndr.csv>`_
  46   - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-ndr.csv>`_
  47
  48 PDR Comparison
  49 ``````````````
  50
  51 Comparison tables in HTML, ASCII and CSV formats:
  52
  53   - `HTML 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-pdr.html>`_
  54   - `HTML 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-pdr.html>`_
  55   - `ASCII 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-pdr.txt>`_
  56   - `ASCII 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-pdr.txt>`_
  57   - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-pdr.csv>`_
  58   - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-pdr.csv>`_
  59
  60 MRR Comparison
  61 ``````````````
  62
  63 Comparison tables in HTML, ASCII and CSV formats:
  64
  65   - `HTML 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-mrr.html>`_
  66   - `HTML 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr.html>`_
  67   - `HTML 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr.html>`_
  68   - `ASCII 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-mrr.txt>`_
  69   - `ASCII 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr.txt>`_
  70   - `ASCII 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr.txt>`_
  71   - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-mrr.csv>`_
  72   - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr.csv>`_
  73   - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr.csv>`_
  74
  75 3n-skx
  76 ~~~~~~
  77
  78 NDR Comparison
  79 ``````````````
  80
  81 Comparison tables in HTML, ASCII and CSV formats:
  82
  83   - `HTML 2t1c NDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-ndr.html>`_
  84   - `HTML 4t2c NDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-ndr.html>`_
  85   - `ASCII 2t1c NDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-ndr.txt>`_
  86   - `ASCII 4t2c NDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-ndr.txt>`_
  87   - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-ndr.csv>`_
  88   - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-ndr.csv>`_
  89
  90 PDR Comparison
  91 ``````````````
  92
  93 Comparison tables in HTML, ASCII and CSV formats:
  94
  95   - `HTML 2t1c PDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-pdr.html>`_
  96   - `HTML 4t2c PDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-pdr.html>`_
  97   - `ASCII 2t1c PDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-pdr.txt>`_
  98   - `ASCII 4t2c PDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-pdr.txt>`_
  99   - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-pdr.csv>`_
 100   - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-pdr.csv>`_
 101
 102 MRR Comparison
 103 ``````````````
 104
 105 Comparison tables in HTML, ASCII and CSV formats:
 106
 107   - `HTML 2t1c MRR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-mrr.html>`_
 108   - `HTML 4t2c MRR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-mrr.html>`_
 109   - `HTML 8t4c MRR comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-mrr.html>`_
 110   - `ASCII 2t1c MRR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-mrr.txt>`_
 111   - `ASCII 4t2c MRR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-mrr.txt>`_
 112   - `ASCII 8t4c MRR comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-mrr.txt>`_
 113   - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-mrr.csv>`_
 114   - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-mrr.csv>`_
 115   - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-mrr.csv>`_
 116
 117 3n-hsw
 118 ~~~~~~
 119
 120 NDR Comparison
 121 ``````````````
 122
 123 Comparison tables in HTML, ASCII and CSV formats:
 124
 125   - `HTML 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-ndr.html>`_
 126   - `HTML 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-ndr.html>`_
 127   - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-ndr.txt>`_
 128   - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-ndr.txt>`_
 129   - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-ndr.csv>`_
 130   - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-ndr.csv>`_
 131
 132 PDR Comparison
 133 ``````````````
 134
 135 Comparison tables in HTML, ASCII and CSV formats:
 136
 137   - `HTML 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-pdr.html>`_
 138   - `HTML 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-pdr.html>`_
 139   - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-pdr.txt>`_
 140   - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-pdr.txt>`_
 141   - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-pdr.csv>`_
 142   - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-pdr.csv>`_
 143
 144 MRR Comparison
 145 ``````````````
 146
 147 Comparison tables in HTML, ASCII and CSV formats:
 148
 149   - `HTML 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-mrr.html>`_
 150   - `HTML 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-mrr.html>`_
 151   - `HTML 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-mrr.html>`_
 152   - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-mrr.txt>`_
 153   - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-mrr.txt>`_
 154   - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-mrr.txt>`_
 155   - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-mrr.csv>`_
 156   - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-mrr.csv>`_
 157   - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-mrr.csv>`_
 158
 159 ..
 160     3n-tsh
 161     ~~~~~~
 162
 163     NDR Comparison
 164     ``````````````
 165
 166     Comparison tables in HTML, ASCII and CSV formats:
 167
 168       - `HTML 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr.html>`_
 169       - `HTML 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr.html>`_
 170       - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr.txt>`_
 171       - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr.txt>`_
 172       - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr.csv>`_
 173       - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr.csv>`_
 174
 175     PDR Comparison
 176     ``````````````
 177
 178     Comparison tables in HTML, ASCII and CSV formats:
 179
 180       - `HTML 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr.html>`_
 181       - `HTML 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr.html>`_
 182       - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr.txt>`_
 183       - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr.txt>`_
 184       - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr.csv>`_
 185       - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr.csv>`_
 186
 187     MRR Comparison
 188     ``````````````
 189
 190     Comparison tables in HTML, ASCII and CSV formats:
 191
 192       - `HTML 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr.html>`_
 193       - `HTML 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr.html>`_
 194       - `HTML 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr.html>`_
 195       - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr.txt>`_
 196       - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr.txt>`_
 197       - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr.txt>`_
 198       - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr.csv>`_
 199       - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr.csv>`_
 200       - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr.csv>`_
 201
 202     2n-dnv
 203     ~~~~~~
 204
 205     NDR Comparison
 206     ``````````````
 207
 208     Comparison tables in HTML, ASCII and CSV formats:
 209
 210       - `HTML 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.html>`_
 211       - `HTML 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.html>`_
 212       - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.txt>`_
 213       - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.txt>`_
 214       - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.csv>`_
 215       - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.csv>`_
 216
 217     PDR Comparison
 218     ``````````````
 219
 220     Comparison tables in HTML, ASCII and CSV formats:
 221
 222       - `HTML 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.html>`_
 223       - `HTML 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.html>`_
 224       - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.txt>`_
 225       - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.txt>`_
 226       - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.csv>`_
 227       - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.csv>`_
 228
 229     MRR Comparison
 230     ``````````````
 231
 232     Comparison tables in HTML, ASCII and CSV formats:
 233
 234       - `HTML 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.html>`_
 235       - `HTML 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.html>`_
 236       - `HTML 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.html>`_
 237       - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.txt>`_
 238       - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.txt>`_
 239       - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.txt>`_
 240       - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.csv>`_
 241       - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.csv>`_
 242       - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.csv>`_
 243
 244     3n-dnv
 245     ~~~~~~
 246
 247     NDR Comparison
 248     ``````````````
 249
 250     Comparison tables in HTML, ASCII and CSV formats:
 251
 252       - `HTML 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr.html>`_
 253       - `HTML 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr.html>`_
 254       - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr.txt>`_
 255       - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr.txt>`_
 256       - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr.csv>`_
 257       - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr.csv>`_
 258
 259     PDR Comparison
 260     ``````````````
 261
 262     Comparison tables in HTML, ASCII and CSV formats:
 263
 264       - `HTML 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr.html>`_
 265       - `HTML 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr.html>`_
 266       - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr.txt>`_
 267       - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr.txt>`_
 268       - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr.csv>`_
 269       - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr.csv>`_
 270
 271     MRR Comparison
 272     ``````````````
 273
 274     Comparison tables in HTML, ASCII and CSV formats:
 275
 276       - `HTML 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr.html>`_
 277       - `HTML 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr.html>`_
 278       - `HTML 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr.html>`_
 279       - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr.txt>`_
 280       - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr.txt>`_
 281       - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr.txt>`_
 282       - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr.csv>`_
 283       - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr.csv>`_
 284       - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr.csv>`_