report 18.07: moved performance change/compare sections out of rls notes to separate...
[csit.git] / docs / report / vpp_performance_tests / comparisons / current_vs_previous_release.rst
1 Current vs. Previous Release
2 ----------------------------
3
4 Relative comparison of VPP packet throughput (NDR, PDR and MRR) between
5 |vpp-release| and |vpp-release-1| (measured for |csit-release| and
6 |csit-release-1| respectively) is calculated from results of tests
7 running on 3-Node Intel Xeon Haswell testbeds (3n-hsw) in 1-core, 2-core
8 and 4-core (MRR only) configurations.
9
10 Listed mean and standard deviation values are computed based on a series
11 of the same tests executed against respective VPP releases to verify
12 test results repeatability, with percentage change calculated for mean
13 values. Note that the standard deviation is quite high for a small
14 number of packet throughput tests, what indicates poor test results
15 repeatability and makes the relative change of mean throughput value not
16 fully representative for these tests. The root causes behind poor
17 results repeatability vary between the test cases.
18
19 NDR Comparison
20 ~~~~~~~~~~~~~~
21
22 NDR throughput comparison between releases is available in CSV and pretty
23 ASCII formats:
24
25   - `CSV 1t1c NDR changes <../_static/vpp/performance-changes-1t1c-ndr.csv>`_,
26   - `CSV 2t2c NDR changes <../_static/vpp/performance-changes-2t2c-ndr.csv>`_,
27   - `ASCII 1t1c NDR changes <../_static/vpp/performance-changes-1t1c-ndr.txt>`_,
28   - `ASCII 2t2c NDR changes <../_static/vpp/performance-changes-2t2c-ndr.txt>`_.
29
30 .. note::
31
32     Test results have been generated by
33     `FD.io test executor vpp performance job 3n-hsw`_,
34     with RF result
35     files csit-vpp-perf-|srelease|-\*.zip
36     `archived here <../_static/archive/>`_.
37
38 PDR Comparison
39 ~~~~~~~~~~~~~~
40
41 PDR throughput comparison between releases is available in CSV and pretty
42 ASCII formats:
43
44   - `CSV 1t1c PDR changes <../_static/vpp/performance-changes-1t1c-pdr.csv>`_,
45   - `CSV 2t2c PDR changes <../_static/vpp/performance-changes-2t2c-pdr.csv>`_,
46   - `ASCII 1t1c PDR changes <../_static/vpp/performance-changes-1t1c-pdr.txt>`_,
47   - `ASCII 2t2c PDR changes <../_static/vpp/performance-changes-2t2c-pdr.txt>`_.
48
49 .. note::
50
51     Test results have been generated by
52     `FD.io test executor vpp performance job 3n-hsw`_,
53     with RF result
54     files csit-vpp-perf-|srelease|-\*.zip
55     `archived here <../_static/archive/>`_.
56
57 MRR Comparison
58 ~~~~~~~~~~~~~~
59
60 MRR throughput comparison between releases is available in CSV and pretty
61 ASCII formats:
62
63   - `CSV 1t1c MRR changes <../_static/vpp/performance-changes-1t1c-mrr.csv>`_,
64   - `CSV 2t2c MRR changes <../_static/vpp/performance-changes-2t2c-mrr.csv>`_,
65   - `CSV 4t4c MRR changes <../_static/vpp/performance-changes-4t4c-mrr.csv>`_,
66   - `ASCII 1t1c MRR changes <../_static/vpp/performance-changes-1t1c-mrr.txt>`_,
67   - `ASCII 2t2c MRR changes <../_static/vpp/performance-changes-2t2c-mrr.txt>`_,
68   - `ASCII 4t4c MRR changes <../_static/vpp/performance-changes-4t4c-mrr.txt>`_.
69
70 .. note::
71
72     Test results have been generated by
73     `FD.io test executor vpp performance job 3n-hsw`_,
74     with RF result
75     files csit-vpp-perf-|srelease|-\*.zip
76     `archived here <../_static/archive/>`_.

©2016 FD.io a Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation. Linux is a registered trademark of Linus Torvalds.
Please see our privacy policy and terms of use.