Report: Configure report 2001.20
[csit.git] / docs / report / introduction / report_history.rst
index 7066b35..6e9708f 100644 (file)
-FD.io CSIT-18.10 Test Report History
-====================================
+Report History
+==============
 
-+---------+--------------------------------------------------------------------+
-| Version | Change                                                             |
-+=========+====================================================================+
-| .w47    | 1. Automatic wrapping of long test names in graphs added.          |
-|         | 2. Data and time format in the header changed.                     |
-|         | 3. Versioning changed.                                             |
-|         | 4. Added more test runs:                                           |
-|         |                                                                    |
-|         |    a. HoneyComb Functional,                                        |
-|         |    b. VPP on 3n-hsw testbed.                                       |
-|         |                                                                    |
-|         | 5. 3n-skx and 2n-skx comparisons added to Report:                  |
-|         |                                                                    |
-|         |    a. VPP Performance Tests:                                       |
-|         |       :ref:`vpp_compare_current_vs_previous_release`               |
-|         |    b. DPDK Performance Tests:                                      |
-|         |       :ref:`dpdk_compare_current_vs_previous_release`              |
-|         |                                                                    |
-+---------+--------------------------------------------------------------------+
-| .w46    | 1. dot1q KVM VMs vhost-user tests added to                         |
-|         |    :ref:`KVM_VMs_vhost`.                                           |
-|         |                                                                    |
-|         | 2. Added number of test runs used to generate data for all graphs  |
-|         |    (throughput, speedup multi-core and latency).                   |
-|         |                                                                    |
-|         | 3. Added more test runs:                                           |
-|         |                                                                    |
-|         |    a. K8s Container Memif,                                         |
-|         |    b. VPP on 3n-hsw testbed.                                       |
-|         |                                                                    |
-+---------+--------------------------------------------------------------------+
-| .w45    | Initial version                                                    |
-+---------+--------------------------------------------------------------------+
+FD.io CSIT-2001 Report history and per .[ww] revision changes are listed below.
+
++----------------+------------------------------------------------------------+
+| .[ww] Revision | Changes                                                    |
++================+============================================================+
+<<<<<<< HEAD   (c842ea Report: Add data)
+<<<<<<< HEAD   (953d13 Report: Fix typo in specification.)
+<<<<<<< HEAD   (793b6b Report: Add data)
+<<<<<<< HEAD   (b95753 PAL: Fix table data model)
+<<<<<<< HEAD   (ede849 Report: Add pdf version)
+<<<<<<< HEAD   (35a360 Report: Add data)
+=======
+<<<<<<< HEAD   (ef4187 Report: Add pdf version)
+>>>>>>> CHANGE (68013a Report: Configure report 2001.15)
+| .13            | 1. Added data:                                             |
+=======
+<<<<<<< HEAD   (ede849 Report: Add pdf version)
+<<<<<<< HEAD   (d4f9e9 CSIT-1597 API cleanup: acl)
+| .12            | 1. Added data:                                             |
+=======
+=======
+<<<<<<< HEAD   (9b9bbb PAL: Fix table data model)
+>>>>>>> CHANGE (8f2816 Report: Configure report 2001.16)
+| .14            | 1. Added data:                                             |
+<<<<<<< HEAD   (b95753 PAL: Fix table data model)
+>>>>>>> CHANGE (0ed80e Report: COnfigure reoirt 2001.14)
+>>>>>>> CHANGE (bdfc93 Report: COnfigure reoirt 2001.14)
+=======
+| .15            | 1. Added data:                                             |
+>>>>>>> CHANGE (8e601d Report: Configure report 2001.15)
+>>>>>>> CHANGE (68013a Report: Configure report 2001.15)
+=======
+=======
+| .16            | 1. Added data:                                             |
+>>>>>>> CHANGE (42a491 Report: Configure report 2001.16)
+>>>>>>> CHANGE (8f2816 Report: Configure report 2001.16)
+=======
+=======
+<<<<<<< HEAD   (d5fe67 Report: Add data)
+>>>>>>> CHANGE (6df891 Report: Configure report 2001.20)
+| .18            | 1. Added data:                                             |
+<<<<<<< HEAD   (c842ea Report: Add data)
+>>>>>>> CHANGE (dfd4a2 Report: Configure Report 2001.18)
+=======
+| .19            | 1. Added data:                                             |
+>>>>>>> CHANGE (413f1d Report: Configure report 2001.19)
+=======
+=======
+| .20            | 1. Added data:                                             |
+>>>>>>> CHANGE (f4c460 Report: Configure report 2001.20)
+>>>>>>> CHANGE (6df891 Report: Configure report 2001.20)
+|                |                                                            |
+|                |    - VPP performance tests 2n-skx                          |
+|                |                                                            |
+|                |      - all set selected for analysis and graphs            |
+|                |      - MRR tests                                           |
+|                |                                                            |
+|                |    - VPP performance tests 3n-skx                          |
+|                |                                                            |
+|                |      - all set selected for analysis and graphs            |
+|                |      - MRR tests                                           |
+|                |                                                            |
+<<<<<<< HEAD   (c842ea Report: Add data)
+<<<<<<< HEAD   (bd7b2e Report: Comparison tables)
+=======
+=======
+|                |      - test sets: crypto, L2, IPv6, memif                  |
+|                |                                                            |
+>>>>>>> CHANGE (6df891 Report: Configure report 2001.20)
+|                |    - Hoststack 3n-hsw tests                                |
+|                |                                                            |
+|                |    - Reconfiguration Tests 2n-skx                          |
+|                |                                                            |
+|                |    - Soak Tests 2n-skx                                     |
+|                |                                                            |
+>>>>>>> CHANGE (b1af33 Report: Add 2n-skx Soak and Reconf tests)
+|                |  2. Added Root Cause Analysis for VPP performance tests    |
+|                |     2n-skx and 3n-skx.                                     |
+|                |                                                            |
++----------------+------------------------------------------------------------+
+<<<<<<< HEAD   (c842ea Report: Add data)
+<<<<<<< HEAD   (953d13 Report: Fix typo in specification.)
+<<<<<<< HEAD   (793b6b Report: Add data)
+<<<<<<< HEAD   (b95753 PAL: Fix table data model)
+<<<<<<< HEAD   (ede849 Report: Add pdf version)
+<<<<<<< HEAD   (35a360 Report: Add data)
+=======
+<<<<<<< HEAD   (ef4187 Report: Add pdf version)
+=======
+=======
+=======
+| .18            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+>>>>>>> CHANGE (413f1d Report: Configure report 2001.19)
+=======
+<<<<<<< HEAD   (d5fe67 Report: Add data)
+=======
+| .19            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+| .18            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+>>>>>>> CHANGE (f4c460 Report: Configure report 2001.20)
+>>>>>>> CHANGE (6df891 Report: Configure report 2001.20)
+| .17            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+| .16            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+| .15            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+>>>>>>> CHANGE (dfd4a2 Report: Configure Report 2001.18)
+| .14            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+| .13            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+>>>>>>> CHANGE (8e601d Report: Configure report 2001.15)
+>>>>>>> CHANGE (68013a Report: Configure report 2001.15)
+| .12            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+=======
+<<<<<<< HEAD   (d4f9e9 CSIT-1597 API cleanup: acl)
+=======
+=======
+<<<<<<< HEAD   (9b9bbb PAL: Fix table data model)
+=======
+| .15            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+| .14            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+>>>>>>> CHANGE (42a491 Report: Configure report 2001.16)
+>>>>>>> CHANGE (8f2816 Report: Configure report 2001.16)
+| .13            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+| .12            | Skipped.                                                   |
++----------------+------------------------------------------------------------+
+>>>>>>> CHANGE (0ed80e Report: COnfigure reoirt 2001.14)
+>>>>>>> CHANGE (bdfc93 Report: COnfigure reoirt 2001.14)
+| .11            | 1. Added Soak tests                                        |
+|                |                                                            |
+|                | 2. Added data:                                             |
+|                |                                                            |
+|                |    - VPP performance tests 3n-tsh                          |
+|                |                                                            |
+|                |      - all set selected for analysis and graphs            |
+|                |                                                            |
++----------------+------------------------------------------------------------+
+| .10            | 1. Added data:                                             |
+|                |                                                            |
+|                |    - VPP performance tests 3n-hsw                          |
+|                |                                                            |
+|                |      - test sets IP4, IPSec                                |
+|                |      - sets selected for analysis and graphs               |
+|                |                                                            |
+|                |    - VPP performance tests 2n-clx                          |
+|                |                                                            |
+|                |      - test sets IP4, IP6, L2, Memif, loadbalancer, VHost  |
+|                |        VTS                                                 |
+|                |      - all sets selected for analysis and graphs           |
+|                |                                                            |
+|                | 2. Added reconfiguration tests:                            |
+|                |                                                            |
+|                |    - 2n-clx                                                |
+|                |                                                            |
+|                | 3. Added Hoststack tests                                   |
+|                |                                                            |
+|                | 4. Edited Test Methodology --> Hoststack Testing           |
+|                |                                                            |
+|                | 5. Added TCP/IP tests                                      |
+|                |                                                            |
++----------------+------------------------------------------------------------+
+| .09            | 1. Added data:                                             |
+|                |                                                            |
+|                |    - VPP performance tests 3n-hsw                          |
+|                |                                                            |
+|                |      - test sets IP4, L2, VHost                            |
+|                |                                                            |
+|                |    - VPP performance tests 2n-clx                          |
+|                |                                                            |
+|                |      - test sets IP4, IP6, L2                              |
+|                |      - all sets selected for analysis and graphs           |
+|                |                                                            |
+|                |    - VPP performance tests 3n-dnv                          |
+|                |                                                            |
+|                |      - test sets IPSec, IP4 Tunnels                        |
+|                |                                                            |
+|                |    - VPP performance tests 3n-tsh                          |
+|                |                                                            |
+|                |      - all set selected for analysis and graphs            |
+|                |                                                            |
+|                |    - VPP MRR tests 2n-clx                                  |
+|                |                                                            |
+|                |      - all sets selected for analysis and graphs           |
+|                |                                                            |
+|                |    - DPDK performance tests 2n-clx                         |
+|                |                                                            |
+|                |      - all sets selected for analysis and graphs           |
+|                |                                                            |
++----------------+------------------------------------------------------------+
+| .08            | 1. Added PDF version                                       |
+|                |                                                            |
+|                | 2. Added data:                                             |
+|                |                                                            |
+|                |    - VPP performance tests 3n-hsw                          |
+|                |    - VPP performance tests 3n-tsh                          |
+|                |    - VPP MRR tests 3n-tsh                                  |
+|                |    - DPDK performance tests 3n-tsh                         |
+|                |                                                            |
+|                | 3. Chapters "Detailed Results", "Test Configuration" and   |
+|                |    "Test Operational Data" split into sub-chapters.        |
+|                |                                                            |
++----------------+------------------------------------------------------------+
+| .07            | Initial version                                            |
+|                |                                                            |
++----------------+------------------------------------------------------------+
+
+FD.io CSIT Reports follow CSIT-[yy][mm].[ww] numbering format, with version
+denoted by concatenation of two digit year [yy] and two digit month [mm], and
+maintenance revision identified by two digit calendar week number [ww].