CSIT-1126: Detection and reporting of failed MRR tests

[csit.git] / docs / cpta / introduction / index.rst

diff --git a/docs/cpta/introduction/index.rst b/docs/cpta/introduction/index.rst
index 016037b..991181a 100644 (file)
--- a/docs/cpta/introduction/index.rst
+++ b/docs/cpta/introduction/index.rst
@@ -20,18 +20,40 @@ executed twice a day (target start: every 12 hrs, 02:00, 14:00 UTC). All
 trend and anomaly evaluation is based on a rolling window of <N=14> data
 samples, covering last 7 days.
 
 trend and anomaly evaluation is based on a rolling window of <N=14> data
 samples, covering last 7 days.
 

-Legend to table:
+Failed tests
+------------

 
 

-    - **Test Case** : name of FD.io CSIT test case, naming convention
+The table lists the tests which failed over the <N=14> runs of the trending
+jobs.
+
+Legend to the table:
+
+    - **Test Case**: name of FD.io CSIT test case, naming convention
+      `here <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
+    - **Fails [#]**: number of fails of the tests over the period.
+    - **Last Fail [Date]**: the date and time when the test failed the last
+      time.
+    - **Last Fail [VPP Build]**: VPP build which was tested when the test failed
+      the last time.
+    - **Last Fail [CSIT Build]**: the last CSIT build where the test failed.
+
+.. include:: ../../../_build/_static/vpp/failed-tests.rst
+
+Dashboard
+---------
+
+Legend to the tables:
+
+    - **Test Case**: name of FD.io CSIT test case, naming convention

       `here <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
       `here <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.

-    - **Trend [Mpps]** : last value of performance trend.
-    - **Short-Term Change [%]** : Relative change of last trend value
+    - **Trend [Mpps]**: last value of performance trend.
+    - **Short-Term Change [%]**: Relative change of last trend value

       vs. last week trend value.
       vs. last week trend value.

-    - **Long-Term Change [%]** : Relative change of last trend value vs.
+    - **Long-Term Change [%]**: Relative change of last trend value vs.

       maximum of trend values over the last quarter except last week.
       maximum of trend values over the last quarter except last week.

-    - **Regressions [#]** : Number of regressions detected.
-    - **Progressions [#]** : Number of progressions detected.
-    - **Outliers [#]** : Number of outliers detected.
+    - **Regressions [#]**: Number of regressions detected.
+    - **Progressions [#]**: Number of progressions detected.
+    - **Outliers [#]**: Number of outliers detected.

 
 Tested VPP worker-thread-core combinations (1t1c, 2t2c, 4t4c) are listed
 in separate tables in section 1.x. Followed by trending methodology in
 
 Tested VPP worker-thread-core combinations (1t1c, 2t2c, 4t4c) are listed
 in separate tables in section 1.x. Followed by trending methodology in


@@ -39,16 +61,16 @@ section 2. and trendline graphs in sections 3.x. Performance test  data
 used for trendline graphs is provided in sections 4.x.
 
 VPP worker on 1t1c
 used for trendline graphs is provided in sections 4.x.
 
 VPP worker on 1t1c

-------------------
+``````````````````

 
 .. include:: ../../../_build/_static/vpp/performance-trending-dashboard-1t1c.rst
 
 VPP worker on 2t2c
 
 .. include:: ../../../_build/_static/vpp/performance-trending-dashboard-1t1c.rst
 
 VPP worker on 2t2c

-------------------
+``````````````````

 
 .. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t2c.rst
 
 VPP worker on 4t4c
 
 .. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t2c.rst
 
 VPP worker on 4t4c

-------------------
+``````````````````

 
 .. include:: ../../../_build/_static/vpp/performance-trending-dashboard-4t4c.rst
 
 .. include:: ../../../_build/_static/vpp/performance-trending-dashboard-4t4c.rst