CSIT-913: Continuous Trending, Analysis and Change Detection

[csit.git] / resources / tools / presentation / doc / pal_lld.rst

diff --git a/resources/tools/presentation/doc/pal_lld.rst b/resources/tools/presentation/doc/pal_lld.rst
index 64bde3e..2e119fa 100644 (file)
--- a/resources/tools/presentation/doc/pal_lld.rst
+++ b/resources/tools/presentation/doc/pal_lld.rst
@@ -840,6 +840,35 @@ latency in a box chart):
         width: 700
         height: 1000
 
+The structure of the section "Plot" is as follows (example of a plot showing
+VPP HTTP server performance in a box chart with pre-defined data
+"plot-vpp-httlp-server-performance" set and  plot layout "plot-cps"):
+
+::
+
+    -
+      type: "plot"
+      title: "VPP HTTP Server Performance"
+      algorithm: "plot_http_server_performance_box"
+      output-file-type: ".html"
+      output-file: "{DIR[STATIC,VPP]}/http-server-performance-cps"
+      data:
+        "plot-vpp-httlp-server-performance"
+      # Keep this formatting, the filter is enclosed with " (quotation mark) and
+      # each tag is enclosed with ' (apostrophe).
+      filter: "'HTTP' and 'TCP_CPS'"
+      parameters:
+      - "result"
+      - "name"
+      traces:
+        hoverinfo: "x+y"
+        boxpoints: "outliers"
+        whiskerwidth: 0
+      layout:
+        title: "VPP HTTP Server Performance"
+        layout:
+          "plot-cps"
+
 
 Section: file
 '''''''''''''
@@ -1339,6 +1368,46 @@ of an element is required, only a new algorithm needs to be implemented
 and integrated.
 
 
+Continuous Performance Measurements and Trending
+------------------------------------------------
+
+Performance analysis and trending execution sequence:
+`````````````````````````````````````````````````````
+
+1. Triggered at completion of Performance Measurements and Archiving (PMA) job.
+
+   a. Periodic, or gerrit triggers are supported too.
+
+2. Download RF output.xml from triggering CPM job.
+3. Parse out the test results listed in PAL specification file.
+4. Reads specified amount of PMA historical data from Nexus.
+5. Calculate specified statistical metrics – see next section.
+6. Evaluate latest results against the historical metrics, quantify relative
+   change and based on defined criteria set the result to Pass (no-change or
+   progression) or Fail (regression).
+7. Add the new data to historical data.
+8. Generate a new set of trend analysis summary and drill-down graphs.
+9. Archive the latest RF output.xml to nexus for future analysis.
+10. Publish trend analysis graphs in html format on https://docs.fd.io/.
+
+Parameters to specify:
+``````````````````````
+
+- job to be monitored - the Jenkins job which results are used as input data for
+  this test;
+- number of builds used for trending plot(s) - specified by an integer greater
+  than zero, or zero for all available builds;
+- tests we are interested in (list) list of tests which results are used for the
+  test;
+- window size for the moving average.
+
+*Example:*
+
+::
+
+    TODO
+
+
 API
 ---