X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Fcpta%2Fmethodology%2Findex.rst;h=349778999ebdd7cb14f75db5f094aa27b053970c;hp=8bff734e4ac2811cbc27db2dc7cd2a102085a318;hb=3dcef45002a1b82c4503ec590d680950930fa193;hpb=39cc1ccaacb7599130bb99ea781b8cc41a7c1f8b diff --git a/docs/cpta/methodology/index.rst b/docs/cpta/methodology/index.rst index 8bff734e4a..349778999e 100644 --- a/docs/cpta/methodology/index.rst +++ b/docs/cpta/methodology/index.rst @@ -94,7 +94,7 @@ over the rolling window of last sets of historical measurement data: Outlier Detection ````````````````` -Outlier evaluation of test result of value follows the definition +Outlier evaluation of test result of value *X* follows the definition from previous section: +----------------------------+----------------------+ @@ -112,16 +112,15 @@ To verify compliance of test result of valid value against defined trend metrics and detect anomalies, three simple evaluation formulas are used: -+-------------------------------------------+------------------+-------------+ -| Anomaly | Compliance | Evaluation | -| Evaluation Formula | Confidence Level | Result | -+===========================================+==================+=============+ -| (TMM - 3 * TMSD) <= X <= (TMM + 3 * TMSD) | 99.73% | Normal | -+-------------------------------------------+------------------+-------------+ -| X < (TMM - 3 * TMSD) | Anomaly | Regression | -+-------------------------------------------+------------------+-------------+ -| X > (TMM + 3 * TMSD) | Anomaly | Progression | -+-------------------------------------------+------------------+-------------+ ++-------------------------------------------+-----------------------------+-------------------+ +| Anomaly Evaluation Formula | Compliance Confidence Level | Evaluation Result | ++===========================================+=============================+===================+ +| (TMM - 3 * TMSD) <= X <= (TMM + 3 * TMSD) | 99.73% | Normal | ++-------------------------------------------+-----------------------------+-------------------+ +| X < (TMM - 3 * TMSD) | Anomaly | Regression | ++-------------------------------------------+-----------------------------+-------------------+ +| X > (TMM + 3 * TMSD) | Anomaly | Progression | ++-------------------------------------------+-----------------------------+-------------------+ TMM is used for the central trend reference point instead of TMA as it is more robust to anomalies. @@ -136,14 +135,13 @@ ago, TMM[last - 1week] and to the maximum of trend values over last quarter except last week, max(TMM[(last - 3mths)..(last - 1week)]), respectively. This results in following trend compliance calculations: -+-------------------+----------------+-----------+------------------------------------------+ -| Trend | | | | -| Compliance Metric | Change Formula | V(alue) | R(eference) | -+===================+================+===========+==========================================+ -| Short-Term Change | ((V - R) / R) | TMM[last] | TMM[last - 1week] | -+-------------------+----------------+-----------+------------------------------------------+ -| Long-Term Change | ((V - R) / R) | TMM[last] | max(TMM[(last - 3mths)..(last - 1week)]) | -+-------------------+----------------+-----------+------------------------------------------+ ++-------------------------+---------------------------------+-----------+------------------------------------------+ +| Trend Compliance Metric | Trend Change Formula | Value | Reference | ++=========================+=================================+===========+==========================================+ +| Short-Term Change | (Value - Reference) / Reference | TMM[last] | TMM[last - 1week] | ++-------------------------+---------------------------------+-----------+------------------------------------------+ +| Long-Term Change | (Value - Reference) / Reference | TMM[last] | max(TMM[(last - 3mths)..(last - 1week)]) | ++-------------------------+---------------------------------+-----------+------------------------------------------+ Trend Presentation ------------------ @@ -247,3 +245,9 @@ PA is defined as follows: b) Generate a new set of trend summary dashboard and graphs. c) Publish trend dashboard and graphs in html format on https://docs.fd.io/. + +Testbed HW configuration +------------------------ + +The testbed HW configuration is described on +`this FD.IO wiki page `_.