4 Performance Trending (PT)
5 `````````````````````````
7 CSIT PT runs regular performance test jobs measuring and collecting MRR
8 data per test case. PT is designed as follows:
12 a) Periodic e.g. twice a day.
13 b) On-demand gerrit triggered.
15 2. Measurements and data calculations per test case:
17 a) Max Received Rate (MRR) - for each trial measurement,
18 send packets at link rate for trial duration,
19 count total received packets, divide by trial duration.
21 3. Archive MRR values per test case.
22 4. Archive all counters collected at MRR.
24 Performance Analysis (PA)
25 `````````````````````````
27 CSIT PA runs performance analysis
28 including anomaly detection as described above.
29 PA is defined as follows:
33 a) By PT jobs at their completion.
34 b) On-demand gerrit triggered.
36 2. Download and parse archived historical data and the new data:
38 a) Download RF output.xml files from latest PT job and compressed
39 archived data from nexus.
40 b) Parse out the data filtering test cases listed in PA specification
41 (part of CSIT PAL specification file).
43 3. Re-calculate new groups and their averages.
45 4. Evaluate new test data:
47 a) If the existing group is prolonged => Result = Pass,
49 b) If a new group is detected with lower average =>
50 Result = Fail, Reason = Regression.
51 c) If a new group is detected with higher average =>
52 Result = Pass, Reason = Progression.
54 5. Generate and publish results
56 a) Relay evaluation result to job result.
57 b) Generate a new set of trend summary dashboard, list of failed
59 c) Publish trend dashboard and graphs in html format on
61 d) Generate an alerting email. This email is sent by Jenkins to
62 csit-report@lists.fd.io