2 64b-2t1c-avf-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc: 5
3 64b-2t1c-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-vppl2xc: 5
4 64b-2t1c-avf-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc: 5
5 64b-2t1c-avf-eth-l2xcbase: 4
6 64b-2t1c-avf-eth-l2xcbase-eth-2vhostvr1024-1vm-vppl2xc: 5
7 64b-2t1c-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc: 5
8 64b-2t1c-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc: 5
9 64b-2t1c-eth-l2xcbase-eth-2vhostvr1024-1vm-vppl2xc: 5
11 64b-2t1c-avf-dot1q-l2xcbase: 3
12 64b-2t1c-avf-ethip4-ip4scale200k-rnd: 2
13 64b-2t1c-avf-ethip4-ip4scale2m-rnd: 2
14 64b-2t1c-avf-ethip4udp-ip4base-iacl50sf-10kflows: 1
15 64b-2t1c-avf-ethip4udp-ip4base-iacl50sl-10kflows: 1
16 64b-2t1c-avf-ethip4udp-ip4base-oacl50sf-10kflows: 1
17 64b-2t1c-avf-ethip4udp-ip4base-oacl50sl-10kflows: 1
18 64b-2t1c-ethip4-ip4scale200k-rnd: 2
19 64b-2t1c-ethip4-ip4scale2m: 7
20 64b-2t1c-ethip4-ip4scale2m-rnd: 2
22 # Multiline string, empty lines (or \n) are required,
23 # see https://yaml-multiline.info/
26 [1] DONE Apparent regression in ACL tests due to Gerrit 26112. With CSIT
27 using changed VPP API, instead of failing, misconfigured VPP passes
28 the test, albeit with reduced performance. [CSIT-1739]
30 [2] DONE Regression in tests using random flow sequencing due to
31 Gerrit 26456. Unpinned random seed to make TRex workers generate
32 different sequences of packets. [CSIT-1731]
34 [3] DONE Regression in base dot1q-l2xcbase suites due to Gerrit
35 25363. Unified dot1q handling across various tests. Previously,
36 this one suite was different, using dot1q on one port only.
38 [4] DONE Progression in high performance tests due to Gerrit 26445.
39 TRex setting improvement. [CSIT-1740]
41 [5] DONE Progression in tests with VPP (l2xc) inside a VM due to
42 Gerrit 25309. Make VPP in VM worker placement deterministic.
45 [7] POSTPONED Possible anomaly, but stdev of results is high.
46 Performance differences considered to probably be just noise.
48 [8] DONE Regressions in some tests with VPP in VM due to Gerrit
49 26744. Switch to clang-9 compiler identified as cause. Tests using
50 l2xcbase as vswitch with small stdev used for bisect. [CSIT-1741]
52 [9] OPEN Progression in all ip6 scale tests. Changes to vppinfra
53 bihash may have improved code efficiency. [CSIT-1735]
55 [11] OPEN Progression in memif tests, mainly in
56 avf-eth-l2xcbase-eth-2memif-1dcr. Two progression candidates found:
57 Gerrit 24053 and Gerrit 26744. Performance results are not consistent enough
58 to separate the two from other possible causes. [CSIT-1744]
60 [12] OPEN Progression in some l2bdscale tests. [CSIT-1745]
62 [13] DONE Stateless ACL tests have the same performance in 2005.0-4
63 as in 2001.0-2. The difference shown here is inflated by an API
64 change, see [1]. [CSIT-1739]
66 [14] DONE Stateful ACL tests have better performance in 2005.0-4 as
67 in 2001.0-2. The CSIT part of API change, Gerrit 26112, has fixed
68 an earlier CSIT regression in the same tests. The difference shown here
69 is further inflated by the API mismatch, see [1]. [CSIT-1739]
71 [15] OPEN Progression in nat44 test (not scale ones). [CSIT-1747]
73 [17] OPEN Progression in some vhost tests (dpdk in VM). [CSIT-1749]