Code Review / csit.git / blobdiff
? search:
summary | shortlog | log | commit | commitdiff | review | tree
raw | inline | side by side
feat(docs): Move job_specs to resources
[csit.git] / resources / tools / presentation / doc / trending_test_groups.md
diff --git a/resources/tools/presentation/doc/trending_test_groups.md b/resources/tools/presentation/doc/trending_test_groups.md
index 9d406ae..8f0b7ea 100644 (file)
--- a/resources/tools/presentation/doc/trending_test_groups.md
+++ b/resources/tools/presentation/doc/trending_test_groups.md
@@ -30,7 +30,7 @@ will be presented in the Trending.
 #### Processor Cores in Trending Graphs
 - 1c (1t1c or 2t1c),
 - 2c (2t2c or 4t2c),
-- 4c (4t4c or 8t4c), see question nr 3.
+- 4c (4t4c or 8t4c).
 
 See item nr 3 in TODO list.
 
@@ -38,14 +38,19 @@ See item nr 3 in TODO list.
 - See [Performance Tests Job Specifications, chapter Frame Sizes]
 - See item nr 2 in TODO list.
 
+### Backward compatibility
+
+The new structure will not be backward compatible with the previous one.
+The old data and graphs will be stored and accessible on the link:
+<https://docs.fd.io/csit/master/trending-archive/>. A link to this page
+will be on the Trending web page.
+
 ### Notes
 1. The lists of tests are based on running tests in trending jobs
    - [2n-skx]
    - [3n-skx]
    - [3n-hsw]
    - [3n-tsh]
-   - [2n-dnv]
-   - [3n-dnv]
    - [2n-clx]
 
    and on the lists of selected tests (test_select_list_*.md) stored in
@@ -60,21 +65,6 @@ See item nr 3 in TODO list.
    [Performance Tests Job Specifications, chapter Frame Sizes].
 3. Change the trending jobs to use processor cores as they are defined
    in [Performance Tests Job Specifications, chapter Processor Cores].
-   From run time point of view, consider using only 1c and 2c for some
-   test beds.
-
-### Questions
-1. If items *1, 2, 3* from *TODO* list are implemented, it will be
-   really hard to make it backward compatible. It means, the trending
-   will start again from the beginning. Is it OK?
-2. Should we add also VICs to the Trending? They are not defined in
-   [Performance Tests Job Specifications, chapter NICs]. They are listed
-   in this document.
-3. Which numbers of processor cores should be used? See point 3 in TODO
-   list.
-
- *Note:* The answers will be implemented into this note and the
- questions will be removed before merge.
 
 # Test Groups
 
@@ -87,7 +77,6 @@ See item nr 3 in TODO list.
     2n1l-25ge2p1xxv710-avf-eth-l2xcbase-mrr
     2n1l-25ge2p1xxv710-avf-dot1q-l2bdbasemaclrn-mrr
     2n1l-25ge2p1xxv710-avf-eth-l2bdbasemaclrn-mrr
-    2n1l-25ge2p1xxv710-avf-dot1q-l2bdbasemaclrn-gbp-mrr
 
 #### 64b-?t?c-l2switching-base-i40e
     2n1l-25ge2p1xxv710-eth-l2patch-mrr
@@ -161,14 +150,6 @@ See item nr 3 in TODO list.
     40ge2p1xl710-eth-l2bdbasemaclrn-mrr
     40ge2p1xl710-eth-l2bdscale1mmaclrn-mrr
 
-### 3n-hsw-vic
-
-#### 64b-?t?c-l2switching-base-i40e
-    10Ge2P1Vic1227-eth-l2xcbase-mrr
-    10Ge2P1Vic1227-eth-l2bdbasemaclrn-mrr
-    40Ge2P1Vic1385-eth-l2xcbase-mrr
-    40Ge2P1Vic1385-eth-l2bdbasemaclrn-mrr
-
 ### 3n-tsh-x520
 
 #### 64b-?t?c-l2switching-base-ixgbe
@@ -193,46 +174,6 @@ See item nr 3 in TODO list.
     10ge2p1x520-eth-l2bdbasemaclrn-oacl50sl-10kflows-mrr
     10ge2p1x520-eth-l2bdbasemaclrn-macip-iacl50sl-10kflows-mrr
 
-### 2n-dnv-x553
-
-#### 64b-?t?c-l2switching-base-ixgbe
-    10ge2p1x520-dot1q-l2xcbase-mrr
-    10ge2p1x520-eth-l2xcbase-mrr
-    10ge2p1x520-dot1q-l2bdbasemaclrn-mrr
-    10ge2p1x520-eth-l2bdbasemaclrn-mrr
-
-#### 64b-?t?c-l2switching-base-scale-ixgbe
-    10ge2p1x520-eth-l2patch-mrr
-    10ge2p1x520-eth-l2xcbase-mrr
-    10ge2p1x520-eth-l2bdbasemaclrn-mrr
-    10ge2p1x520-eth-l2bdscale10kmaclrn-mrr
-    10ge2p1x520-eth-l2bdscale100kmaclrn-mrr
-    10ge2p1x520-eth-l2bdscale1mmaclrn-mrr
-
-### 3n-dnv-x553
-
-#### 64b-?t?c-l2switching-base-ixgbe
-    10ge2p1x520-dot1q-l2xcbase-mrr
-    10ge2p1x520-eth-l2xcbase-mrr
-    10ge2p1x520-dot1q-l2bdbasemaclrn-mrr
-    10ge2p1x520-eth-l2bdbasemaclrn-mrr
-
-#### 64b-?t?c-l2switching-base-scale-ixgbe
-    10ge2p1x520-eth-l2patch-mrr
-    10ge2p1x520-eth-l2xcbase-mrr
-    10ge2p1x520-eth-l2bdbasemaclrn-mrr
-    10ge2p1x520-eth-l2bdscale10kmaclrn-mrr
-    10ge2p1x520-eth-l2bdscale100kmaclrn-mrr
-    10ge2p1x520-eth-l2bdscale1mmaclrn-mrr
-
-####  64b-?t?c-features-l2switching-base-ixgbe
-    10ge2p1x520-eth-l2bdbasemaclrn-mrr
-    10ge2p1x520-eth-l2bdbasemaclrn-iacl50sf-10kflows-mrr
-    10ge2p1x520-eth-l2bdbasemaclrn-iacl50sl-10kflows-mrr
-    10ge2p1x520-eth-l2bdbasemaclrn-oacl50sf-10kflows-mrr
-    10ge2p1x520-eth-l2bdbasemaclrn-oacl50sl-10kflows-mrr
-    10ge2p1x520-eth-l2bdbasemaclrn-macip-iacl50sl-10kflows-mrr
-
 ### 2n-clx-xxv710
 
 #### 64b-?t?c-l2switching-base-avf
@@ -240,7 +181,6 @@ See item nr 3 in TODO list.
     2n1l-25ge2p1xxv710-avf-eth-l2xcbase-mrr
     2n1l-25ge2p1xxv710-avf-dot1q-l2bdbasemaclrn-mrr
     2n1l-25ge2p1xxv710-avf-eth-l2bdbasemaclrn-mrr
-    2n1l-25ge2p1xxv710-avf-dot1q-l2bdbasemaclrn-gbp-mrr
 
 #### 64b-?t?c-l2switching-base-i40e
     2n1l-25ge2p1xxv710-eth-l2patch-mrr
@@ -339,48 +279,8 @@ See item nr 3 in TODO list.
     40ge2p1xl710-ethip4-ip4base-mrr
     40ge2p1xl710-ethip4-ip4scale2m-mrr
 
-### 3n-hsw-vic
-
-#### 64b-?t?c-ip4routing-base-i40e
-    10Ge2P1Vic1227-ethip4-ip4base-mrr
-    40Ge2P1Vic1385-ethip4-ip4base-mrr
-
 ### 3n-tsh-x520
 
-#### 64b-?t?c-ip4routing-base-scale-ixgbe
-    10ge2p1x520-dot1q-ip4base-mrr
-    10ge2p1x520-ethip4-ip4base-mrr
-    10ge2p1x520-ethip4-ip4scale20k-mrr
-    10ge2p1x520-ethip4-ip4scale200k-mrr
-    10ge2p1x520-ethip4-ip4scale2m-mrr
-
-#### 64b-?t?c-features-ip4routing-base-ixgbe
-    10ge2p1x520-ethip4-ip4base-mrr
-    10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-mrr
-    10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-mrr
-    10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-mrr
-    10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-mrr
-    10ge2p1x520-ethip4udp-ip4base-nat44-mrr
-
-### 2n-dnv-x553
-
-#### 64b-?t?c-ip4routing-base-scale-ixgbe
-    10ge2p1x520-dot1q-ip4base-mrr
-    10ge2p1x520-ethip4-ip4base-mrr
-    10ge2p1x520-ethip4-ip4scale20k-mrr
-    10ge2p1x520-ethip4-ip4scale200k-mrr
-    10ge2p1x520-ethip4-ip4scale2m-mrr
-
-#### 64b-?t?c-features-ip4routing-base-ixgbe
-    10ge2p1x520-ethip4-ip4base-mrr
-    10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-mrr
-    10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-mrr
-    10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-mrr
-    10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-mrr
-    10ge2p1x520-ethip4udp-ip4base-nat44-mrr
-
-### 3n-dnv-x553
-
 #### 64b-?t?c-ip4routing-base-scale-ixgbe
     10ge2p1x520-dot1q-ip4base-mrr
     10ge2p1x520-ethip4-ip4base-mrr
@@ -468,32 +368,8 @@ See item nr 3 in TODO list.
     40ge2p1xl710-ethip6-ip6base-mrr
     40ge2p1xl710-ethip6-ip6scale2m-mrr
 
-### 3n-hsw-vic
-
-#### 78b-?t?c-ip6routing-base-i40e
-    10Ge2P1Vic1227-ethip6-ip6base-mrr
-    40Ge2P1Vic1385-ethip6-ip6base-mrr
-
 ### 3n-tsh-x520
 
-#### 78b-?t?c-ip6routing-base-scale-ixgbe
-    10ge2p1x520-dot1q-ip6base-mrr
-    10ge2p1x520-ethip6-ip6base-mrr
-    10ge2p1x520-ethip6-ip6scale20k-mrr
-    10ge2p1x520-ethip6-ip6scale200k-mrr
-    10ge2p1x520-ethip6-ip6scale2m-mrr
-
-### 2n-dnv-x553
-
-#### 78b-?t?c-ip6routing-base-scale-ixgbe
-    10ge2p1x520-dot1q-ip6base-mrr
-    10ge2p1x520-ethip6-ip6base-mrr
-    10ge2p1x520-ethip6-ip6scale20k-mrr
-    10ge2p1x520-ethip6-ip6scale200k-mrr
-    10ge2p1x520-ethip6-ip6scale2m-mrr
-
-### 3n-dnv-x553
-
 #### 78b-?t?c-ip6routing-base-scale-ixgbe
     10ge2p1x520-dot1q-ip6base-mrr
     10ge2p1x520-ethip6-ip6base-mrr
@@ -540,16 +416,6 @@ See item nr 3 in TODO list.
 
 ### 3n-tsh-x520
 
-#### 78b-?t?c-srv6-ip6routing-base-ixgbe
-    10ge2p1x520-ethip6ip6-ip6base-srv6enc1sid-mrr
-    10ge2p1x520-ethip6srhip6-ip6base-srv6enc2sids-mrr
-    10ge2p1x520-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr
-    10ge2p1x520-ethip6srhip6-ip6base-srv6proxy-dyn-mrr
-    10ge2p1x520-ethip6srhip6-ip6base-srv6proxy-masq-mrr
-    10ge2p1x520-ethip6srhip6-ip6base-srv6proxy-stat-mrr
-
-### 3n-dnv-x553
-
 #### 78b-?t?c-srv6-ip6routing-base-ixgbe
     10ge2p1x520-ethip6ip6-ip6base-srv6enc1sid-mrr
     10ge2p1x520-ethip6srhip6-ip6base-srv6enc2sids-mrr
@@ -571,19 +437,11 @@ See item nr 3 in TODO list.
 ### 3n-hsw-xl710
 
 #### 64b-?t?c-ip4tunnel-base-i40e
-    40ge2p1xl710-ethip4vxlan-l2xcbase-mrr
+    40ge2p1xl710-ethip4vxlan-l2xcbase-mrrjaja
     40ge2p1xl710-ethip4vxlan-l2bdbasemaclrn-mrr
 
 ### 3n-tsh-x520
 
-#### 64b-?t?c-ip4tunnel-base-scale-ixgbe
-    10ge2p1x520-ethip4vxlan-l2xcbase-mrr
-    10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-mrr
-    10ge2p1x520-dot1q--ethip4vxlan-l2bdscale1l2bd1vlan1vxlan-mrr
-    10ge2p1x520-dot1q--ethip4vxlan-l2bdscale100l2bd100vlan100vxlan-mrr
-
-### 3n-dnv-x553
-
 #### 64b-?t?c-ip4tunnel-base-scale-ixgbe
     10ge2p1x520-ethip4vxlan-l2xcbase-mrr
     10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-mrr
@@ -714,11 +572,11 @@ See item nr 3 in TODO list.
 ### 3n-skx-xxv710
 
 #### 64b-?t?c-memif-base-i40e
-    10ge2p1xxv710-eth-l2xcbase-eth-2memif-1lxc-mrr
-    10ge2p1xxv710-eth-l2xcbase-eth-2memif-1dcr-mrr
-    10ge2p1xxv710-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr
-    10ge2p1xxv710-eth-l2bdbasemaclrn-eth-2memif-1lxc-mrr
-    10ge2p1xxv710-ethip4-ip4base-eth-2memif-1dcr-mrr
+    25ge2p1xxv710-eth-l2xcbase-eth-2memif-1lxc-mrr
+    25ge2p1xxv710-eth-l2xcbase-eth-2memif-1dcr-mrr
+    25ge2p1xxv710-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr
+    25ge2p1xxv710-eth-l2bdbasemaclrn-eth-2memif-1lxc-mrr
+    25ge2p1xxv710-ethip4-ip4base-eth-2memif-1dcr-mrr
 
 ### 3n-tsh-x520
 
@@ -741,7 +599,7 @@ See item nr 3 in TODO list.
 
 ### 3n-skx-xxv710
 
-#### imix-?t?c-ipsec-ip4routing-base-scale-i40e
+#### imix-?t?c-ipsec-ip4routing-base-scale-sw-i40e
     25ge2p1xxv710-ethip4ipsec4tnlsw-ip4base-int-aes256gcm-mrr
     25ge2p1xxv710-ethip4ipsec4tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr
     25ge2p1xxv710-ethip4ipsec1000tnlsw-ip4base-int-aes256gcm-mrr
@@ -767,16 +625,6 @@ See item nr 3 in TODO list.
 
 ### 3n-tsh-x520
 
-#### imix-?t?c-ipsec-ip4routing-base-scale-sw-ixgbe
-    10ge2p1x520-ethip4ipsec4tnlsw-ip4base-int-aes256gcm-mrr
-    10ge2p1x520-ethip4ipsec4tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr
-    10ge2p1x520-ethip4ipsec1000tnlsw-ip4base-int-aes256gcm-mrr
-    10ge2p1x520-ethip4ipsec1000tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr
-    10ge2p1x520-ethip4ipsec10000tnlsw-ip4base-int-aes256gcm-mrr
-    10ge2p1x520-ethip4ipsec10000tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr
-
-### 3n-dnv-x553
-
 #### imix-?t?c-ipsec-ip4routing-base-scale-sw-ixgbe
     10ge2p1x520-ethip4ipsec4tnlsw-ip4base-int-aes256gcm-mrr
     10ge2p1x520-ethip4ipsec4tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr
@@ -797,13 +645,13 @@ See item nr 3 in TODO list.
 ### 3n-hsw-xl710
 
 #### 114b-?t?c-vts-l2switching-base-i40e
+    40ge2p1xl710-ethip4vxlan-l2bdbasemaclrn-eth-iacldstbase-noacl-2vhostvr1024-1vm-mrr
     40ge2p1xl710-ethip4vxlan-l2bdbasemaclrn-eth-iacldstbase-aclpermit-2vhostvr1024-1vm-mrr
     40ge2p1xl710-ethip4vxlan-l2bdbasemaclrn-eth-iacldstbase-aclpermitreflect-2vhostvr1024-1vm-mrr
-    40ge2p1xl710-ethip4vxlan-l2bdbasemaclrn-eth-iacldstbase-noacl-2vhostvr1024-1vm-mrr
 
 ### 3n-tsh-x520
 
-#### 64b-?t?c-vts-l2switching-base-ixgbe
+#### 114b-?t?c-vts-l2switching-base-ixgbe
     10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-iacldstbase-noacl-2vhostvr1024-1vm-mrr
     10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-iacldstbase-aclpermit-2vhostvr1024-1vm-mrr
     10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-iacldstbase-aclpermitreflect-2vhostvr1024-1vm-mrr
@@ -923,13 +771,13 @@ See item nr 3 in TODO list.
     10ge2p1x710-eth-l2xcbase-testpmd-mrr
     10ge2p1x710-ethip4-ip4base-l3fwd-mrr
 
-### 3n-skx-xl710
+### 3n-hsw-xl710
 
 #### 64b-?t?c-testpmd-l3fwd
     40ge2p1xl710-eth-l2xcbase-testpmd-mrr
     40ge2p1xl710-ethip4-ip4base-l3fwd-mrr
 
-### 3n-skx-x710
+### 3n-hsw-x710
 
 #### 64b-?t?c-testpmd-l3fwd
     10ge2p1x710-eth-l2xcbase-testpmd-mrr
@@ -940,12 +788,10 @@ See item nr 3 in TODO list.
 [3n-skx]: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-skx/
 [3n-hsw]: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master/
 [3n-tsh]: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-tsh/
-[2n-dnv]: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-dnv/
-[3n-dnv]: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-dnv/
 [2n-clx]: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-clx/
-[Performance Tests Job Specifications, chapter NICs]: https://github.com/FDio/csit/blob/master/docs/job_specs/perf_tests_job_specs.md#nics
-[Performance Tests Job Specifications, chapter Frame Sizes]: https://github.com/FDio/csit/blob/master/docs/job_specs/perf_tests_job_specs.md#frame-sizes
-[Performance Tests Job Specifications, chapter Processor Cores]: https://github.com/FDio/csit/blob/master/docs/job_specs/perf_tests_job_specs.md#processor-cores
-[Jobs specifications]: https://github.com/FDio/csit/tree/master/docs/job_specs
+[Performance Tests Job Specifications, chapter NICs]: https://github.com/FDio/csit/blob/master/resources/job_specs/perf_tests_job_specs.md#nics
+[Performance Tests Job Specifications, chapter Frame Sizes]: https://github.com/FDio/csit/blob/master/resources/job_specs/perf_tests_job_specs.md#frame-sizes
+[Performance Tests Job Specifications, chapter Processor Cores]: https://github.com/FDio/csit/blob/master/resources/job_specs/perf_tests_job_specs.md#processor-cores
+[Jobs specifications]: https://github.com/FDio/csit/tree/master/resources/job_specs
 
 # END OF DOCUMENT
Integration tests