4 Changes in |csit-release|
5 -------------------------
7 #. VPP PERFORMANCE TESTS
9 - CSIT test environment is versioned, see
10 :ref:`test_environment_versioning`.
12 - To identify performance changes due to VPP code changes from
13 v20.01.0 to v20.05.0, both have been tested in CSIT environment
14 ver. 4 and compared against each other. All substantial
15 progressions has been marked up with RCA analysis. See
16 :ref:`vpp_compare_current_vs_previous_release` and
17 :ref:`vpp_known_issues`.
21 - **CSIT PAPI support**: Due to issues with PAPI performance, VAT is
22 still used in CSIT for all VPP scale tests. See known issues below.
24 - **General Code Housekeeping**: Ongoing RF keywords optimizations,
25 removal of redundant RF keywords and aligning of suite/test
28 #. PRESENTATION AND ANALYTICS LAYER
30 - **Graphs layout improvements**: Improved performance graphs layout
31 for better readibility and maintenance: test grouping, axis
32 labels, descriptions, other informative decoration.
43 List of known issues in |csit-release| for VPP performance tests:
45 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
46 | # | JiraID | Issue Description |
47 +====+=========================================+===========================================================================================================+
48 | 1 | `CSIT-570 | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue. |
49 | | <https://jira.fd.io/browse/CSIT-570>`_ | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds. |
50 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
51 | 2 | `VPP-662 | 9000B packets not supported by NICs VIC1227 and VIC1387. |
52 | | <https://jira.fd.io/browse/VPP-662>`_ | |
53 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
54 | 3 | `VPP-1677 | 9000B ip4 nat44: VPP crash + coredump. |
55 | | <https://jira.fd.io/browse/VPP-1677>`_ | VPP crashes very often in case that NAT44 is configured and it has to process IP4 jumbo frames (9000B). |
56 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
57 | 4 | `CSIT-1591 | All CSIT scale tests can not use PAPI due to much slower performance compared to VAT/CLI (it takes much |
58 | | <https://jira.fd.io/browse/CSIT-1499>`_ | longer to program VPP). This needs to be addressed on the PAPI side. |
59 | +-----------------------------------------+ |
61 | | <https://jira.fd.io/browse/VPP-1763>`_ | |
62 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
63 | 5 | `VPP-1675 | IPv4 IPSEC 9000B packet tests are failing as no packet is forwarded. |
64 | | <https://jira.fd.io/browse/VPP-1675>`_ | Reason: chained buffers are not supported. |
65 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
66 | 6 | `CSIT-1593 | IPv4 AVF 9000B packet tests are failing on 3n-skx while passing on 2n-skx. |
67 | | <https://jira.fd.io/browse/CSIT-1593>`_ | |
68 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
69 | 7 | `CSIT-1679 | 2n-clx VPP ip4 tests with xxv710 and avf driver are sporadically failing. |
70 | | <https://jira.fd.io/browse/CSIT-1679>`_ | |
71 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
73 Root Cause Analysis for Performance Changes
74 -------------------------------------------
76 List of RCAs in |csit-release| for VPP performance changes:
78 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
79 | # | JiraID | Issue Description |
80 +====+=========================================+===========================================================================================================+
81 | 1 | `CSIT-1731 | Confirm the cause for ip4scale -rnd regressions. |
82 | | <https://jira.fd.io/browse/CSIT-1731>`_ | |
83 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
84 | 2 | `CSIT-1735 | Identify cause of ethip6-ip6scale2m progression. |
85 | | <https://jira.fd.io/browse/CSIT-1735>`_ | |
86 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
87 | 3 | `CSIT-1738 | Identify cause for vppl2xc CSIT progressions. |
88 | | <https://jira.fd.io/browse/CSIT-1738>`_ | |
89 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
90 | 4 | `CSIT-1739 | Identify cause of ACL regressions. |
91 | | <https://jira.fd.io/browse/CSIT-1739>`_ | |
92 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
93 | 5 | `CSIT-1740 | Identify cause for avf-eth-l2xcbase CSIT progression. |
94 | | <https://jira.fd.io/browse/CSIT-1740>`_ | |
95 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
96 | 6 | `CSIT-1741 | Identify cause for vppl2xc VPP regressions. |
97 | | <https://jira.fd.io/browse/CSIT-1741>`_ | |
98 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
99 | 7 | `CSIT-1742 | Identify cause of ipsec CSIT regression. |
100 | | <https://jira.fd.io/browse/CSIT-1742>`_ | |
101 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
102 | 8 | `CSIT-1744 | Identify cause of memif VPP progression. |
103 | | <https://jira.fd.io/browse/CSIT-1744>`_ | |
104 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
105 | 9 | `CSIT-1745 | Verify cause of l2bdscale10kmaclrn VPP progression. |
106 | | <https://jira.fd.io/browse/CSIT-1745>`_ | |
107 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
108 | 10 | `CSIT-1746 | Identify cause for avf-dot1q-ip6base VPP progression. |
109 | | <https://jira.fd.io/browse/CSIT-1746>`_ | |
110 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
111 | 11 | `CSIT-1747 | Identify cause of ip4base-nat44 VPP progression. |
112 | | <https://jira.fd.io/browse/CSIT-1747>`_ | |
113 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
114 | 12 | `CSIT-1748 | Identify cause of aes128cbc-hmac512sha VPP progression. |
115 | | <https://jira.fd.io/browse/CSIT-1748>`_ | |
116 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
117 | 13 | `CSIT-1749 | Identify cause for l2bdbasemaclrn VPP progression in tests with dpdk app in VM. |
118 | | <https://jira.fd.io/browse/CSIT-1749>`_ | |
119 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+