4 Changes in |csit-release|
5 -------------------------
7 #. VPP PERFORMANCE TESTS
9 - **Service density 2n-skx tests**: Added higher NF density tests with
10 802.1q (vlan) and VXLAN encapsulation from Traffic Generator.
12 - **GBP tests**: Added GBP (Group Based Policy) routing test cases
13 with 802.1q (vlan) external traffic.
15 - **AVF IPv4 scale tests**: Increased coverage of AVF IPv4 base and
16 scale test cases (Fortville NICs only).
18 - **2n-skx tests**: Increased coverage of selected (COP, iACL,
21 - **IPsec scale tests**: Added IPsec interface mode scale tests with
22 1, 40, 400, 1000, 5000, 10000, 20000, 40000, 60000 IPsec tunnels.
23 Removed DPDK backend dependency. Major IPsec test code
26 - **Hoststack TCP/IP tests**: Major refactor of Hoststack TCP
27 performance tests using WRK generator talking to the VPP HTTP
28 static server plugin measuring connections per second and
31 <<<<<<< HEAD (e2c90b Report methodology: PLRsearch update with graphs)
33 - **Changed methodology of dot1q tests in 2-Node testbeds**: dot1q
34 encapsulation is now used on both links of SUT. Previously dot1q
35 was used only on a single link with the other link carrying
36 untagged Ethernet frames. This change results in slightly lower
37 throughput in CSIT-1908 for all dot1q tests in all 2-Node
40 - **KVM VM vhost-user tests**: completed move to Kernel-VM for all
41 tests. In addition to running DPDK Testpmd as VM workload, new
42 tests created with VPP as VM workload. VPP in VM is the same
43 version as the DUT VPP (acting as vSwitch) and its configuration
44 depends on the test type. For all L2 Ethernet Switching tests
45 it's vpp-l2xc (L2 cross-connect), for all IPv4 Routing tests it's
46 vpp-ip4 (VPP IPv4 routing).
48 >>>>>>> CHANGE (1abf61 docs: report vpp performance rls notes update.)
51 - **CSIT PAPI Support**: Finished conversion of CSIT VAT L1 keywords
52 to PAPI L1 KWs in CSIT using VPP Python bindings (VPP PAPI).
53 Redesign of key components of PAPI Socket Executor and PAPI
54 history. Due to issues with PAPI performance, VAT is still used
55 in CSIT for all VPP scale tests. See known issues below.
57 - **General Code Housekeeping**: Ongoing RF keywords optimizations,
58 removal of redundant RF keywords and aligning of suite/test
62 #. PRESENTATION AND ANALYTICS LAYER
64 - **Graphs Layout Improvements**: Improved performance graphs layout
65 for better readibility and maintenance: test grouping, axis
66 labels, descriptions, other informative decoration.
77 List of known issues in |csit-release| for VPP performance tests:
79 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
80 | # | JiraID | Issue Description |
81 +====+=========================================+==========================================================================================================+
82 | 1 | `CSIT-570 | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue. |
83 | | <https://jira.fd.io/browse/CSIT-570>`_ | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds. |
84 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
85 | 2 | `VPP-662 | 9000B packets not supported by NICs VIC1227 and VIC1387. |
86 | | <https://jira.fd.io/browse/VPP-662>`_ | |
87 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
88 | 3 | `CSIT-1503 | [`TRex-519 <https://trex-tgn.cisco.com/youtrack/issue/trex-519>`_] XL710/XXV710 with FW 6.0.1 will have |
89 | | <https://jira.fd.io/browse/CSIT-1503>`_ | Rx drop rate of 27MPPS. |
90 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
91 | 4 | `CSIT-1498 | Memif tests are sporadically failing on initialization of memif connection. |
92 | | <https://jira.fd.io/browse/CSIT-1498>`_ | |
93 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
94 | 5 | `CSIT-1499 | AVF tests are sporadically failing on initialization of AVF interface. |
95 | | <https://jira.fd.io/browse/CSIT-1499>`_ | |
96 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
97 | 6 | `VPP-1676 | 9000B ip4 memif errors - ip4-input: ip4 length > l2 length. |
98 | | <https://jira.fd.io/browse/VPP-1676>`_ | IP4 jumbo frames (9000B) are dropped in case of tests with memif. |
99 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
100 | 7 | `VPP-1677 | 9000B ip4 nat44: VPP crash + coredump. |
101 | | <https://jira.fd.io/browse/VPP-1677>`_ | VPP crashes very often in case that NAT44 is configured and it has to process IP4 jumbo frames (9000B). |
102 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
103 | 8 | `CSIT-1591 | All CSIT scale tests can not use PAPI due to much slower performance compared to VAT/CLI (it takes much |
104 | | <https://jira.fd.io/browse/CSIT-1499>`_ | longer to program VPP). This needs to be addressed on the PAPI side. |
107 | | <https://jira.fd.io/browse/VPP-1763>`_ | |
108 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
109 | 9 | `CSIT-1592 | VPP memif API does not enable memif zero-copy, resulting in different memif configuration vs. previously |
110 | | <https://jira.fd.io/browse/CSIT-1592>`_ | tested VAT/CLI where memif zero-copy was enabled by default. Needs to be fixed in VPP. |
113 | | <https://jira.fd.io/browse/VPP-1764>`_ | |
114 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
115 | 10 | `VPP-1675 | IPv4 IPSEC 9000B packet tests are failing as no packet is forwarded. |
116 | | <https://jira.fd.io/browse/VPP-1675>`_ | Reason: chained buffers are not supported. |
117 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+
118 | 11 | `CSIT-1593 | IPv4 AVF 9000B packet tests are failing on 3n-skx while passing on 2n-skx. |
119 | | <https://jira.fd.io/browse/CSIT-1593>`_ | |
120 +----+-----------------------------------------+----------------------------------------------------------------------------------------------------------+