1 .. _vpp_performance_tests_release_notes:
6 Changes in |csit-release|
7 -------------------------
9 #. VPP PERFORMANCE TESTS
11 - **MLRsearch improvements**: Added support for multiple packet
12 throughput rates in a single search, each rate is associated
13 with a distinct Packet Loss Ratio (PLR) criterion. Previously
14 only Non Drop Rate (NDR) (PLR=0) and single Partial Drop Rate
15 (PDR) (PLR<0.5%) were supported. Implemented number of
16 optimizations improving rate discovery efficiency.
18 - **Reduction of tests**: Removed obsolete VPP use cases and
19 superfluous test combinations from continuous and report test
20 executions, including:
22 - All vts tests, obsolete use cases.
23 - dot1q tests apart from dot1q-l2bd, superfluous combinations.
24 - -100flows, -100kflows in all acl tests.
27 - -pps tests, replaced by -tput tests.
28 - h1-p1-s1 single session tests, unessential combination.
29 - h4096-p63-s258048 tests, unessential scale combination.
33 - ethip4ipsectptlispgpe.
35 - policy-aes128cbc-hmac256sha.
36 - policy-aes128cbc-hmac512sha.
37 - int-aes128cbc-hmac256sha.
47 - **Telemetry retouch**: Refactored telemetry retrieval from DUTs and
48 SUTs. Included VPP perfmon plugin telemetry with all perfmon
49 bundles available in VPP release.
51 - **Upgrade to Ubuntu 20.04 LTS**: Re-installed base operating system
52 to Ubuntu 20.04.2 LTS. Upgrade included also baseline Docker
53 containers used for spawning topology.
55 - **TRex upgrade v2.86 to v2.88**: Included move to DPDK 21.02 and
56 changed the way egress low latency queues are used in FVL NICs.
57 This broke latency measurements for majority of FVL NICs in
58 CSIT. Latency values look better after upgrading FVL FW on TRex
59 servers, but still somewhat higher than before the TRex upgrade.
60 Tracked by `CSIT-1790 <https://jira.fd.io/browse/CSIT-1790>`_.
62 - **CSIT test environment** version has been updated to ver. 7, see
63 :ref:`test_environment_versioning`.
65 - **CSIT PAPI support**: Due to issues with PAPI performance, VAT is
66 still used in CSIT for all VPP scale tests. See known issues
69 - **General Code Housekeeping**: Ongoing code optimizations and bug
72 #. PRESENTATION AND ANALYTICS LAYER
74 - **Graphs improvements**: Updated Packet Latency graphs,
75 see :ref:`latency_methodology`.
86 List of known issues in |csit-release| for VPP performance tests:
88 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
89 | # | JiraID | Issue Description |
90 +====+=========================================+===========================================================================================================+
91 | 1 | `CSIT-1763 | Adapt ramp-up phase of nat44 tests for different frame sizes. |
92 | | <https://jira.fd.io/browse/CSIT-1763>`_ | Currently ramp-up phase rate and duration values are correctly set for tests with 64B frame size. |
93 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
94 | 2 | `CSIT-1671 | All CSIT scale tests can not use PAPI due to much slower performance compared to VAT/CLI (it takes much |
95 | | <https://jira.fd.io/browse/CSIT-1671>`_ | longer to program VPP). This needs to be addressed on the PAPI side. |
96 | +-----------------------------------------+ The usual PAPI library spends too much time parsing arguments, so even with async processing (hundreds of |
97 | | `VPP-1763 | commands in flight over socket), the VPP configuration for large scale tests (millions of messages) takes |
98 | | <https://jira.fd.io/browse/VPP-1763>`_ | too long. |
99 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
100 | 3 | `CSIT-1790 | Broken TRex latency measurements with TRex v2.88, DPDK 21.02 and FVL FW 6.01. |
101 | | <https://jira.fd.io/browse/CSIT-1790>`_ | After upgrading TRex to v2.88, we also needed to upgrade FVL firmware on TG machines, |
102 | | | to avoid high latency O(5msec) for all VPP and testpmd/l3fwd test cases. |
103 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
104 | 4 | `CSIT-1789 | AVF driver does not perform RSS in a deterministic way. |
105 | | <https://jira.fd.io/browse/CSIT-1789>`_ | This increases standard deviation of tests with small number of flows (mainly ipsec). |
106 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
107 | 5 | `CSIT-1780 | IPSEC SW async scheduler MRR tests failing with no traffic forwarded. |
108 | | <https://jira.fd.io/browse/CSIT-1780>`_ | |
109 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
110 | 6 | `CSIT-1785 | NAT44ED tests failing to establish all TCP sessions. |
111 | | <https://jira.fd.io/browse/CSIT-1785>`_ | |
112 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
114 Root Cause Analysis for Performance Changes
115 -------------------------------------------
117 List of RCAs in |csit-release| for VPP performance changes:
119 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+
120 | # | JiraID | Issue Description |
121 +====+=========================================+===========================================================================================================+
122 | 1 | `VPP-1972 | One VPP change has decreased performance of NAT44ed processing, both slow path and fast path. |
123 | | <https://jira.fd.io/browse/VPP-1972>`_ | |
124 +----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+