Release Notes ============= Changes in |csit-release| ------------------------- #. **DPDK performance tests** - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now configurable trial duration and number of consecutive executions. Coverage of MRR tests has been extended across more test scenarios. MRR tests are used for continuous performance trending and for comparison between DPDK releases. - **MLRsearch Throughput**: MLRsearch algorithm has been introduced for all NDR and PDR throughput tests. All tests that previously used binary search got converted to MLRsearch. Coverage of NDR/PDR tests has been extended across more test scenarios. - **2-Node Tests**: A new set of 2-node tests covering DPDK Testpmd and L3fwd, running on new Xeon Skylake testbeds. - **More NICs**: Increased coverage of NIC models tested: Intel- xxv710-da2, Intel-x710-da4. - **Generated tests**: Simplified and unified test structure, semi- autogenerated by generator script. Test generator is currently able to create test combinations with various frame size and cores combinations. All existing test cases were converted to new format. #. **Presentation and Analytics Layer** - **Performance trending**: Added continuous performance trending with anomaly detection and analysis for DPDK. Performance Changes ------------------- Relative performance changes in measured NDR, PDR and MRR packet throughput in |csit-release| are calculated against the test results from |csit-release-1| report, for tests running on 3-Node Intel Xeon Haswell testbeds (3n-hsw) in 1-core, 2-core and 4-core (MRR only) configurations. Listed mean and standard deviation values are computed based on a series of the same tests executed against respective VPP releases to verify test results repeatability, with percentage change calculated for mean values. NDR Changes ~~~~~~~~~~~ NDR throughput changes between releases are available in CSV and pretty ASCII formats: - `CSV 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_, - `CSV 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_, - `ASCII 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_, - `ASCII 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_. .. note:: Test results have been generated by `FD.io test executor dpdk performance job 3n-hsw`_ with RF result files csit-dpdk-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. PDR Changes ~~~~~~~~~~~ PDR throughput changes between releases are available in CSV and pretty ASCII formats: - `CSV 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_, - `CSV 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_, - `ASCII 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_, - `ASCII 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_. .. note:: Test results have been generated by `FD.io test executor dpdk performance job 3n-hsw`_ with RF result files csit-dpdk-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. MRR Changes ~~~~~~~~~~~ MRR throughput changes between releases are available in CSV and pretty ASCII formats: - `CSV 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.csv>`_, - `CSV 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.csv>`_, - `CSV 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.csv>`_, - `ASCII 1t1c MRR changes <../_static/dpdk/performance-changes-1t1c-mrr.txt>`_, - `ASCII 2t2c MRR changes <../_static/dpdk/performance-changes-2t2c-mrr.txt>`_, - `ASCII 4t4c MRR changes <../_static/dpdk/performance-changes-4t4c-mrr.txt>`_. .. note:: Test results have been generated by `FD.io test executor dpdk performance job 3n-hsw`_, with RF result files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. Skx vs. Hsw Comparison ---------------------- Relative performance changes in measured packet throughput on 3-Node Skx testbed are calculated against the results measured on 3-Node Hsw testbed. NDR Comparison ~~~~~~~~~~~~~~ NDR throughput changes between testbeds are available in a CSV and pretty ASCII formats: - `CSV 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_, - `ASCII 1c NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_. .. note:: Test results have been generated by `FD.io test executor dpdk performance job 3n-hsw`_ and `FD.io test executor dpdk performance job 3n-skx`_ with RF result files csit-dpdk-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. PDR Comparison ~~~~~~~~~~~~~~ PDR throughput changes between testbeds are available in a CSV and pretty ASCII formats: - `CSV 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_, - `ASCII 1c PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_. .. note:: Test results have been generated by `FD.io test executor dpdk performance job 3n-hsw`_ and `FD.io test executor dpdk performance job 3n-skx`_ with RF result files csit-dpdk-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. MRR Comparison ~~~~~~~~~~~~~~ MRR throughput changes between testbeds are available in a CSV and pretty ASCII formats: - `CSV 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_, - `ASCII 1c MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_. .. note:: Test results have been generated by `FD.io test executor dpdk performance job 3n-hsw`_ and `FD.io test executor dpdk performance job 3n-skx`_ with RF result files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. Known Issues ------------ Here is the list of known issues in |csit-release| for Testpmd performance tests: +---+---------------------------------------------------+------------+-----------------------------------------------------------------+ | # | Issue | Jira ID | Description | +---+---------------------------------------------------+------------+-----------------------------------------------------------------+ | | No known issues | | | +---+---------------------------------------------------+------------+-----------------------------------------------------------------+