Release Notes
=============

Changes in |csit-release|
-------------------------

No code changes apart from bug fixes.

Performance Changes
-------------------

Relative performance changes in measured packet throughput in |csit-release|
are calculated against the results from |csit-release-1|
report. Listed mean and standard deviation values are computed based on
a series of the same tests executed against respective VPP releases to
verify test results repeatability, with percentage change calculated for
mean values. Note that the standard deviation is quite high for a small
number of packet throughput tests, what indicates poor test results
repeatability and makes the relative change of mean throughput value not
fully representative for these tests. The root causes behind poor
results repeatability vary between the test cases.

NDR Changes
~~~~~~~~~~~

NDR throughput changes between releases are available in a
CSV and pretty ASCII formats:

  - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
  - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
  - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
  - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.

.. note::

    Test results have been generated by
    `FD.io test executor dpdk performance job 3n-hsw`_
    with Robot Framework result
    files csit-vpp-perf-|srelease|-\*.zip
    `archived here <../_static/archive/>`_.

PDR Changes
~~~~~~~~~~~

PDR throughput changes between releases are available in a
CSV and pretty ASCII formats:

  - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
  - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
  - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
  - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.

.. note::

    Test results have been generated by
    `FD.io test executor dpdk performance job 3n-hsw`_
    with Robot Framework result
    files csit-vpp-perf-|srelease|-\*.zip
    `archived here <../_static/archive/>`_.

Comparison Across Testbeds
--------------------------

Relative performance changes in measured packet throughputon 3-Node Skx testbed
are calculated against the results measured on 3-Node Hsw testbed.

NDR Changes
~~~~~~~~~~~

NDR changes between testbeds are available in a
CSV and pretty ASCII formats:

  - `csv format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
  - `pretty ASCII format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.

.. note::

    Test results have been generated by
    `FD.io test executor dpdk performance job 3n-hsw`_ and
    `FD.io test executor dpdk performance job 3n-skx`_
    with Robot Framework result
    files csit-vpp-perf-|srelease|-\*.zip
    `archived here <../_static/archive/>`_.

PDR Changes
~~~~~~~~~~~

PDR throughput changes between testbeds are available in a
CSV and pretty ASCII formats:

  - `csv format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
  - `pretty ASCII format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.

.. note::

    Test results have been generated by
    `FD.io test executor dpdk performance job 3n-hsw`_ and
    `FD.io test executor dpdk performance job 3n-skx`_
    with Robot Framework result
    files csit-vpp-perf-|srelease|-\*.zip
    `archived here <../_static/archive/>`_.

Known Issues
------------

Here is the list of known issues in |csit-release| for Testpmd performance tests:

+---+---------------------------------------------------+------------+-----------------------------------------------------------------+
| # | Issue                                             | Jira ID    | Description                                                     |
+---+---------------------------------------------------+------------+-----------------------------------------------------------------+
| 1 | Testpmd in 1t1c and 2t2c setups - large variation | CSIT-569   | Suspected NIC firmware or DPDK driver issue affecting NDR       |
|   | of discovered NDR throughput values across        |            | throughput. Applies to XL710 and X710 NICs, no issues observed  |
|   | multiple test runs with xl710 and x710 NICs.      |            | on x520 NICs.                                                   |
+---+---------------------------------------------------+------------+-----------------------------------------------------------------+
| 2 | Lower than expected NDR throughput with xl710     | CSIT-571   | Suspected NIC firmware or DPDK driver issue affecting NDR       |
|   | and x710 NICs, compared to x520 NICs.             |            | throughput. Applies to XL710 and X710 NICs.                     |
+---+---------------------------------------------------+------------+-----------------------------------------------------------------+