X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Freport%2Fvpp_performance_tests%2Fcsit_release_notes.rst;h=3ee4ff33fdec4cb89c0d39c3a3020215cf7bd0f8;hp=3434182744335b87ae8cb166cecf22c0561a52ea;hb=b6d7373834eb3fa6e8543be9c8379507d6d91273;hpb=7a307e1c3df11defdd7cc46b5f07c30cd3770e72 diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 3434182744..3ee4ff33fd 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -4,63 +4,43 @@ CSIT Release Notes Changes in CSIT |release| ------------------------- -#. Test environment changes in VPP data plane performance tests: - - - Further characterization and optimizations of VPP vhost-user and VM test - methodology and test environment; - - - Tests with varying Qemu virtio queue (a.k.a. vring) sizes: - [vr256] default 256 descriptors, [vr1024] 1024 descriptors to - optimize for packet throughput; - - - Tests with varying Linux :abbr:`CFS (Completely Fair Scheduler)` - settings: [cfs] default settings, [cfsrr1] :abbr:`CFS (Completely Fair - Scheduler)` RoundRobin(1) policy applied to all data plane threads - handling test packet path including all VPP worker threads and all Qemu - testpmd poll-mode threads; - - - Resulting test cases are all combinations with [vr256,vr1024] and - [cfs,cfsrr1] settings; - - - For more detail see performance results observations section in - this report; - -#. Code updates and optimizations in CSIT performance framework: +#. Added VPP performance tests - - Complete CSIT framework code revision and optimizations as descried - on CSIT wiki page `Design_Optimizations - `_. + - **L2BD MAC scale tests** - - For more detail see the :ref:`CSIT Framework Design ` section - in this report; + - VPP L2 Bridge-Domain with MAC learning and large size L2FIB (10k, + 100k, 1M MACs), tested in NIC-to-NIC and VM vhost topologies. -#. Changes to CSIT driver for TRex Traffic Generator: + - **Linux Container VPP memif tests** - - Complete refactor of TRex CSIT driver; + - Tests with VPP in L2 Bridge-Domain configuration connecting over + memif virtual interfaces to VPPs running in LXCs; - - Introduction of packet traffic profiles to improve usability and - manageability of traffic profiles for a growing number of test - scenarios. + - **Docker Container VPP memif tests** - - Support for packet traffic profiles to test IPv4/IPv6 stateful and - stateless DUT data plane features; + - Tests with VPP in L2 Cross-Connect configuration connecting over + memif virtual interfaces VPPs running in Docker containers; -#. Added VPP performance tests + - **Container Topologies Orchestrated by K8s with VPP memif tests** - - **Linux Container VPP memif virtual interface tests** - - - New VPP Memif virtual interface (shared memory interface) tests - with L2 Bridge-Domain switched-forwarding; + - Tests with VPP in L2 Cross-Connect and Bridge-Domain configurations + connecting over memif virtual interfaces VPPs running in Docker + containers, with service chain topologies orchestrated by Kubernetes; - **Stateful Security Groups** - - New m-thread m-core VPP stateful security-groups tests; + - m-thread m-core VPP stateful and stateless security-groups tests; - **MAC-IP binding** - - New MACIP iACL single-thread single-core and m-thread m-core tests; + - MACIP input access-lists, single-thread single-core and m-thread + m-core tests; + +#. Presentation and Analytics Layer - - Statistical analysis of repeatibility of results; + - New Presentation and Analytics Layer (PAL) for automated CSIT test + results analysis and presentation, including statistical analysis + of results repeatibility and test report auto-generation; Performance Improvements ------------------------ @@ -71,8 +51,8 @@ double-digit percentage points. Relative improvements for this release are calculated against the test results listed in CSIT |release-1| report. The comparison is calculated between the mean values based on collected and archived test results' samples for involved VPP releases. Standard deviation -has been also listed for CSIT |release|. VPP-16.09 and VPP-17.01 numbers are -provided for reference. +has been also listed for CSIT |release|. Performance numbers since release +VPP-16.09 are provided for reference. NDR Throughput ~~~~~~~~~~~~~~ @@ -91,7 +71,7 @@ Non-Drop Rate Throughput discovery tests: \makeatletter \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{5cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, + perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, } \makeatother @@ -123,7 +103,7 @@ Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%: \makeatletter \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{5cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, + perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, } \makeatother @@ -167,7 +147,7 @@ Non-Drop Rate Throughput discovery tests: \makeatletter \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{5cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, + perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, } \makeatother @@ -199,7 +179,7 @@ Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%: \makeatletter \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{5cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, + perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{ m{1.5cm} m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, } \makeatother @@ -232,7 +212,7 @@ Here is the list of known issues in CSIT |release| for VPP performance tests: | | of discovered NDR throughput values across | | throughput. Applies to XL710 and X710 NICs, x520 NICs are fine. | | | multiple test runs with xl710 and x710 NICs. | | | +---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| 4 | Lower than expected NDR and PDR throughput with | CSIT-569 | Suspected NIC firmware or DPDK driver issue affecting NDR and | +| 4 | Lower than expected NDR throughput with | CSIT-569 | Suspected NIC firmware or DPDK driver issue affecting NDR and | | | xl710 and x710 NICs, compared to x520 NICs. | | PDR throughput. Applies to XL710 and X710 NICs. | +---+-------------------------------------------------+------------+-----------------------------------------------------------------+