Report: vpp performance release notes update
[csit.git] / docs / report / vpp_performance_tests / overview.rst
index e9c4b8d..b316e81 100644 (file)
@@ -1,18 +1,19 @@
 Overview
 ========
 
+VPP performance test results are reported for a range of processors.
 For description of physical testbeds used for VPP performance tests
-please refer to :ref:`physical_testbeds`.
+please refer to :ref:`tested_physical_topologies`.
+
+.. _tested_logical_topologies:
 
 Logical Topologies
 ------------------
 
-.. _tested_logical_topologies:
-
 CSIT VPP performance tests are executed on physical testbeds described
-in :ref:`physical_testbeds`. Based on the packet path thru server SUTs,
-three distinct logical topology types are used for VPP DUT data plane
-testing:
+in :ref:`tested_physical_topologies`. Based on the packet path thru
+server SUTs, three distinct logical topology types are used for VPP DUT
+data plane testing:
 
 #. NIC-to-NIC switching topologies.
 #. VM service switching topologies.
@@ -30,9 +31,10 @@ testbeds are shown in figures below.
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-2n-nic2nic}
-            \label{fig:logical-2n-nic2nic}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-2n-nic2nic}
+                \label{fig:logical-2n-nic2nic}
         \end{figure}
 
 .. only:: html
@@ -47,9 +49,10 @@ testbeds are shown in figures below.
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-3n-nic2nic}
-            \label{fig:logical-3n-nic2nic}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-3n-nic2nic}
+                \label{fig:logical-3n-nic2nic}
         \end{figure}
 
 .. only:: html
@@ -87,7 +90,7 @@ VM Service Switching
 VM service switching topology test cases require VPP DUT to communicate
 with Virtual Machines (VMs) over vhost-user virtual interfaces.
 
-Two types of VM service topologies are tested in CSIT |release|:
+Two types of VM service topologies are tested in |csit-release|:
 
 #. "Parallel" topology with packets flowing within SUT from NIC(s) via
    VPP DUT to VM, back to VPP DUT, then out thru NIC(s).
@@ -107,9 +110,10 @@ SUT running N of VM instances is shown in the figures below.
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-2n-vm-vhost}
-            \label{fig:logical-2n-vm-vhost}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-2n-vm-vhost}
+                \label{fig:logical-2n-vm-vhost}
         \end{figure}
 
 .. only:: html
@@ -124,9 +128,10 @@ SUT running N of VM instances is shown in the figures below.
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-3n-vm-vhost}
-            \label{fig:logical-3n-vm-vhost}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-3n-vm-vhost}
+                \label{fig:logical-3n-vm-vhost}
         \end{figure}
 
 .. only:: html
@@ -161,7 +166,7 @@ Container Service Switching
 Container service switching topology test cases require VPP DUT to
 communicate with Containers (Ctrs) over memif virtual interfaces.
 
-Three types of VM service topologies are tested in CSIT |release|:
+Three types of VM service topologies are tested in |csit-release|:
 
 #. "Parallel" topology with packets flowing within SUT from NIC(s) via
    VPP DUT to Container, back to VPP DUT, then out thru NIC(s).
@@ -187,9 +192,10 @@ below.
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-2n-container-memif}
-            \label{fig:logical-2n-container-memif}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-2n-container-memif}
+                \label{fig:logical-2n-container-memif}
         \end{figure}
 
 .. only:: html
@@ -204,9 +210,10 @@ below.
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-3n-container-memif}
-            \label{fig:logical-3n-container-memif}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-3n-container-memif}
+                \label{fig:logical-3n-container-memif}
         \end{figure}
 
 .. only:: html
@@ -251,19 +258,26 @@ topologies and configurations:
 
 - One-Way Packet Latency: measured at different offered packet loads:
 
-  - 100% of discovered NDR throughput.
-  - 100% of discovered PDR throughput.
+  - 90% of discovered PDR throughput.
+  - 50% of discovered PDR throughput.
+  - 10% of discovered PDR throughput.
+  - Minimal offered load.
 
 - Maximum Receive Rate (MRR): measure packet forwarding rate under the
   maximum load offered by traffic generator over a set trial duration,
   regardless of packet loss. Maximum load for specified Ethernet frame
-  size is set to the bi-directional link rate.
+  size is set to the bi-directional link rate, unless there is a known
+  limitation preventing Traffic Generator from achieving the line rate.
+
+.. todo::
 
-CSIT |release| includes following performance test areas covered across
-a range of NIC drivers and NIC models:
+   - Connections per second (CPS): TODO
+
+|csit-release| includes following VPP data plane functionality
+performance tested across a range of NIC drivers and NIC models:
 
 +-----------------------+----------------------------------------------+
-| Test Area             |  Description                                 |
+| Functionality         |  Description                                 |
 +=======================+==============================================+
 | ACL                   | L2 Bridge-Domain switching and               |
 |                       | IPv4and IPv6 routing with iACL and oACL IP   |
@@ -282,7 +296,11 @@ a range of NIC drivers and NIC models:
 | IPv6 Scale            | IPv6 routing with 20k, 200k and 2M FIB       |
 |                       | entries.                                     |
 +-----------------------+----------------------------------------------+
-| IPSecHW               | IPSec encryption with AES-GCM, CBC-SHA1      |
+| IPSecAsyncHW          | IPSec encryption with AES-GCM, CBC-SHA-256   |
+|                       | ciphers in async mode, in combination with   |
+|                       | IPv4 routing. Intel QAT HW acceleration.     |
++-----------------------+----------------------------------------------+
+| IPSecHW               | IPSec encryption with AES-GCM, CBC-SHA-256   |
 |                       | ciphers, in combination with IPv4 routing.   |
 |                       | Intel QAT HW acceleration.                   |
 +-----------------------+----------------------------------------------+
@@ -290,15 +308,11 @@ a range of NIC drivers and NIC models:
 |                       | combination with LISP-GPE overlay tunneling  |
 |                       | for IPv4-over-IPv4.                          |
 +-----------------------+----------------------------------------------+
-| IPSecSW               | IPSec encryption with AES-GCM, CBC-SHA1      |
+| IPSecSW               | IPSec encryption with AES-GCM, CBC-SHA-256   |
 |                       | ciphers, in combination with IPv4 routing.   |
 +-----------------------+----------------------------------------------+
-| K8s Containers Memif  | K8s orchestrated container VPP service chain |
-|                       | topologies connected over the memif virtual  |
-|                       | interface.                                   |
-+-----------------------+----------------------------------------------+
 | KVM VMs vhost-user    | Virtual topologies with service              |
-|                       | chains of 1 and 2 VMs using vhost-user       |
+|                       | chains of 1 VM using vhost-user              |
 |                       | interfaces, with different VPP forwarding    |
 |                       | modes incl. L2XC, L2BD, VXLAN with L2BD,     |
 |                       | IPv4 routing.                                |
@@ -325,9 +339,10 @@ a range of NIC drivers and NIC models:
 | Memif                 | with different VPP forwarding modes incl.    |
 |                       | L2XC, L2BD.                                  |
 +-----------------------+----------------------------------------------+
-| NAT                   | (Source) Network Address Translation tests   |
-|                       | with varying number of users and ports per   |
-|                       | user.                                        |
+| NAT44                 | (Source) Network Address Translation         |
+|                       | deterministic mode and endpoint-dependent    |
+|                       | mode tests with varying number of users and  |
+|                       | ports per user for IPv4.                     |
 +-----------------------+----------------------------------------------+
 | QoS Policer           | Ingress packet rate measuring, marking and   |
 |                       | limiting (IPv4).                             |
@@ -337,6 +352,10 @@ a range of NIC drivers and NIC models:
 | VPP TCP/IP stack      | Tests of VPP TCP/IP stack used with VPP      |
 |                       | built-in HTTP server.                        |
 +-----------------------+----------------------------------------------+
+| VTS                   | Virtual Topology System use case tests       |
+|                       | combining VXLAN overlay tunneling with L2BD, |
+|                       | ACL and KVM VM vhost-user features.          |
++-----------------------+----------------------------------------------+
 | VXLAN                 | VXLAN overlay tunnelling integration with    |
 |                       | L2XC and L2BD.                               |
 +-----------------------+----------------------------------------------+
@@ -349,8 +368,8 @@ NIC models has been limited to few baseline tests.
 Performance Tests Naming
 ------------------------
 
-FD.io CSIT |release| follows a common structured naming convention for
-all performance and system functional tests, introduced in CSIT rls1701.
+FD.io |csit-release| follows a common structured naming convention for
+all performance and system functional tests, introduced in CSIT-17.01.
 
 The naming should be intuitive for majority of the tests. Complete
 description of FD.io CSIT test naming convention is provided on