Code Review / csit.git / blobdiff
? search:
summary | shortlog | log | commit | commitdiff | review | tree
raw | inline | side by side
Report: vpp performance release notes update
[csit.git] / docs / report / vpp_performance_tests / overview.rst
diff --git a/docs/report/vpp_performance_tests/overview.rst b/docs/report/vpp_performance_tests/overview.rst
index e9c4b8d..b316e81 100644 (file)
--- a/docs/report/vpp_performance_tests/overview.rst
+++ b/docs/report/vpp_performance_tests/overview.rst
@@ -1,18 +1,19 @@
 Overview
 ========
 
 Overview
 ========
 
+VPP performance test results are reported for a range of processors.
 For description of physical testbeds used for VPP performance tests
 For description of physical testbeds used for VPP performance tests
-please refer to :ref:`physical_testbeds`.
+please refer to :ref:`tested_physical_topologies`.
+
+.. _tested_logical_topologies:
 
 Logical Topologies
 ------------------
 
 
 Logical Topologies
 ------------------
 
-.. _tested_logical_topologies:
-
 CSIT VPP performance tests are executed on physical testbeds described
 CSIT VPP performance tests are executed on physical testbeds described
-in :ref:`physical_testbeds`. Based on the packet path thru server SUTs,
-three distinct logical topology types are used for VPP DUT data plane
-testing:
+in :ref:`tested_physical_topologies`. Based on the packet path thru
+server SUTs, three distinct logical topology types are used for VPP DUT
+data plane testing:
 
 #. NIC-to-NIC switching topologies.
 #. VM service switching topologies.
 
 #. NIC-to-NIC switching topologies.
 #. VM service switching topologies.
@@ -30,9 +31,10 @@ testbeds are shown in figures below.
     .. raw:: latex
 
         \begin{figure}[H]
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-2n-nic2nic}
-            \label{fig:logical-2n-nic2nic}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-2n-nic2nic}
+                \label{fig:logical-2n-nic2nic}
         \end{figure}
 
 .. only:: html
         \end{figure}
 
 .. only:: html
@@ -47,9 +49,10 @@ testbeds are shown in figures below.
     .. raw:: latex
 
         \begin{figure}[H]
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-3n-nic2nic}
-            \label{fig:logical-3n-nic2nic}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-3n-nic2nic}
+                \label{fig:logical-3n-nic2nic}
         \end{figure}
 
 .. only:: html
         \end{figure}
 
 .. only:: html
@@ -87,7 +90,7 @@ VM Service Switching
 VM service switching topology test cases require VPP DUT to communicate
 with Virtual Machines (VMs) over vhost-user virtual interfaces.
 
 VM service switching topology test cases require VPP DUT to communicate
 with Virtual Machines (VMs) over vhost-user virtual interfaces.
 
-Two types of VM service topologies are tested in CSIT |release|:
+Two types of VM service topologies are tested in |csit-release|:
 
 #. "Parallel" topology with packets flowing within SUT from NIC(s) via
    VPP DUT to VM, back to VPP DUT, then out thru NIC(s).
 
 #. "Parallel" topology with packets flowing within SUT from NIC(s) via
    VPP DUT to VM, back to VPP DUT, then out thru NIC(s).
@@ -107,9 +110,10 @@ SUT running N of VM instances is shown in the figures below.
     .. raw:: latex
 
         \begin{figure}[H]
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-2n-vm-vhost}
-            \label{fig:logical-2n-vm-vhost}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-2n-vm-vhost}
+                \label{fig:logical-2n-vm-vhost}
         \end{figure}
 
 .. only:: html
         \end{figure}
 
 .. only:: html
@@ -124,9 +128,10 @@ SUT running N of VM instances is shown in the figures below.
     .. raw:: latex
 
         \begin{figure}[H]
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-3n-vm-vhost}
-            \label{fig:logical-3n-vm-vhost}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-3n-vm-vhost}
+                \label{fig:logical-3n-vm-vhost}
         \end{figure}
 
 .. only:: html
         \end{figure}
 
 .. only:: html
@@ -161,7 +166,7 @@ Container Service Switching
 Container service switching topology test cases require VPP DUT to
 communicate with Containers (Ctrs) over memif virtual interfaces.
 
 Container service switching topology test cases require VPP DUT to
 communicate with Containers (Ctrs) over memif virtual interfaces.
 
-Three types of VM service topologies are tested in CSIT |release|:
+Three types of VM service topologies are tested in |csit-release|:
 
 #. "Parallel" topology with packets flowing within SUT from NIC(s) via
    VPP DUT to Container, back to VPP DUT, then out thru NIC(s).
 
 #. "Parallel" topology with packets flowing within SUT from NIC(s) via
    VPP DUT to Container, back to VPP DUT, then out thru NIC(s).
@@ -187,9 +192,10 @@ below.
     .. raw:: latex
 
         \begin{figure}[H]
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-2n-container-memif}
-            \label{fig:logical-2n-container-memif}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-2n-container-memif}
+                \label{fig:logical-2n-container-memif}
         \end{figure}
 
 .. only:: html
         \end{figure}
 
 .. only:: html
@@ -204,9 +210,10 @@ below.
     .. raw:: latex
 
         \begin{figure}[H]
     .. raw:: latex
 
         \begin{figure}[H]
-        \centering
-            \includesvg[width=0.90\textwidth]{../_tmp/src/vpp_performance_tests/logical-3n-container-memif}
-            \label{fig:logical-3n-container-memif}
+            \centering
+                \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+                \includegraphics[width=0.90\textwidth]{logical-3n-container-memif}
+                \label{fig:logical-3n-container-memif}
         \end{figure}
 
 .. only:: html
         \end{figure}
 
 .. only:: html
@@ -251,19 +258,26 @@ topologies and configurations:
 
 - One-Way Packet Latency: measured at different offered packet loads:
 
 
 - One-Way Packet Latency: measured at different offered packet loads:
 
-  - 100% of discovered NDR throughput.
-  - 100% of discovered PDR throughput.
+  - 90% of discovered PDR throughput.
+  - 50% of discovered PDR throughput.
+  - 10% of discovered PDR throughput.
+  - Minimal offered load.
 
 - Maximum Receive Rate (MRR): measure packet forwarding rate under the
   maximum load offered by traffic generator over a set trial duration,
   regardless of packet loss. Maximum load for specified Ethernet frame
 
 - Maximum Receive Rate (MRR): measure packet forwarding rate under the
   maximum load offered by traffic generator over a set trial duration,
   regardless of packet loss. Maximum load for specified Ethernet frame
-  size is set to the bi-directional link rate.
+  size is set to the bi-directional link rate, unless there is a known
+  limitation preventing Traffic Generator from achieving the line rate.
+
+.. todo::
 
 
-CSIT |release| includes following performance test areas covered across
-a range of NIC drivers and NIC models:
+   - Connections per second (CPS): TODO
+
+|csit-release| includes following VPP data plane functionality
+performance tested across a range of NIC drivers and NIC models:
 
 +-----------------------+----------------------------------------------+
 
 +-----------------------+----------------------------------------------+
-| Test Area             |  Description                                 |
+| Functionality         |  Description                                 |
 +=======================+==============================================+
 | ACL                   | L2 Bridge-Domain switching and               |
 |                       | IPv4and IPv6 routing with iACL and oACL IP   |
 +=======================+==============================================+
 | ACL                   | L2 Bridge-Domain switching and               |
 |                       | IPv4and IPv6 routing with iACL and oACL IP   |
@@ -282,7 +296,11 @@ a range of NIC drivers and NIC models:
 | IPv6 Scale            | IPv6 routing with 20k, 200k and 2M FIB       |
 |                       | entries.                                     |
 +-----------------------+----------------------------------------------+
 | IPv6 Scale            | IPv6 routing with 20k, 200k and 2M FIB       |
 |                       | entries.                                     |
 +-----------------------+----------------------------------------------+
-| IPSecHW               | IPSec encryption with AES-GCM, CBC-SHA1      |
+| IPSecAsyncHW          | IPSec encryption with AES-GCM, CBC-SHA-256   |
+|                       | ciphers in async mode, in combination with   |
+|                       | IPv4 routing. Intel QAT HW acceleration.     |
++-----------------------+----------------------------------------------+
+| IPSecHW               | IPSec encryption with AES-GCM, CBC-SHA-256   |
 |                       | ciphers, in combination with IPv4 routing.   |
 |                       | Intel QAT HW acceleration.                   |
 +-----------------------+----------------------------------------------+
 |                       | ciphers, in combination with IPv4 routing.   |
 |                       | Intel QAT HW acceleration.                   |
 +-----------------------+----------------------------------------------+
@@ -290,15 +308,11 @@ a range of NIC drivers and NIC models:
 |                       | combination with LISP-GPE overlay tunneling  |
 |                       | for IPv4-over-IPv4.                          |
 +-----------------------+----------------------------------------------+
 |                       | combination with LISP-GPE overlay tunneling  |
 |                       | for IPv4-over-IPv4.                          |
 +-----------------------+----------------------------------------------+
-| IPSecSW               | IPSec encryption with AES-GCM, CBC-SHA1      |
+| IPSecSW               | IPSec encryption with AES-GCM, CBC-SHA-256   |
 |                       | ciphers, in combination with IPv4 routing.   |
 +-----------------------+----------------------------------------------+
 |                       | ciphers, in combination with IPv4 routing.   |
 +-----------------------+----------------------------------------------+
-| K8s Containers Memif  | K8s orchestrated container VPP service chain |
-|                       | topologies connected over the memif virtual  |
-|                       | interface.                                   |
-+-----------------------+----------------------------------------------+
 | KVM VMs vhost-user    | Virtual topologies with service              |
 | KVM VMs vhost-user    | Virtual topologies with service              |
-|                       | chains of 1 and 2 VMs using vhost-user       |
+|                       | chains of 1 VM using vhost-user              |
 |                       | interfaces, with different VPP forwarding    |
 |                       | modes incl. L2XC, L2BD, VXLAN with L2BD,     |
 |                       | IPv4 routing.                                |
 |                       | interfaces, with different VPP forwarding    |
 |                       | modes incl. L2XC, L2BD, VXLAN with L2BD,     |
 |                       | IPv4 routing.                                |
@@ -325,9 +339,10 @@ a range of NIC drivers and NIC models:
 | Memif                 | with different VPP forwarding modes incl.    |
 |                       | L2XC, L2BD.                                  |
 +-----------------------+----------------------------------------------+
 | Memif                 | with different VPP forwarding modes incl.    |
 |                       | L2XC, L2BD.                                  |
 +-----------------------+----------------------------------------------+
-| NAT                   | (Source) Network Address Translation tests   |
-|                       | with varying number of users and ports per   |
-|                       | user.                                        |
+| NAT44                 | (Source) Network Address Translation         |
+|                       | deterministic mode and endpoint-dependent    |
+|                       | mode tests with varying number of users and  |
+|                       | ports per user for IPv4.                     |
 +-----------------------+----------------------------------------------+
 | QoS Policer           | Ingress packet rate measuring, marking and   |
 |                       | limiting (IPv4).                             |
 +-----------------------+----------------------------------------------+
 | QoS Policer           | Ingress packet rate measuring, marking and   |
 |                       | limiting (IPv4).                             |
@@ -337,6 +352,10 @@ a range of NIC drivers and NIC models:
 | VPP TCP/IP stack      | Tests of VPP TCP/IP stack used with VPP      |
 |                       | built-in HTTP server.                        |
 +-----------------------+----------------------------------------------+
 | VPP TCP/IP stack      | Tests of VPP TCP/IP stack used with VPP      |
 |                       | built-in HTTP server.                        |
 +-----------------------+----------------------------------------------+
+| VTS                   | Virtual Topology System use case tests       |
+|                       | combining VXLAN overlay tunneling with L2BD, |
+|                       | ACL and KVM VM vhost-user features.          |
++-----------------------+----------------------------------------------+
 | VXLAN                 | VXLAN overlay tunnelling integration with    |
 |                       | L2XC and L2BD.                               |
 +-----------------------+----------------------------------------------+
 | VXLAN                 | VXLAN overlay tunnelling integration with    |
 |                       | L2XC and L2BD.                               |
 +-----------------------+----------------------------------------------+
@@ -349,8 +368,8 @@ NIC models has been limited to few baseline tests.
 Performance Tests Naming
 ------------------------
 
 Performance Tests Naming
 ------------------------
 
-FD.io CSIT |release| follows a common structured naming convention for
-all performance and system functional tests, introduced in CSIT rls1701.
+FD.io |csit-release| follows a common structured naming convention for
+all performance and system functional tests, introduced in CSIT-17.01.
 
 The naming should be intuitive for majority of the tests. Complete
 description of FD.io CSIT test naming convention is provided on
 
 The naming should be intuitive for majority of the tests. Complete
 description of FD.io CSIT test naming convention is provided on
Integration tests