From f85930d7b1e3cda191a2e60cbc4e64d93c5fd139 Mon Sep 17 00:00:00 2001 From: Maciek Konstantynowicz Date: Thu, 8 Jul 2021 15:47:53 +0100 Subject: [PATCH] report: add 2n-icx and 3n-icx testbed documentation Add: - Physical testbeds description in Introduction. - Add physical topology diagrams. - VPP pre-release 21.06 version used in Intel labs for CSIT tests. Change-Id: I3c17402de8c657f66ea840c9d8692212cbf0501a Signed-off-by: Maciek Konstantynowicz --- .../introduction/methodology_packet_latency.rst | 2 + docs/report/introduction/physical_testbeds.rst | 82 +- docs/report/introduction/testbed-2n-icx.svg | 1893 +++++++++++++ docs/report/introduction/testbed-3n-icx.svg | 2842 ++++++++++++++++++++ .../vpp_performance_tests/csit_release_notes.rst | 19 +- 5 files changed, 4829 insertions(+), 9 deletions(-) create mode 100644 docs/report/introduction/testbed-2n-icx.svg create mode 100644 docs/report/introduction/testbed-3n-icx.svg diff --git a/docs/report/introduction/methodology_packet_latency.rst b/docs/report/introduction/methodology_packet_latency.rst index d5486786b6..35e2aad029 100644 --- a/docs/report/introduction/methodology_packet_latency.rst +++ b/docs/report/introduction/methodology_packet_latency.rst @@ -1,3 +1,5 @@ +.. _latency_methodology: + Packet Latency -------------- diff --git a/docs/report/introduction/physical_testbeds.rst b/docs/report/introduction/physical_testbeds.rst index 7755bdddeb..fbc16eeaa3 100644 --- a/docs/report/introduction/physical_testbeds.rst +++ b/docs/report/introduction/physical_testbeds.rst @@ -26,12 +26,18 @@ Two physical server topology types are used: Current FD.io production testbeds are built with SUT servers based on the following processor architectures: -- Intel Xeon: Skylake Platinum 8180, Cascade Lake Platinum 8280, - Cascade Lake 6252N. +- Intel Xeon: Skylake Platinum 8180, Cascadelake 6252N, (Icelake 8358 + installation in progress). - Intel Atom: Denverton C3858. - Arm: TaiShan 2280, hip07-d05. - AMD EPYC: Zen2 7532. +CSIT-2106 report data for Intel Xeon Icelake testbeds comes from +testbeds in Intel labs set up per CSIT specification and running CSIT +code. Physical setup used is specified in 2n-icx and 3n-icx sections +below. For details about tested VPP and CSIT versions +see :ref:`vpp_performance_tests_release_notes`. + Server SUT performance depends on server and processor type, hence results for testbeds based on different servers must be reported separately, and compared if appropriate. @@ -83,7 +89,7 @@ NIC models: All AMD EPYC Zen2 7532 servers run with AMD SMT enabled, doubling the number of logical cores exposed to Linux. -2-Node Xeon Cascade Lake (2n-clx) +2-Node Xeon Cascadelake (2n-clx) --------------------------------- Three 2n-clx testbeds are in operation in FD.io labs. Each 2n-clx testbed @@ -128,7 +134,75 @@ NIC models: #. NIC-5: empty, future expansion. #. NIC-6: x710-DA4 4p10GE Intel. (For self-tests.) -All Intel Xeon Cascade Lake servers run with Intel Hyper-Threading enabled, +All Intel Xeon Cascadelake servers run with Intel Hyper-Threading enabled, +doubling the number of logical cores exposed to Linux. + +2-Node Xeon Icelake (2n-icx) EXPERIMENTAL +----------------------------------------- + +One 2n-icx testbed located in Intel labs was used for CSIT testing. It +is built with two SuperMicro SYS-740GP-TNRT servers. SUT is equipped +with two Intel Xeon Gold 6338N processors (48 MB Cache, 2.20 GHz, 32 +cores). TG is equiped with two Intel Xeon Ice Lake Platinum 8360Y +processors (54 MB Cache, 2.40 GHz, 36 cores). 2n-icx physical topology +is shown below. + +.. only:: latex + + .. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_tmp/src/introduction/}} + \includegraphics[width=0.90\textwidth]{testbed-2n-icx} + \label{fig:testbed-2n-icx} + \end{figure} + +.. only:: html + + .. figure:: testbed-2n-icx.svg + :alt: testbed-2n-icx + :align: center + +SUT and TG servers are populated with the following NIC models: + +#. NIC-1: E810-2CQDA2 2p100GbE Intel. + +All Intel Xeon Icelake servers run with Intel Hyper-Threading enabled, +doubling the number of logical cores exposed to Linux. + +3-Node Xeon Icelake (3n-icx) EXPERIMENTAL +----------------------------------------- + +One 3n-icx testbed located in Intel labs was used for CSIT testing. It +is built with three SuperMicro SYS-740GP-TNRT servers. SUTs are +equipped each with two Intel Xeon Platinum 8360Y processors (54 MB +Cache, 2.40 GHz, 36 cores). TG is equiped with two Intel Xeon Ice Lake +Platinum 8360Y processors (54 MB Cache, 2.40 GHz, 36 cores). 3n-icx +physical topology is shown below. + +.. only:: latex + + .. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_tmp/src/introduction/}} + \includegraphics[width=0.90\textwidth]{testbed-3n-icx} + \label{fig:testbed-3n-icx} + \end{figure} + +.. only:: html + + .. figure:: testbed-3n-icx.svg + :alt: testbed-3n-icx + :align: center + +SUT and TG servers are populated with the following NIC models: + +#. NIC-1: E810-2CQDA2 2p100GbE Intel. + +All Intel Xeon Icelake servers run with Intel Hyper-Threading enabled, doubling the number of logical cores exposed to Linux. 2-Node Xeon Skylake (2n-skx) diff --git a/docs/report/introduction/testbed-2n-icx.svg b/docs/report/introduction/testbed-2n-icx.svg new file mode 100644 index 0000000000..ecb99042ce --- /dev/null +++ b/docs/report/introduction/testbed-2n-icx.svg @@ -0,0 +1,1893 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x86Server + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC6 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Socket 1Intel XeonPlatinum 8360Y + + + + + + + + + NIC5 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC4 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + Socket 0Intel XeonPlatinum 8360Y + + + + + + + + + NIC3 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC2 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + + + + + x86Server + + + + + + + + + + + + + + + + 2-Node Xeon Icelake (2n-icx) + + + + + + Traffic Generator (TG) + + + + + + DDR4 + + + + + + PCIeGen4 + + + + + + + + + NIC1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Socket 0Intel Xeon Gold 6338N + + + + + + + + + NIC2 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC3 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + DDR4 + + + + + + + + Socket 1Intel XeonGold 6338N + + + + + + + + + NIC4 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC5 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC6 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + + + + + + + + + + + System Under Test (SUT) + + + + + + PCIeGen4 + + + + + + + UPI + + + + + + + + + + + + + + UPI + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/report/introduction/testbed-3n-icx.svg b/docs/report/introduction/testbed-3n-icx.svg new file mode 100644 index 0000000000..2ab61e2179 --- /dev/null +++ b/docs/report/introduction/testbed-3n-icx.svg @@ -0,0 +1,2842 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 3-Node Xeon Icelake (3n-icx) + + + + + + + + + + + + + + + + NIC6 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Socket 1Intel XeonPlatinum 8360Y + + + + + + + + + NIC5 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC4 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + Socket 0Intel XeonPlatinum 8360Y + + + + + + + + + NIC3 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC2 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + + + + + x86Server + + + + + + Traffic Generator (TG) + + + + + + DDR4 + + + + + + PCIeGen4 + + + + + + + + + + + + + + UPI + + + + + + + + + x86Server + + + + + + + + + NIC1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Socket 0Intel Xeon Platinum 8360Y + + + + + + + + + NIC2 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC3 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + DDR4 + + + + + + + + Socket 1Intel XeonPlatinum 8360Y + + + + + + + + + NIC4 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC5 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC6 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + + + + + + + + + + + System Under Test 1 (SUT1) + + + + + + PCIeGen4 + + + + + + + UPI + + + + + + + + x86Server + + + + + + + + + NIC1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Socket 0Intel Xeon Platinum 8360Y + + + + + + + + + NIC2 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC3 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + DDR4 + + + + + + + + Socket 1Intel XeonPlatinum 8360Y + + + + + + + + + NIC4 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC5 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NIC6 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + x16 + + + + + + + + + + + + + + + + + + + + + + + + + System Under Test 2 (SUT2) + + + + + + PCIeGen4 + + + + + + + UPI + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 3129340595..a0777918da 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -1,3 +1,5 @@ +.. _vpp_performance_tests_release_notes: + Release Notes ============= @@ -9,16 +11,23 @@ Changes in |csit-release| - CSIT test environment is versioned, see :ref:`test_environment_versioning`. - - **Upgrade to Ubuntu 20.04 LTS**: Reinstall base operating system to Ubuntu - 20.04.2 LTS. Upgrade includes also baseline Docker containers used for - spawning topology. + - **Upgrade to Ubuntu 20.04 LTS**: Reinstall base operating system to + Ubuntu 20.04.2 LTS. Upgrade includes also baseline Docker + containers used for spawning topology. + + - Initial test data for Intel Xeon Ice Lake platforms. Current + CSIT-2106 report data for Intel Xeon Ice Lake comes from an + external source (Intel labs running CSIT code on 8360Y D Stepping + and 6338N processors). For detail about the physical setup + see :ref:`tested_physical_topologies`. Tested VPP and CSIT + versions are pre-release, VPP 21.06-rc0~779-gd640ae52f. - **AF_XDP**: Added af_xdp driver testing for all testcases. - **GTPU tunnel**: Added GTPU HW Offload IPv4 routing tests. - - **Telemetry retouch**: Redesign telemetry retrieval from DUT. Include - VPP perfmon plugin telemetry. + - **Telemetry retouch**: Redesign telemetry retrieval from DUT. + Include VPP perfmon plugin telemetry. #. TEST FRAMEWORK -- 2.16.6