X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Freport%2Fintroduction%2Fphysical_testbeds.rst;h=5f62f1bb063b93ed2beeff3943636a24ca3f1c11;hp=9babb5fb1f940f9c9487854fc222bc4de9a30388;hb=c10980893443b59c464deb6cd0d66d5be972593f;hpb=a10342c5b6d989f64bd1fe49f0edd0c7585e54ea diff --git a/docs/report/introduction/physical_testbeds.rst b/docs/report/introduction/physical_testbeds.rst index 9babb5fb1f..5f62f1bb06 100644 --- a/docs/report/introduction/physical_testbeds.rst +++ b/docs/report/introduction/physical_testbeds.rst @@ -26,9 +26,11 @@ Two physical server topology types are used: Current FD.io production testbeds are built with SUT servers based on the following processor architectures: -- Intel Xeon: Skylake Platinum 8180 and Haswell-SP E5-2699v3. +- Intel Xeon: Skylake Platinum 8180, Haswell-SP E5-2699v3, + Cascade Lake Platinum 8280, Cascade Lake 6252N. - Intel Atom: Denverton C3858. -- ARM: TaiShan 2280, hip07-d05. +- Arm: TaiShan 2280, hip07-d05. +- AMD EPYC: Zen2 7532. Server SUT performance depends on server and processor type, hence results for testbeds based on different servers must be reported @@ -40,13 +42,13 @@ https://git.fd.io/csit/tree/docs/lab/testbed_specifications.md. Following is the description of existing production testbeds. -2-Node Xeon Skylake (2n-skx) ----------------------------- +2-Node AMD EPYC Zen2 (2n-zn2) +----------------------------- -Four 2n-skx testbeds are in operation in FD.io labs. Each 2n-skx testbed -is built with two SuperMicro SYS-7049GP-TRT servers, each in turn -equipped with two Intel Xeon Skylake Platinum 8180 processors (38.5 MB -Cache, 2.50 GHz, 28 cores). 2n-skx physical topology is shown below. +One 2n-zn2 testbed in in operation in FD.io labs. It is built based on +two SuperMicro SuperMicro AS-1114S-WTRT servers, with SUT and TG servers +equipped with one AMD EPYC Zen2 7532 processor each (256 MB Cache, 2.40 +GHz, 32 cores). 2n-zn2 physical topology is shown below. .. only:: latex @@ -55,21 +57,63 @@ Cache, 2.50 GHz, 28 cores). 2n-skx physical topology is shown below. \begin{figure}[H] \centering \graphicspath{{../_tmp/src/introduction/}} - \includegraphics[width=0.90\textwidth]{testbed-2n-skx} - \label{fig:testbed-2n-skx} + \includegraphics[width=0.90\textwidth]{testbed-2n-zn2} + \label{fig:testbed-2n-zn2} \end{figure} .. only:: html - .. figure:: testbed-2n-skx.svg - :alt: testbed-2n-skx + .. figure:: testbed-2n-zn2.svg + :alt: testbed-2n-zn2 + :align: center + +SUT server is populated with the following NIC models: + +#. NIC-1: x710-DA4 4p10GE Intel. +#. NIC-2: xxv710-DA2 2p25GE Intel. +#. NIC-3: cx556a-edat ConnectX5 2p100GE Mellanox. + +TG server runs TRex application and is populated with the following +NIC models: + +#. NIC-1: x710-DA4 4p10GE Intel. +#. NIC-2: xxv710-DA2 2p25GE Intel. +#. NIC-3: cx556a-edat ConnectX5 2p100GE Mellanox. + +All AMD EPYC Zen2 7532 servers run with AMD SMT enabled, doubling the +number of logical cores exposed to Linux. + +2-Node Xeon Cascade Lake (2n-clx) +--------------------------------- + +Three 2n-clx testbeds are in operation in FD.io labs. Each 2n-clx testbed +is built with two SuperMicro SYS-7049GP-TRT servers, SUTs are equipped with two +Intel Xeon Gold 6252N processors (35.75 MB Cache, 2.30 GHz, 24 cores). +TGs are equiped with Intel Xeon Cascade Lake Platinum 8280 processors (38.5 MB +Cache, 2.70 GHz, 28 cores). 2n-clx physical topology is shown below. + +.. only:: latex + + .. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_tmp/src/introduction/}} + \includegraphics[width=0.90\textwidth]{testbed-2n-clx} + \label{fig:testbed-2n-clx} + \end{figure} + +.. only:: html + + .. figure:: testbed-2n-clx.svg + :alt: testbed-2n-clx :align: center SUT servers are populated with the following NIC models: #. NIC-1: x710-DA4 4p10GE Intel. #. NIC-2: xxv710-DA2 2p25GE Intel. -#. NIC-3: mcx556a-edat ConnectX5 2p100GE Mellanox. (Not used yet.) +#. NIC-3: cx556a-edat ConnectX5 2p100GE Mellanox. #. NIC-4: empty, future expansion. #. NIC-5: empty, future expansion. #. NIC-6: empty, future expansion. @@ -79,22 +123,21 @@ NIC models: #. NIC-1: x710-DA4 4p10GE Intel. #. NIC-2: xxv710-DA2 2p25GE Intel. -#. NIC-3: mcx556a-edat ConnectX5 2p100GE Mellanox. (Not used yet.) +#. NIC-3: cx556a-edat ConnectX5 2p100GE Mellanox. #. NIC-4: empty, future expansion. #. NIC-5: empty, future expansion. #. NIC-6: x710-DA4 4p10GE Intel. (For self-tests.) -All Intel Xeon Skylake servers run with Intel Hyper-Threading enabled, -doubling the number of logical cores exposed to Linux, with 56 logical -cores and 28 physical cores per processor socket. +All Intel Xeon Cascade Lake servers run with Intel Hyper-Threading enabled, +doubling the number of logical cores exposed to Linux. -3-Node Xeon Skylake (3n-skx) +2-Node Xeon Skylake (2n-skx) ---------------------------- -Two 3n-skx testbeds are in operation in FD.io labs. Each 3n-skx testbed -is built with three SuperMicro SYS-7049GP-TRT servers, each in turn +Four 2n-skx testbeds are in operation in FD.io labs. Each 2n-skx testbed +is built with two SuperMicro SYS-7049GP-TRT servers, each in turn equipped with two Intel Xeon Skylake Platinum 8180 processors (38.5 MB -Cache, 2.50 GHz, 28 cores). 3n-skx physical topology is shown below. +Cache, 2.50 GHz, 28 cores). 2n-skx physical topology is shown below. .. only:: latex @@ -103,17 +146,17 @@ Cache, 2.50 GHz, 28 cores). 3n-skx physical topology is shown below. \begin{figure}[H] \centering \graphicspath{{../_tmp/src/introduction/}} - \includegraphics[width=0.90\textwidth]{testbed-3n-skx} - \label{fig:testbed-3n-skx} + \includegraphics[width=0.90\textwidth]{testbed-2n-skx} + \label{fig:testbed-2n-skx} \end{figure} .. only:: html - .. figure:: testbed-3n-skx.svg - :alt: testbed-3n-skx + .. figure:: testbed-2n-skx.svg + :alt: testbed-2n-skx :align: center -SUT1 and SUT2 servers are populated with the following NIC models: +SUT servers are populated with the following NIC models: #. NIC-1: x710-DA4 4p10GE Intel. #. NIC-2: xxv710-DA2 2p25GE Intel. @@ -136,13 +179,13 @@ All Intel Xeon Skylake servers run with Intel Hyper-Threading enabled, doubling the number of logical cores exposed to Linux, with 56 logical cores and 28 physical cores per processor socket. -3-Node Xeon Haswell (3n-hsw) +3-Node Xeon Skylake (3n-skx) ---------------------------- -Three 3n-hsw testbeds are in operation in FD.io labs. Each 3n-hsw -testbed is built with three Cisco UCS-c240m3 servers, each in turn -equipped with two Intel Xeon Haswell-SP E5-2699v3 processors (45 MB -Cache, 2.3 GHz, 18 cores). 3n-hsw physical topology is shown below. +Two 3n-skx testbeds are in operation in FD.io labs. Each 3n-skx testbed +is built with three SuperMicro SYS-7049GP-TRT servers, each in turn +equipped with two Intel Xeon Skylake Platinum 8180 processors (38.5 MB +Cache, 2.50 GHz, 28 cores). 3n-skx physical topology is shown below. .. only:: latex @@ -151,38 +194,38 @@ Cache, 2.3 GHz, 18 cores). 3n-hsw physical topology is shown below. \begin{figure}[H] \centering \graphicspath{{../_tmp/src/introduction/}} - \includegraphics[width=0.90\textwidth]{testbed-3n-hsw} - \label{fig:testbed-3n-hsw} + \includegraphics[width=0.90\textwidth]{testbed-3n-skx} + \label{fig:testbed-3n-skx} \end{figure} .. only:: html - .. figure:: testbed-3n-hsw.svg - :alt: testbed-3n-hsw + .. figure:: testbed-3n-skx.svg + :alt: testbed-3n-skx :align: center SUT1 and SUT2 servers are populated with the following NIC models: -#. NIC-1: VIC 1385 2p40GE Cisco. -#. NIC-2: NIC x520 2p10GE Intel. -#. NIC-3: empty. -#. NIC-4: NIC xl710-QDA2 2p40GE Intel. -#. NIC-5: NIC x710-DA2 2p10GE Intel. -#. NIC-6: QAT 8950 50G (Walnut Hill) Intel. +#. NIC-1: x710-DA4 4p10GE Intel. +#. NIC-2: xxv710-DA2 2p25GE Intel. +#. NIC-3: empty, future expansion. +#. NIC-4: empty, future expansion. +#. NIC-5: empty, future expansion. +#. NIC-6: empty, future expansion. TG servers run T-Rex application and are populated with the following NIC models: -#. NIC-1: NIC xl710-QDA2 2p40GE Intel. -#. NIC-2: NIC x710-DA2 2p10GE Intel. -#. NIC-3: empty. -#. NIC-4: NIC xl710-QDA2 2p40GE Intel. -#. NIC-5: NIC x710-DA2 2p10GE Intel. -#. NIC-6: NIC x710-DA2 2p10GE Intel. (For self-tests.) +#. NIC-1: x710-DA4 4p10GE Intel. +#. NIC-2: xxv710-DA2 2p25GE Intel. +#. NIC-3: empty, future expansion. +#. NIC-4: empty, future expansion. +#. NIC-5: empty, future expansion. +#. NIC-6: x710-DA4 4p10GE Intel. (For self-tests.) -All Intel Xeon Haswell servers run with Intel Hyper-Threading disabled, -making the number of logical cores exposed to Linux match the number of -18 physical cores per processor socket. +All Intel Xeon Skylake servers run with Intel Hyper-Threading enabled, +doubling the number of logical cores exposed to Linux, with 56 logical +cores and 28 physical cores per processor socket. 2-Node Atom Denverton (2n-dnv) ------------------------------ @@ -295,8 +338,45 @@ SUT1 and SUT2 servers are populated with the following NIC models: #. NIC-1: connectx4 2p25GE Mellanox. #. NIC-2: x520 2p10GE Intel. -TG servers run T-Rex application and are populated with the following +TG server runs T-Rex application and is populated with the following NIC models: #. NIC-1: x710-DA4 4p10GE Intel. #. NIC-2: xxv710-DA2 2p25GE Intel. +#. NIC-3: xl710-QDA2 2p40GE Intel. + +2-Node ARM ThunderX2 (2n-tx2) +--------------------------- + +One 2n-tx2 testbed is built with: i) one SuperMicro SYS-7049GP-TRT +server acting as TG and equipped with two Intel Xeon Skylake Platinum +8180 processors (38.5 MB Cache, 2.50 GHz, 28 cores), and ii) one Marvell +ThnderX2 9975 (28* ThunderX2) server acting as SUT and equipped with two +ThunderX2 ARMv8 CN9975 processors. 2n-tx2 physical topology is shown below. + +.. only:: latex + + .. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_tmp/src/introduction/}} + \includegraphics[width=0.90\textwidth]{testbed-2n-tx2} + \label{fig:testbed-2n-tx2} + \end{figure} + +.. only:: html + + .. figure:: testbed-2n-tx2.svg + :alt: testbed-2n-tx2 + :align: center + +SUT server is populated with the following NIC models: + +#. NIC-1: xl710-QDA2 2p40GE Intel (not connected). +#. NIC-2: xl710-QDA2 2p40GE Intel. + +TG server run T-Rex application and is populated with the following +NIC models: + +#. NIC-1: xl710-QDA2 2p40GE Intel.