Code Review / csit.git / blobdiff
? search:
summary | shortlog | log | commit | commitdiff | review | tree
raw | inline | side by side
docs: TRex static documentation
[csit.git] / docs / report / introduction / test_environment_intro.rst
diff --git a/docs/report/introduction/test_environment_intro.rst b/docs/report/introduction/test_environment_intro.rst
index c35cba3..1999906 100644 (file)
--- a/docs/report/introduction/test_environment_intro.rst
+++ b/docs/report/introduction/test_environment_intro.rst
@@ -1,6 +1,8 @@
 Test Environment
 ================
 
 Test Environment
 ================
 
+.. _test_environment_versioning:
+
 Environment Versioning
 ----------------------
 
 Environment Versioning
 ----------------------
 
@@ -8,7 +10,7 @@ CSIT test environment versioning has been introduced to track
 modifications of the test environment.
 
 Any benchmark anomalies (progressions, regressions) between releases of
 modifications of the test environment.
 
 Any benchmark anomalies (progressions, regressions) between releases of
-a DUT application (e.g. VPP, DPDK), are determined by testing it in the
+a DUT application (e.g. VPP, DPDK, TRex), are determined by testing it in the
 same test environment, to avoid test environment changes clouding the
 picture.
 
 same test environment, to avoid test environment changes clouding the
 picture.
 
@@ -30,12 +32,12 @@ included in the CSIT environment versioning include:
   <https://git.fd.io/csit/tree/docs/lab/testbeds_sm_skx_hw_bios_cfg.md#n556>`_.
 - **Linux** Server Linux OS version and configuration, tracked in CSIT
   Reports in `SUT Settings
   <https://git.fd.io/csit/tree/docs/lab/testbeds_sm_skx_hw_bios_cfg.md#n556>`_.
 - **Linux** Server Linux OS version and configuration, tracked in CSIT
   Reports in `SUT Settings
-  <https://docs.fd.io/csit/master/report/vpp_performance_tests/test_environment.html#sut-settings-linux>`_
+  <https://s3-docs.fd.io/csit/master/report/vpp_performance_tests/test_environment.html#sut-settings-linux>`_
   and `Pre-Test Server Calibration
   and `Pre-Test Server Calibration
-  <https://docs.fd.io/csit/master/report/vpp_performance_tests/test_environment.html#pre-test-server-calibration>`_.
+  <https://s3-docs.fd.io/csit/master/report/vpp_performance_tests/test_environment.html#id21>`_.
 - **TRex** TRex Traffic Generator version, drivers and configuration
   tracked in `TG Settings
 - **TRex** TRex Traffic Generator version, drivers and configuration
   tracked in `TG Settings
-  <https://docs.fd.io/csit/master/report/vpp_performance_tests/test_environment.html#tg-settings-trex>`_.
+  <https://s3-docs.fd.io/csit/master/report/vpp_performance_tests/test_environment.html#tg-settings-trex>`_.
 - **CSIT** CSIT framework code tracked in CSIT release branches.
 
 Following is the list of CSIT versions to date:
 - **CSIT** CSIT framework code tracked in CSIT release branches.
 
 Following is the list of CSIT versions to date:
@@ -58,40 +60,42 @@ Following is the list of CSIT versions to date:
   `TRex
   <https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/test_environment.html#tg-settings-trex>`_,
   `CSIT <https://git.fd.io/csit/tree/?h=rls2005>`_).
   `TRex
   <https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/test_environment.html#tg-settings-trex>`_,
   `CSIT <https://git.fd.io/csit/tree/?h=rls2005>`_).
+- Ver. 5 associated with CSIT rls2009 branch (`HW
+  <https://git.fd.io/csit/tree/docs/lab?h=rls2009>`_, `Linux
+  <https://docs.fd.io/csit/rls2009/report/vpp_performance_tests/test_environment.html#sut-settings-linux>`_,
+  `TRex
+  <https://docs.fd.io/csit/rls2009/report/vpp_performance_tests/test_environment.html#tg-settings-trex>`_,
+  `CSIT <https://git.fd.io/csit/tree/?h=rls2009>`_).
+
+  - The main change is TRex data-plane core resource adjustments:
+    `increase from 7 to 8 cores and pinning cores to interfaces <https://gerrit.fd.io/r/c/csit/+/28184>`_
+    for better TRex performance with symmetric traffic profiles.
+- Ver. 6 associated with CSIT rls2101 branch (`HW
+  <https://git.fd.io/csit/tree/docs/lab?h=rls2101>`_, `Linux
+  <https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/test_environment.html#sut-settings-linux>`_,
+  `TRex
+  <https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/test_environment.html#tg-settings-trex>`_,
+  `CSIT <https://git.fd.io/csit/tree/?h=rls2101>`_).
+
+  - The main change is TRex version upgrade:
+    `increase from 2.82 to 2.86 <https://gerrit.fd.io/r/c/csit/+/29980>`_.
+- Ver. 7 associated with CSIT rls2106 branch (`HW
+  <https://git.fd.io/csit/tree/docs/lab?h=rls2106>`_, `Linux
+  <https://s3-docs.fd.io/csit/rls2106/report/vpp_performance_tests/test_environment.html#sut-settings-linux>`_,
+  `TRex
+  <https://s3-docs.fd.io/csit/rls2106/report/vpp_performance_tests/test_environment.html#tg-settings-trex>`_,
+  `CSIT <https://git.fd.io/csit/tree/?h=rls2106>`_).
+
+  - TRex version upgrade:
+    `increase from 2.86 to 2.88 <https://gerrit.fd.io/r/c/csit/+/31652>`_.
+  - Ubuntu upgrade:
+    `upgrade from 18.04 LTS to 20.04.2 LTS <https://gerrit.fd.io/r/c/csit/+/31290>`_.
+- Ver. 8 associated with CSIT rls2110 branch (`HW
+  <https://git.fd.io/csit/tree/docs/lab?h=rls2110>`_, `Linux
+  <https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/test_environment.html#sut-settings-linux>`_,
+  `TRex
+  <https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/test_environment.html#tg-settings-trex>`_,
+  `CSIT <https://git.fd.io/csit/tree/?h=rls2110>`_).
 
 
-To identify performance changes due to VPP code development from
-v20.01.0 to v20.05.0, both have been tested in CSIT environment ver. 4
-and compared against each other. All substantial progressions and
-regressions have been marked up with RCA analysis.
-:ref:`vpp_throughput_comparisons` and :ref:`vpp_known_issues`.
-
-CSIT environment ver. 4 has been evaluated against the ver. 2 by
-benchmarking VPP v20.01.0 in both environment versions.
-
-Physical Testbeds
------------------
-
-FD.io CSIT performance tests are executed in physical testbeds hosted by
-:abbr:`LF (Linux Foundation)` for FD.io project. Two physical testbed
-topology types are used:
-
-- **3-Node Topology**: Consisting of two servers acting as SUTs
-  (Systems Under Test) and one server as TG (Traffic Generator), all
-  connected in ring topology.
-- **2-Node Topology**: Consisting of one server acting as SUTs and one
-  server as TG both connected in ring topology.
-
-Tested SUT servers are based on a range of processors including Intel
-Xeon Haswell-SP, Intel Xeon Skylake-SP, Intel Xeon Cascade Lake-SP, Arm,
-Intel Atom. More detailed description is provided in
-:ref:`tested_physical_topologies`. Tested logical topologies are
-described in :ref:`tested_logical_topologies`.
-
-Server Specifications
----------------------
-
-Complete technical specifications of compute servers used in CSIT
-physical testbeds are maintained in FD.io CSIT repository:
-`FD.io CSIT testbeds - Xeon Cascade Lake`_,
-`FD.io CSIT testbeds - Xeon Skylake, Arm, Atom`_ and
-`FD.io CSIT Testbeds - Xeon Haswell`_.
+  - Intel NIC 700/800 series firmware upgrade based on DPDK compatibility
+    matrix: `depends on testbed type <https://gerrit.fd.io/r/c/csit/+/33311>`_.
Integration tests