From cc58afc7de521e571fe4cbaa4c9841a2d3e9be52 Mon Sep 17 00:00:00 2001 From: Vratko Polak Date: Tue, 16 Nov 2021 17:32:50 +0100 Subject: [PATCH] Report: Improve docs related to TRex Change-Id: I0ed346ff30c61d28b5a2232ef2f9d32d26c1ae2c Signed-off-by: Vratko Polak --- docs/report/introduction/test_environment_changes_tg.rst | 2 +- docs/report/introduction/test_environment_intro.rst | 5 ++++- docs/report/trex_performance_tests/overview.rst | 11 ++++++----- docs/report/trex_performance_tests/throughput_trending.rst | 6 ++++-- 4 files changed, 15 insertions(+), 9 deletions(-) diff --git a/docs/report/introduction/test_environment_changes_tg.rst b/docs/report/introduction/test_environment_changes_tg.rst index 21175f2ba5..d21e43e04a 100644 --- a/docs/report/introduction/test_environment_changes_tg.rst +++ b/docs/report/introduction/test_environment_changes_tg.rst @@ -1,5 +1,5 @@ To identify performance changes due to TRex code development between previous -and current TRex version, both have been tested in CSIT environment of latest +and current TRex version, both have been tested in CSIT environment of latest version and compared against each other. All substantial progressions and regressions have been marked up with RCA analysis. See :ref:`trex_known_issues`. diff --git a/docs/report/introduction/test_environment_intro.rst b/docs/report/introduction/test_environment_intro.rst index 1999906669..c2feb1b4c4 100644 --- a/docs/report/introduction/test_environment_intro.rst +++ b/docs/report/introduction/test_environment_intro.rst @@ -10,9 +10,12 @@ CSIT test environment versioning has been introduced to track modifications of the test environment. Any benchmark anomalies (progressions, regressions) between releases of -a DUT application (e.g. VPP, DPDK, TRex), are determined by testing it in the +a DUT application (e.g. VPP, DPDK), are determined by testing it in the same test environment, to avoid test environment changes clouding the picture. +To beter distinguish impact of test environment changes, +we also execute tests without any SUT (just with TRex TG sending packets +over a link looping back to TG). A mirror approach is introduced to determine benchmarking anomalies due to the test environment change. This is achieved by testing the same DUT diff --git a/docs/report/trex_performance_tests/overview.rst b/docs/report/trex_performance_tests/overview.rst index 9740179df7..8ad8850ea8 100644 --- a/docs/report/trex_performance_tests/overview.rst +++ b/docs/report/trex_performance_tests/overview.rst @@ -1,14 +1,14 @@ Overview ======== -TREX performance test results are reported for a range of processors. -For description of physical testbeds used for TREX performance tests +TRex performance test results are reported for a range of processors. +For description of physical testbeds used for TRex performance tests please refer to :ref:`tested_physical_topologies`. Logical Topology ---------------- -CSIT TREX performance tests are executed on physical testbeds described +CSIT TRex performance tests are executed on physical testbeds described in :ref:`tested_physical_topologies`. Logical topology use 1 nic that has loopback connected ports. See figure below. @@ -52,11 +52,12 @@ topologies and configurations: - 10% of discovered PDR throughput. - Minimal offered load. -|csit-release| includes following TRex data plane functionality +|csit-release| includes tests using the following TRex traffic profiles +(corresponding to data plane functionality when DUT is used) performance tested across a range of NIC drivers and NIC models: +-----------------------+----------------------------------------------+ -| Functionality | Description | +| Traffic profile | Corresponding dataplane functionality | +=======================+==============================================+ | IPv4 Base | IPv4 routing. | +-----------------------+----------------------------------------------+ diff --git a/docs/report/trex_performance_tests/throughput_trending.rst b/docs/report/trex_performance_tests/throughput_trending.rst index cb2751972f..9106cb4bdf 100644 --- a/docs/report/trex_performance_tests/throughput_trending.rst +++ b/docs/report/trex_performance_tests/throughput_trending.rst @@ -1,12 +1,14 @@ Throughput Trending ------------------- -In addition to reporting throughput comparison between TRex releases, CSIT provides continuous performance trending for master branch: #. `TRex Trending Graphs `_: per TRex test case throughput trend, trend compliance and summary of - detected anomalies. + detected anomalies. We expect TRex to hit the curently used bps or pps limit, + so no anomalies here (unless we change those limits in CSIT). #. `TRex Latency Graphs `_: per TRex build NDRPDR latency measurements against the trendline. + We have seen in past that the latency numbers can depend on TRex version, + NIC firmware, or driver used. -- 2.16.6