X-Git-Url: https://gerrit.fd.io/r/gitweb?a=blobdiff_plain;f=docs%2Freport%2Fintroduction%2Fmethodology_data_plane_throughput%2Fmethodology_data_plane_throughput.rst;h=24e76ef3912f106b9f03786f22d2920394871251;hb=6809aeefd0f2b951cd4861f676ee2dff0df47ff2;hp=202b4281b7a7a0bc3289f0ec71242827ea2ca6c5;hpb=b297d7fca14caa89c24ff3e34cbe58f40aba7c88;p=csit.git diff --git a/docs/report/introduction/methodology_data_plane_throughput/methodology_data_plane_throughput.rst b/docs/report/introduction/methodology_data_plane_throughput/methodology_data_plane_throughput.rst index 202b4281b7..24e76ef391 100644 --- a/docs/report/introduction/methodology_data_plane_throughput/methodology_data_plane_throughput.rst +++ b/docs/report/introduction/methodology_data_plane_throughput/methodology_data_plane_throughput.rst @@ -1,5 +1,7 @@ +.. _data_plane_throughput: + Data Plane Throughput Tests ---------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^ Network data plane throughput is measured using multiple test methods in order to obtain representative and repeatable results across the large @@ -11,14 +13,18 @@ Following throughput test methods are used: - MRR - Maximum Receive Rate - PLRsearch - Probabilistic Loss Ratio search +.. + TODO: Add RECONF. + TODO: Link to method-specific pages instead of duplicate info below. + Description of each test method is followed by generic test properties shared by all methods. MLRsearch Tests -^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~ Description -~~~~~~~~~~~ +``````````` Multiple Loss Ratio search (MLRsearch) tests discover multiple packet throughput rates in a single search, reducing the overall test execution @@ -29,7 +35,7 @@ and Partial Drop Rate (PDR, with PLR<0.5%). MLRsearch is compliant with :rfc:`2544`. Usage -~~~~~ +````` MLRsearch tests are run to discover NDR and PDR rates for each VPP and DPDK release covered by CSIT report. Results for small frame sizes @@ -44,17 +50,17 @@ all frame sizes and for all tests are presented in detailed results tables. Details -~~~~~~~ +``````` See :ref:`mlrsearch_algorithm` section for more detail. MLRsearch is -being standardized in IETF in `draft-vpolak-mkonstan-mlrsearch -`_. +being standardized in IETF in `draft-ietf-bmwg-mlrsearch +`_. MRR Tests -^^^^^^^^^ +~~~~~~~~~ Description -~~~~~~~~~~~ +``````````` Maximum Receive Rate (MRR) tests are complementary to MLRsearch tests, as they provide a maximum “raw” throughput benchmark for development and @@ -66,7 +72,7 @@ a set trial duration, regardless of packet loss. Maximum load for specified Ethernet frame size is set to the bi-directional link rate. Usage -~~~~~ +````` MRR tests are much faster than MLRsearch as they rely on a single trial or a small set of trials with very short duration. It is this property @@ -75,27 +81,27 @@ trending jobs enabling detection of performance anomalies (regressions, progressions) resulting from data plane code changes. MRR tests are also used for VPP per patch performance jobs verifying -patch performance vs. parent. CSIT reports include MRR throughput +patch performance vs parent. CSIT reports include MRR throughput comparisons between releases and test environments. Small frame sizes only (64b/78B, IMIX). Details -~~~~~~~ +``````` See :ref:`mrr_throughput` section for more detail about MRR tests configuration. FD.io CSIT performance dashboard includes complete description of `daily performance trending tests -`_ +`_ and `VPP per patch tests -`_. +`_. PLRsearch Tests -^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~ Description -~~~~~~~~~~~ +``````````` Probabilistic Loss Ratio search (PLRsearch) tests discovers a packet throughput rate associated with configured Packet Loss Ratio (PLR) @@ -104,25 +110,25 @@ testing. PLRsearch assumes that system under test is probabilistic in nature, and not deterministic. Usage -~~~~~ +````` PLRsearch are run to discover a sustained throughput for PLR=10^-7 (close to NDR) for VPP release covered by CSIT report. Results for small frame sizes (64b/78B) are presented in packet throughput graphs (Box Plots) for a small subset of baseline tests. -Each soak test lasts 2hrs and is executed at least twice. Results are +Each soak test lasts 30 minutes and is executed at least twice. Results are compared against NDR and PDR rates discovered with MLRsearch. Details -~~~~~~~ +``````` See :ref:`plrsearch` methodology section for more detail. PLRsearch is being standardized in IETF in `draft-vpolak-bmwg-plrsearch `_. Generic Test Properties -^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~ All data plane throughput test methodologies share following generic properties: @@ -138,3 +144,8 @@ properties: - All measured and reported packet and bandwidth rates are aggregate bi-directional rates reported from external Traffic Generator perspective. + +.. + TODO: Incorporate ASTF specifics: No IMIX, transactions instead of packets, + slightly non-symmetric traffic with TCP profiles, unsure max_rate. + TODO: Mention latency.