From 0546e1656070b734ec57a4d8f732392bcafee0df Mon Sep 17 00:00:00 2001 From: Maciek Konstantynowicz Date: Fri, 17 Aug 2018 22:40:16 +0100 Subject: [PATCH] report 18.07: draft v0.9 editorial nit-picking. Change-Id: I7fb9dff8f5ebce0ac03ef99c74b14b02638042ac Signed-off-by: Maciek Konstantynowicz --- .../dmm_functional_tests/csit_release_notes.rst | 2 +- .../comparisons/3n-skx_vs_3n-hsw_testbeds.rst | 47 +++++------------ .../comparisons/current_vs_previous_release.rst | 46 +++++++--------- .../dpdk_performance_tests/csit_release_notes.rst | 10 ++-- .../dpdk_performance_tests/throughput_trending.rst | 2 +- docs/report/introduction/introduction.rst | 42 ++++++--------- .../introduction/test_scenarios_overview.rst | 8 +-- .../vpp_functional_tests/csit_release_notes.rst | 4 +- .../comparisons/3n-skx_vs_3n-hsw_testbeds.rst | 47 +++++------------ .../comparisons/current_vs_previous_release.rst | 61 ++++++++-------------- .../vpp_performance_tests/csit_release_notes.rst | 10 ++-- .../vpp_performance_tests/throughput_trending.rst | 4 +- 12 files changed, 100 insertions(+), 183 deletions(-) diff --git a/docs/report/dmm_functional_tests/csit_release_notes.rst b/docs/report/dmm_functional_tests/csit_release_notes.rst index 28c5b63936..7cebeda12d 100644 --- a/docs/report/dmm_functional_tests/csit_release_notes.rst +++ b/docs/report/dmm_functional_tests/csit_release_notes.rst @@ -4,7 +4,7 @@ Release Notes Changes in |csit-release| ------------------------- -#. **DMM Functional Tests** +#. DMM FUNCTIONAL TESTS - Added DMM base vs. epoll test case. diff --git a/docs/report/dpdk_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst b/docs/report/dpdk_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst index c148b0619c..cbaa749e0e 100644 --- a/docs/report/dpdk_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst +++ b/docs/report/dpdk_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst @@ -6,15 +6,6 @@ throughput (NDR, PDR and MRR) is calculated for the same tests executed on 3-Node Skylake (3n-skx) and 3-Node Haswell (3n-hsw) physical testbed types, in 1-core, 2-core and 4-core configurations. -NDR Comparison -~~~~~~~~~~~~~~ - -NDR throughput comparison between testbeds is available in CSV and -pretty ASCII formats: - - - `CSV NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_, - - `ASCII NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_. - .. note:: Test results have been generated by @@ -24,38 +15,26 @@ pretty ASCII formats: files csit-dpdk-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. -PDR Comparison +NDR Comparison ~~~~~~~~~~~~~~ -PDR throughput comparison between testbeds is available in CSV and -pretty ASCII formats: +Comparison tables in ASCII and CSV formats: - - `CSV PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_, - - `ASCII PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_. + - `ASCII NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_ + - `CSV NDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_ -.. note:: +PDR Comparison +~~~~~~~~~~~~~~ - Test results have been generated by - `FD.io test executor dpdk performance job 3n-hsw`_ and - `FD.io test executor dpdk performance job 3n-skx`_ - with RF result - files csit-dpdk-perf-|srelease|-\*.zip - `archived here <../_static/archive/>`_. +Comparison tables in ASCII and CSV formats: + + - `ASCII PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_ + - `CSV PDR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_ MRR Comparison ~~~~~~~~~~~~~~ -MRR throughput comparison between testbeds is available in CSV and -pretty ASCII formats: - - - `CSV MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_, - - `ASCII MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_. +Comparison tables in ASCII and CSV formats: -.. note:: - - Test results have been generated by - `FD.io test executor dpdk performance job 3n-hsw`_ and - `FD.io test executor dpdk performance job 3n-skx`_ - with RF result - files csit-vpp-perf-|srelease|-\*.zip - `archived here <../_static/archive/>`_. \ No newline at end of file + - `ASCII MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_ + - `CSV MRR comparison <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_ diff --git a/docs/report/dpdk_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/dpdk_performance_tests/comparisons/current_vs_previous_release.rst index 6a7829ac8a..6ca7ede318 100644 --- a/docs/report/dpdk_performance_tests/comparisons/current_vs_previous_release.rst +++ b/docs/report/dpdk_performance_tests/comparisons/current_vs_previous_release.rst @@ -2,27 +2,16 @@ Current vs. Previous Release ---------------------------- Relative comparison of DPDK Testpmd and L3fwd packet throughput (NDR, -PDR and MRR) between |dpdk-release| and DPDK-18.02 (measured for |csit- -release| and |csit-release-1| respectively) is calculated from results -of tests running on 3-Node Intel Xeon Haswell testbeds (3n-hsw) in -1-core, 2-core and 4-core (MRR only) configurations. +PDR and MRR) between |dpdk-release| and DPDK-18.02 (measured for +|csit-release| and |csit-release-1| respectively) is calculated from +results of tests running on 3-Node Intel Xeon Haswell testbeds (3n-hsw) +in 1-core and 2-core configurations. Listed mean and standard deviation values are computed based on a series of the same tests executed against respective DPDK releases to verify test results repeatability, with percentage change calculated for mean values. -NDR Comparison -~~~~~~~~~~~~~~ - -NDR throughput comparison between releases is available in CSV and pretty -ASCII formats: - - - `CSV 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_, - - `CSV 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_, - - `ASCII 1t1c NDR changes <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_, - - `ASCII 2t2c NDR changes <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_. - .. note:: Test results have been generated by @@ -31,21 +20,22 @@ ASCII formats: files csit-dpdk-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. -PDR Comparison +NDR Comparison ~~~~~~~~~~~~~~ -PDR throughput comparison between releases is available in CSV and pretty -ASCII formats: +Comparison tables in ASCII and CSV formats: - - `CSV 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_, - - `CSV 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_, - - `ASCII 1t1c PDR changes <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_, - - `ASCII 2t2c PDR changes <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_. + - `ASCII 1t1c NDR comparison <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_ + - `ASCII 2t2c NDR comparison <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_ + - `CSV 1t1c NDR comparison <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_ + - `CSV 2t2c NDR comparison <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_ -.. note:: +PDR Comparison +~~~~~~~~~~~~~~ - Test results have been generated by - `FD.io test executor dpdk performance job 3n-hsw`_ - with RF result - files csit-dpdk-perf-|srelease|-\*.zip - `archived here <../_static/archive/>`_. +Comparison tables in ASCII and CSV formats: + + - `ASCII 1t1c PDR comparison <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_ + - `ASCII 2t2c PDR comparison <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_ + - `CSV 1t1c PDR comparison <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_ + - `CSV 2t2c PDR comparison <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_ diff --git a/docs/report/dpdk_performance_tests/csit_release_notes.rst b/docs/report/dpdk_performance_tests/csit_release_notes.rst index e0e20289b0..26917432d5 100644 --- a/docs/report/dpdk_performance_tests/csit_release_notes.rst +++ b/docs/report/dpdk_performance_tests/csit_release_notes.rst @@ -4,11 +4,11 @@ Release Notes Changes in |csit-release| ------------------------- -#. **DPDK release version change** +#. DPDK RELEASE VERSION CHANGE - - |csit-release| tested DPDK-18.05, as used by |vpp-release|. + - |csit-release| tested |dpdk-release|, as used by |vpp-release|. -#. **DPDK performance tests** +#. DPDK PERFORMANCE TESTS - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now configurable trial duration and number of consecutive executions. @@ -32,9 +32,9 @@ Changes in |csit-release| cores combinations. All existing test cases were converted to new format. -#. **Presentation and Analytics Layer** +#. PRESENTATION AND ANALYTICS LAYER - - **Performance trending**: Added continuous performance trending with + - **Performance trending**: Added regular performance trending, anomaly detection and analysis for DPDK. Known Issues diff --git a/docs/report/dpdk_performance_tests/throughput_trending.rst b/docs/report/dpdk_performance_tests/throughput_trending.rst index 2b45f4bdaf..961d3ea7a1 100644 --- a/docs/report/dpdk_performance_tests/throughput_trending.rst +++ b/docs/report/dpdk_performance_tests/throughput_trending.rst @@ -12,6 +12,6 @@ CSIT provides regular performance trending for DPDK release branches: - throughput test metrics, trend calculations and anomaly classification (progression, regression). -#. `DPDK Apps Trendline Graphs `_ +#. `DPDK Trendline Graphs `_ - weekly DPDK Testpmd and L3fwd MRR throughput measurements against the trendline with anomaly highlights and associated CSIT test jobs. \ No newline at end of file diff --git a/docs/report/introduction/introduction.rst b/docs/report/introduction/introduction.rst index e5fa19b08d..e08edc0486 100644 --- a/docs/report/introduction/introduction.rst +++ b/docs/report/introduction/introduction.rst @@ -1,26 +1,14 @@ Introduction ============ -This is FD.io CSIT (Fast Data i/o Continuous System Integration and -Testing) project report for |csit-release| system performance and -functional testing of |vpp-release|. +FD.io |csit-release| report contains system performance and functional +testing data for |vpp-release|. -There is also a downloadable `PDF version of this report`_. +`PDF version of this report`_ is also available for download. -This report describes CSIT performance and functional tests delivered in -|csit-release|. A high-level overview is provided for each CSIT test -environment running in :abbr:`LF (Linux Foundation)` FD.io Continuous -Performance Labs. This is followed by summary of all executed tests -against the |vpp-release| and associated FD.io projects and sub-systems -(Honeycomb, DPDK, NSH_SFC, DMM), |csit-release| release notes, result -highlights and known issues discovered in CSIT. More detailed -description of each environment, pointers to CSIT test code -documentation and detailed test resuls with links to the source data -files are also provided. +|csit-release| report is structured as follows: -|csit-release| report contains following main sections and sub-sections: - -#. **Introduction**: General introduction to CSIT project. +#. INTRODUCTION: General introduction to CSIT project. - **Introduction**: This section. - **Test Scenarios Overview**: A brief overview of test scenarios @@ -28,7 +16,7 @@ files are also provided. - **Physical Testbeds**: Description of physical testbeds. - **Performance Test Methodology**: Benchmarking methodologies. -#. **VPP Performance**: VPP performance tests executed in physical +#. VPP PERFORMANCE: VPP performance tests executed in physical FD.io testbeds. - **Overview**: Tested logical topologies, test coverage and naming @@ -53,7 +41,7 @@ files are also provided. in CSIT and pointers to CSIT source code documentation for VPP performance tests. -#. **DPDK Performance**: DPDK performance tests executed in physical +#. DPDK PERFORMANCE: DPDK performance tests executed in physical FD.io testbeds. - **Overview**: Tested logical topologies, test coverage. @@ -71,7 +59,7 @@ files are also provided. - **Documentation**: Pointers to CSIT source code documentation for DPDK performance tests. -#. **VPP Functional**: VPP functional tests executed in virtual FD.io +#. VPP FUNCTIONAL: VPP functional tests executed in virtual FD.io testbeds. - **Overview**: Tested virtual topologies, test coverage and naming @@ -82,7 +70,7 @@ files are also provided. - **Documentation**: Pointers to CSIT source code documentation for VPP functional tests. -#. **Honeycomb Functional**: Honeycomb functional tests executed in +#. HONEYCOMBE FUNCTIONAL: Honeycomb functional tests executed in virtual FD.io testbeds. - **Overview**: Tested virtual topologies, test coverage and naming @@ -92,7 +80,7 @@ files are also provided. - **Documentation**: Pointers to CSIT source code documentation for Honeycomb functional tests. -#. **NSH_SFC Functional**: NSH_SFC functional tests executed in +#. NSH_SFC FUNCTIONAL: NSH_SFC functional tests executed in virtual FD.io testbeds. - **Overview**: Tested virtual topologies, test coverage and naming @@ -102,7 +90,7 @@ files are also provided. - **Documentation**: Pointers to CSIT source code documentation for NSH_SFC functional tests. -#. **DMM Functional**: DMM functional tests executed in +#. DMM FUNCTIONAL: DMM functional tests executed in virtual FD.io testbeds. - **Overview**: Tested virtual topologies, test coverage and naming @@ -112,7 +100,7 @@ files are also provided. - **Documentation**: Pointers to CSIT source code documentation for DMM functional tests. -#. **Detailed Results**: Detailed result tables auto-generated from CSIT +#. DETAILED RESULTS: Detailed result tables auto-generated from CSIT test job executions using RF (Robot Framework) output files as sources. @@ -127,7 +115,7 @@ files are also provided. - **NSH_SFC Functional**: Detailed nsh-plugin functional results. - **DMM Functional**: Detailed DMM functional results. -#. **Test Configuration**: VPP DUT configuration data based on VPP API +#. TEST CONFIGURATION: VPP DUT configuration data based on VPP API Test (VAT) Commands History auto-generated from CSIT test job executions using RF output files as sources. @@ -136,13 +124,13 @@ files are also provided. - **VPP K8s Container Memif**: Configuration data. - **VPP Functional**: Configuration data. -#. **Test Operational Data**: VPP DUT operational data auto-generated +#. TEST OPERATIONAL DATA: VPP DUT operational data auto-generated from CSIT test job executions using RFoutput files as sources. - **VPP Performance NDR/PDR**: VPP `show run` outputs under test load. -#. **CSIT Framework Documentation**: Description of the overall FD.io +#. CSIT FRAMEWORK DOCUMENTATION: Description of the overall FD.io CSIT framework. - **Design**: Framework modular design hierarchy. diff --git a/docs/report/introduction/test_scenarios_overview.rst b/docs/report/introduction/test_scenarios_overview.rst index 8b7266d0a4..aebb0b1900 100644 --- a/docs/report/introduction/test_scenarios_overview.rst +++ b/docs/report/introduction/test_scenarios_overview.rst @@ -1,11 +1,11 @@ Test Scenarios Overview ======================= -FD.io CSIT report includes multiple test scenarios of a number of VPP +FD.io |csit-release| report includes multiple test scenarios of VPP centric applications, topologies and use cases. In addition it also -covers baseline tests of DPDK sample applications. Tests are executed -executed in physical (performance tests) and virtual environments -(functional tests). +covers baseline tests of DPDK sample applications. Tests are executed in +physical (performance tests) and virtual environments (functional +tests). Following list provides a brief overview of test scenarios covered in this report: diff --git a/docs/report/vpp_functional_tests/csit_release_notes.rst b/docs/report/vpp_functional_tests/csit_release_notes.rst index 8eda865348..f70ead3091 100644 --- a/docs/report/vpp_functional_tests/csit_release_notes.rst +++ b/docs/report/vpp_functional_tests/csit_release_notes.rst @@ -4,11 +4,11 @@ Release Notes Changes in |csit-release| ------------------------- -#. **CSIT Framework** +#. CSIT FRAMEWORK - Few test case bug fixes. -#. **Test Case Porting to VPP_make_test** +#. TEST CASE PORTING TO VPP_MAKE_TEST - Implementation of VIRL functional integration tests in VPP_make_test. diff --git a/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst b/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst index c9e0bbbb99..b49a447405 100644 --- a/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst +++ b/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst @@ -6,15 +6,6 @@ MRR) is calculated for the same tests executed on 3-Node Skylake (3n- skx) and 3-Node Haswell (3n-hsw) physical testbed types, in 1-core, 2-core and 4-core configurations. -NDR Comparison -~~~~~~~~~~~~~~ - -NDR comparison between testbed types is available in CSV and pretty -ASCII formats: - - - `CSV NDR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_, - - `ASCII NDR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_. - .. note:: Test results have been generated by @@ -24,38 +15,26 @@ ASCII formats: files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. -PDR Comparison +NDR Comparison ~~~~~~~~~~~~~~ -PDR comparison between testbed types is available in CSV and pretty -ASCII formats: +Comparison tables in ASCII and CSV formats: - - `CSV PDR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_, - - `ASCII PDR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_. + - `ASCII NDR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_ + - `CSV NDR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_ -.. note:: +PDR Comparison +~~~~~~~~~~~~~~ - Test results have been generated by - `FD.io test executor vpp performance job 3n-hsw`_ and - `FD.io test executor vpp performance job 3n-skx`_ - with RF result - files csit-vpp-perf-|srelease|-\*.zip - `archived here <../_static/archive/>`_. +Comparison tables in ASCII and CSV formats: + + - `ASCII PDR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_ + - `CSV PDR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_ MRR Comparison ~~~~~~~~~~~~~~ -MRR comparison between testbed types is available in CSV and pretty -ASCII formats: - - - `CSV MRR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_, - - `ASCII MRR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_. - -.. note:: +Comparison tables in ASCII and CSV formats: - Test results have been generated by - `FD.io test executor vpp performance job 3n-hsw`_ and - `FD.io test executor vpp performance job 3n-skx`_ - with RF result - files csit-vpp-perf-|srelease|-\*.zip - `archived here <../_static/archive/>`_. + - `ASCII MRR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_ + - `CSV MRR comparison <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_ diff --git a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst index c2a87dc2ab..228d8f889b 100644 --- a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst +++ b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst @@ -16,17 +16,6 @@ repeatability and makes the relative change of mean throughput value not fully representative for these tests. The root causes behind poor results repeatability vary between the test cases. -NDR Comparison -~~~~~~~~~~~~~~ - -NDR throughput comparison between releases is available in CSV and pretty -ASCII formats: - - - `CSV 1t1c NDR changes <../_static/vpp/performance-changes-1t1c-ndr.csv>`_, - - `CSV 2t2c NDR changes <../_static/vpp/performance-changes-2t2c-ndr.csv>`_, - - `ASCII 1t1c NDR changes <../_static/vpp/performance-changes-1t1c-ndr.txt>`_, - - `ASCII 2t2c NDR changes <../_static/vpp/performance-changes-2t2c-ndr.txt>`_. - .. note:: Test results have been generated by @@ -35,42 +24,34 @@ ASCII formats: files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. -PDR Comparison +NDR Comparison ~~~~~~~~~~~~~~ -PDR throughput comparison between releases is available in CSV and pretty -ASCII formats: +Comparison tables in ASCII and CSV formats: - - `CSV 1t1c PDR changes <../_static/vpp/performance-changes-1t1c-pdr.csv>`_, - - `CSV 2t2c PDR changes <../_static/vpp/performance-changes-2t2c-pdr.csv>`_, - - `ASCII 1t1c PDR changes <../_static/vpp/performance-changes-1t1c-pdr.txt>`_, - - `ASCII 2t2c PDR changes <../_static/vpp/performance-changes-2t2c-pdr.txt>`_. + - `ASCII 1t1c NDR comparison <../_static/vpp/performance-changes-1t1c-ndr.txt>`_ + - `ASCII 2t2c NDR comparison <../_static/vpp/performance-changes-2t2c-ndr.txt>`_ + - `CSV 1t1c NDR comparison <../_static/vpp/performance-changes-1t1c-ndr.csv>`_ + - `CSV 2t2c NDR comparison <../_static/vpp/performance-changes-2t2c-ndr.csv>`_ -.. note:: +PDR Comparison +~~~~~~~~~~~~~~ - Test results have been generated by - `FD.io test executor vpp performance job 3n-hsw`_, - with RF result - files csit-vpp-perf-|srelease|-\*.zip - `archived here <../_static/archive/>`_. +Comparison tables in ASCII and CSV formats: + + - `ASCII 1t1c PDR comparison <../_static/vpp/performance-changes-1t1c-pdr.txt>`_ + - `ASCII 2t2c PDR comparison <../_static/vpp/performance-changes-2t2c-pdr.txt>`_ + - `CSV 1t1c PDR comparison <../_static/vpp/performance-changes-1t1c-pdr.csv>`_ + - `CSV 2t2c PDR comparison <../_static/vpp/performance-changes-2t2c-pdr.csv>`_ MRR Comparison ~~~~~~~~~~~~~~ -MRR throughput comparison between releases is available in CSV and pretty -ASCII formats: - - - `CSV 1t1c MRR changes <../_static/vpp/performance-changes-1t1c-mrr.csv>`_, - - `CSV 2t2c MRR changes <../_static/vpp/performance-changes-2t2c-mrr.csv>`_, - - `CSV 4t4c MRR changes <../_static/vpp/performance-changes-4t4c-mrr.csv>`_, - - `ASCII 1t1c MRR changes <../_static/vpp/performance-changes-1t1c-mrr.txt>`_, - - `ASCII 2t2c MRR changes <../_static/vpp/performance-changes-2t2c-mrr.txt>`_, - - `ASCII 4t4c MRR changes <../_static/vpp/performance-changes-4t4c-mrr.txt>`_. - -.. note:: +Comparison tables in ASCII and CSV formats: - Test results have been generated by - `FD.io test executor vpp performance job 3n-hsw`_, - with RF result - files csit-vpp-perf-|srelease|-\*.zip - `archived here <../_static/archive/>`_. + - `ASCII 1t1c MRR comparison <../_static/vpp/performance-changes-1t1c-mrr.txt>`_ + - `ASCII 2t2c MRR comparison <../_static/vpp/performance-changes-2t2c-mrr.txt>`_ + - `ASCII 4t4c MRR comparison <../_static/vpp/performance-changes-4t4c-mrr.txt>`_ + - `CSV 1t1c MRR comparison <../_static/vpp/performance-changes-1t1c-mrr.csv>`_ + - `CSV 2t2c MRR comparison <../_static/vpp/performance-changes-2t2c-mrr.csv>`_ + - `CSV 4t4c MRR comparison <../_static/vpp/performance-changes-4t4c-mrr.csv>`_ diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 0318e7a01e..233bf4545a 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -4,7 +4,7 @@ Release Notes Changes in |csit-release| ------------------------- -#. **VPP Performance Tests** +#. VPP PERFORMANCE TESTS - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now configurable trial duration and number of consecutive executions. @@ -44,12 +44,12 @@ Changes in |csit-release| Ansible playbooks refactor for quick server (re)installation and reference pointers of configuration. -#. **Presentation and Analytics Layer** +#. PRESENTATION AND ANALYTICS LAYER - - **Performance trending**: Further improved continuous performance - trending with anomaly detection and analysis. + - **Performance trending**: Further improvements of continuous + performance trending, anomaly detection and analysis. -#. **Test Framework Optimizations** +#. TEST FRAMEWORK OPTIMIZATIONS - **General Code Housekeeping**: Ongoing RF keywords optimizations, removal of redundant RF keywords. diff --git a/docs/report/vpp_performance_tests/throughput_trending.rst b/docs/report/vpp_performance_tests/throughput_trending.rst index c2ca421d57..6607423e0f 100644 --- a/docs/report/vpp_performance_tests/throughput_trending.rst +++ b/docs/report/vpp_performance_tests/throughput_trending.rst @@ -4,7 +4,7 @@ Throughput Trending In addition to reporting throughput comparison between VPP releases, CSIT provides continuous performance trending for VPP master branch: -#. `VPP Performance Dashboard `_ +#. `Performance Dashboard `_ - per VPP test case throughput trend, trend compliance and summary of detected anomalies. @@ -12,6 +12,6 @@ CSIT provides continuous performance trending for VPP master branch: - throughput test metrics, trend calculations and anomaly classification (progression, regression). -#. `Trendline Graphs `_ +#. `VPP Trendline Graphs `_ - per VPP build MRR throughput measurements against the trendline with anomaly highlights and associated CSIT test jobs. \ No newline at end of file -- 2.16.6