X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Freport%2Fintroduction%2Fintroduction.rst;h=5470cca4e9df8f955bc6264ab275be820c08f26b;hp=24c5970725e4e527fc9e93f4f05f2a1b18978044;hb=b659ab3d2638652de0357d29be1ec7704df97555;hpb=5a1f4570778a7511415d94e58cc2299d02b871cd diff --git a/docs/report/introduction/introduction.rst b/docs/report/introduction/introduction.rst index 24c5970725..5470cca4e9 100644 --- a/docs/report/introduction/introduction.rst +++ b/docs/report/introduction/introduction.rst @@ -1,5 +1,5 @@ -Introduction -============ +Report Structure +================ FD.io |csit-release| report contains system performance and functional testing data of |vpp-release|. `PDF version of this report`_ is @@ -13,7 +13,8 @@ available for download. - **Test Scenarios Overview**: A brief overview of test scenarios covered in this report. - **Physical Testbeds**: Description of physical testbeds. - - **Performance Test Methodology**: Benchmarking test methodologies. + - **Test Methodology**: Performance benchmarking and functional test + methodologies. #. VPP PERFORMANCE: VPP performance tests executed in physical FD.io testbeds. @@ -25,19 +26,21 @@ available for download. - **Packet Throughput**: NDR, PDR throughput graphs based on results from repeated same test job executions to verify repeatibility of measurements. - - **Packet Latency**: Latency graphs based on results from test job - executions. - **Speedup Multi-Core**: NDR, PDR throughput multi-core speedup graphs based on results from test job executions. - - **HTTP/TCP Performance**: HTTP/TCP VPP test server performance - graphs. + - **Packet Latency**: Latency graphs based on results from test job + executions. + - **Soak Tests**: Long duration soak tests are executed using PLRsearch + algorithm. + - **NFV Service Density**: Network Function Virtualization (NFV) service + density tests focus on measuring total per server throughput at varied NFV + service "packing" densities with vswitch providing host dataplane. - **Comparisons**: Performance comparisons between VPP releases and between different testbed types. - **Throughput Trending**: References to continuous VPP performance trending. - **Test Environment**: Performance test environment configuration. - - **Documentation**: Documentation of K8s Pod/Container orchestration - in CSIT and pointers to CSIT source code documentation for VPP + - **Documentation**: Pointers to CSIT source code documentation for VPP performance tests. #. DPDK PERFORMANCE: DPDK performance tests executed in physical @@ -58,38 +61,29 @@ available for download. - **Documentation**: Pointers to CSIT source code documentation for DPDK performance tests. -#. VPP FUNCTIONAL: VPP functional tests executed in virtual FD.io - testbeds. +#. VPP DEVICE: VPP functional tests executed in physical FD.io + testbeds using containers. - **Overview**: Tested virtual topologies, test coverage and naming specifics; - **Release Notes**: Changes in |csit-release|, added tests, environment or methodology changes, known issues. - - **Test Environment**: Functional test environment configuration. + - **Integration Tests**: Functional test environment configuration. - **Documentation**: Pointers to CSIT source code documentation for VPP functional tests. -#. HONEYCOMBE FUNCTIONAL: Honeycomb functional tests executed in - virtual FD.io testbeds. - - - **Overview**: Tested virtual topologies, test coverage and naming - specifics; - - **Release Notes**: Changes in |csit-release|, known issues. - - **Test Environment**: Functional test environment configuration. - - **Documentation**: Pointers to CSIT source code documentation for - Honeycomb functional tests. - -#. NSH_SFC FUNCTIONAL: NSH_SFC functional tests executed in - virtual FD.io testbeds. +#. VPP FUNCTIONAL: VPP functional tests executed in virtual FD.io + testbeds. - **Overview**: Tested virtual topologies, test coverage and naming specifics; - - **Release Notes**: Changes in |csit-release|, known issues. + - **Release Notes**: Changes in |csit-release|, added tests, + environment or methodology changes, known issues. - **Test Environment**: Functional test environment configuration. - **Documentation**: Pointers to CSIT source code documentation for - NSH_SFC functional tests. + VPP functional tests. -#. DMM FUNCTIONAL: DMM functional tests executed in +#. HONEYCOMB FUNCTIONAL: Honeycomb functional tests executed in virtual FD.io testbeds. - **Overview**: Tested virtual topologies, test coverage and naming @@ -97,7 +91,7 @@ available for download. - **Release Notes**: Changes in |csit-release|, known issues. - **Test Environment**: Functional test environment configuration. - **Documentation**: Pointers to CSIT source code documentation for - DMM functional tests. + Honeycomb functional tests. #. DETAILED RESULTS: Detailed result tables auto-generated from CSIT test job executions using RF (Robot Framework) output files as @@ -105,14 +99,10 @@ available for download. - **VPP Performance NDR/PDR**: VPP NDR/PDR throughput and latency. - **VPP Performance MRR**: VPP MRR throughput. - - **VPP K8s Container Memif**: VPP K8s Container/Pod topologies - NDR/PDR throughput. - **DPDK Performance**: DPDK Testpmd and L3fwd NDR/PDR throughput and latency. - **VPP Functional**: Detailed VPP functional results. - **Honeycomb Functional**: Detailed HoneyComb functional results. - - **NSH_SFC Functional**: Detailed nsh-plugin functional results. - - **DMM Functional**: Detailed DMM functional results. #. TEST CONFIGURATION: VPP DUT configuration data based on VPP API Test (VAT) Commands History auto-generated from CSIT test job @@ -120,7 +110,6 @@ available for download. - **VPP Performance NDR/PDR**: Configuration data. - **VPP Performance MRR**: Configuration data. - - **VPP K8s Container Memif**: Configuration data. - **VPP Functional**: Configuration data. #. TEST OPERATIONAL DATA: VPP DUT operational data auto-generated @@ -138,3 +127,24 @@ available for download. analytics module. - **CSIT RF Tags Descriptions**: CSIT RF Tags used for test suite and test case grouping and selection. + +.. + #. DMM FUNCTIONAL: DMM functional tests executed in + virtual FD.io testbeds. + + - **Overview**: Tested virtual topologies, test coverage and naming + specifics; + - **Release Notes**: Changes in |csit-release|, known issues. + - **Test Environment**: Functional test environment configuration. + - **Documentation**: Pointers to CSIT source code documentation for + DMM functional tests. + + #. NSH_SFC FUNCTIONAL: NSH_SFC functional tests executed in + virtual FD.io testbeds. + + - **Overview**: Tested virtual topologies, test coverage and naming + specifics; + - **Release Notes**: Changes in |csit-release|, known issues. + - **Test Environment**: Functional test environment configuration. + - **Documentation**: Pointers to CSIT source code documentation for + NSH_SFC functional tests.