From ceb7c2545cda1bc7fe3d4bf35cdf75253efe8c6d Mon Sep 17 00:00:00 2001 From: Peter Mikus Date: Thu, 3 May 2018 10:14:11 +0200 Subject: [PATCH] Update release note for rls1804 report Change-Id: I3e80997ace8073edc90f662f6393bd62005fc815 Signed-off-by: Peter Mikus --- docs/report/introduction/overview.rst | 13 ++++------ .../vpp_performance_tests/csit_release_notes.rst | 29 +++++++++------------- 2 files changed, 17 insertions(+), 25 deletions(-) diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst index 536d5d3cf1..2d5a1e0671 100644 --- a/docs/report/introduction/overview.rst +++ b/docs/report/introduction/overview.rst @@ -33,8 +33,11 @@ CSIT |release| report contains following main sections and sub-sections: tests, performance changes, environment or methodology changes, known CSIT issues; *Packet Throughput Graphs* and *Packet Latency Graphs* - plotted NDR, PDR throughput and latency results from multiple - test job executions; *Test Environment* - environment description; - *Documentation* - CSIT source code documentation for VPP performance tests. + test job executions; *Throughput Speedup Multi-Core* - plotted core + configuration speedup comparision; *Test Environment* - environment + description; *VPP HTTP Server Performance Results* - plotted HTTP Server + performance; *Documentation* - CSIT source code documentation for VPP + performance tests. #. **DPDK Performance Tests** - DPDK performance tests executed in physical FD.io testbeds; *Overview* - tested topologies, test coverage; @@ -58,12 +61,6 @@ CSIT |release| report contains following main sections and sub-sections: *Test Environment* - environment description ; *Documentation* - source code documentation for Honeycomb functional tests. -#. **VPP Unit Tests** - refers to VPP functional unit tests executed as - part of vpp make test verify option within the FD.io VPP project; listed in - this report to give a more complete view about executed VPP functional tests; - *Overview* - short overview of unit test framework and executed tests; - *Documentation* - source code documentation of VPP unit tests. - #. **NSH_SFC Functional Tests** - NSH_SFC functional tests executed in virtual FD.io testbeds; *Overview* - tested virtual topologies, test coverage and naming specifics; *CSIT Release Notes* - changes in CSIT diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index d7c2bbc7d4..3ce11bdae5 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -89,32 +89,27 @@ pretty ASCII formats: Known Issues ------------ - - Here is the list of known issues in CSIT |release| for VPP performance tests: +---+-------------------------------------------------+------------+-----------------------------------------------------------------+ | # | Issue | Jira ID | Description | +---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| 1 | Vic1385 and Vic1227 low performance. | VPP-664 | Low NDR performance. | -| | | | | -+---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| 2 | Sporadic (1 in 200) NDR discovery test failures | CSIT-570 | DPDK reporting rx-errors, indicating L1 issue. Suspected issue | +| 1 | Sporadic (1 in 200) NDR discovery test failures | CSIT-570 | DPDK reporting rx-errors, indicating L1 issue. Suspected issue | | | on x520. | | with HW combination of X710-X520 in LF testbeds. Not observed | | | | | outside of LF testbeds. | +---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| 3 | Lower than expected NDR throughput with | CSIT-571 | Suspected NIC firmware or DPDK driver issue affecting NDR and | -| | xl710 and x710 NICs, compared to x520 NICs. | | PDR throughput. Applies to XL710 and X710 NICs. | +| 2 | Lower than expected NDR throughput of DPDK | CSIT-571 | Suspected NIC firmware or DPDK driver issue affecting NDR and | +| | testpmd and VPP L2 path NDR throughput with | | PDR throughput on XL710 and X710 NICs. | +| | xl710 and x710 NICs, compared to x520 NICs. | | | +---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| 4 | rls1801 plugin related performance regression | CSIT-925 | With all plugins loaded NDR, PDR and MaxRates vary | -| | | | intermittently from 3% to 5% across multiple test executions. | -| | | | Requires plugin code bisecting. | +| 3 | Tagged Ethernet dot1q and dot1ad L2 path | CSIT-1066 | Tagged Ethernet dot1q and dot1ad L2 path throughput regression: | +| | throughput regression. | | NDR -2%..-5%, PDR -2%..-6%, MRR. Affects l2xc and l2bd | +| | | | performance tests. | +---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| 5 | rls1801 generic small performance regression | CSIT-926 | Generic performance regression of discovered NDR, PDR and | -| | ip4base, l2xcbase, l2bdbase | | MaxRates of -3%..-1% vs. rls1710, affects ip4base, l2xcbase, | -| | | | l2bdbase test suites. Not detected by CSIT performance trending | -| | | | scheme as it was masked out by another issue CSIT-925. | +| 4 | IPSec (software, no QAT HW) throughput | CSIT-1064 | IPSec throughput regression: NDR -3%..-8%, PDR -2%..-8%, MRR | +| | regression. | | -3%..-7%. | +---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| 6 | rls1801 substantial NDR/PDR regression for | CSIT-928 | NDR regression of -7%..-15%, PDR regression of -3%..-15% | -| | IPSec tunnel scale with HW QAT crypto-dev | | compared to rls1710. | +| 5 | High failure rate of creating working container | CSIT-1065 | Orchestrated container topology tests failing data plane | +| | topologies with K8s/Ligato orchestration. | | verification indicating configuration issue. Suspected issue | +| | | | with Ligato vpp-agent. | +---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -- 2.16.6