From 4c1ef1193f02bfdb518341e632ee09ddd6ecc2d3 Mon Sep 17 00:00:00 2001 From: selias Date: Thu, 26 Oct 2017 12:14:48 +0200 Subject: [PATCH] Update static report content - Honeycomb Change-Id: I9f6612a4495df62f3a43c19457864a4eeff64a3d Signed-off-by: selias --- .../csit_release_notes.rst | 24 ++----- .../report/honeycomb_functional_tests/overview.rst | 14 +++- .../test_environment.rst | 6 +- .../csit_release_notes.rst | 2 +- .../test_result_data.rst | 82 +--------------------- 5 files changed, 25 insertions(+), 103 deletions(-) diff --git a/docs/report/honeycomb_functional_tests/csit_release_notes.rst b/docs/report/honeycomb_functional_tests/csit_release_notes.rst index 38ec001418..7509ca5dd2 100644 --- a/docs/report/honeycomb_functional_tests/csit_release_notes.rst +++ b/docs/report/honeycomb_functional_tests/csit_release_notes.rst @@ -4,15 +4,15 @@ CSIT Release Notes Changes in CSIT |release| ------------------------- -#. Added Honeycomb functional tests for the following features: +#. Added Honeycomb functional tests for the following areas: - - Policer + - LISP GPE + - Honeycomb northbound interfaces over IPv6 + - Honeycomb implementation of ODL BGP #. Improved test coverage for the following features: - - Interface Management - - Vlan - - Port Mirroring + - Vlan sub-interfaces Known Issues ------------ @@ -27,17 +27,7 @@ tests in VIRL: | | | | VPP refuses the configuration but returns code 200:OK. This can cause | | | | | desync between Honeycomb's config and operational data. | +---+--------------------------------------------+------------+----------------------------------------------------------------------------+ -| 2 | Removal of ACP-plugin interface assignment | HC2VPP-173 | Attempting to remove all ACLs from an interface responds with OK but does | -| | | | not remove the assignments. | +| 2 | VxLAN GPE configuration crashes VPP | VPP-875 | Specific VxLAN GPE configurations cause VPP to crash and restart. | +---+--------------------------------------------+------------+----------------------------------------------------------------------------+ -| 3 | VxLAN GPE configuration crashes VPP | VPP-875 | Specific VxLAN GPE configurations cause VPP to crash and restart. | -+---+--------------------------------------------+------------+----------------------------------------------------------------------------+ -| 4 | Policer traffic test failure | CSIT- | Traffic test has begun to fail, likely due to VPP changes. There is more | -| | | | information available yet. | -+---+--------------------------------------------+------------+----------------------------------------------------------------------------+ -| 5 | SPAN traffic test failure | CSIT- | Traffic test has begun to fail, likely due to VPP changes. There is more | -| | | | information available yet. | -+---+--------------------------------------------+------------+----------------------------------------------------------------------------+ -| 6 | Unnumbered interface VIRL issue | CSIT- | CRUD for unnumbered interface appears to fail in VIRL, but not in local | -| | | | test runs. Investigation pending. | +| 3 | Operational data for IPv6 special routes | HC2VPP-228 | Special hop routes are misidentified as regular routes in operational data.| +---+--------------------------------------------+------------+----------------------------------------------------------------------------+ diff --git a/docs/report/honeycomb_functional_tests/overview.rst b/docs/report/honeycomb_functional_tests/overview.rst index a0491b17ed..493499aea5 100644 --- a/docs/report/honeycomb_functional_tests/overview.rst +++ b/docs/report/honeycomb_functional_tests/overview.rst @@ -105,7 +105,12 @@ with results listed in this report: - Configure and delete Lisp mapping as local and remote. - Configure and delete Lisp adjacency mapping - Configure and delete Lisp map resolver, proxy ITR. - - Test case count: 16 + - Test case count: 18 +- **LISP GPE** - CRUD for LISP GPE mappings. + - Toggle Lisp GPE feature status. + - Configure Lisp GPE mappings. + - Traffic test verifying encapsulation. + - Test case count: 12 - **NAT** - CRD for NAT entries, interface assignment. - Configure and delete up to two NAT entries. - Assign NAT entries to a physical interface. @@ -146,11 +151,16 @@ with results listed in this report: - Configure Policing rules. - Assign to interface. - Test case count: 6 +- **Border Gateway Protocol** - CRUD and functional tests for BGP. + - Configure peers and routes + - Check interactions with another BGP peer. + - Test case count: 9 - **Honeycomb Infractructure** - configuration persistence, - Netconf notifications for interface events, - Netconf negative tests aimed at specific issues + - Netconf/Restconf northbound over IPv6 -Total 173 Honeycomb functional tests in the CSIT |release|. +Total 217 Honeycomb functional tests in the CSIT |release|. Operational data in Honeycomb should mirror configuration data at all times. Because of this, test cases follow this general pattern: diff --git a/docs/report/honeycomb_functional_tests/test_environment.rst b/docs/report/honeycomb_functional_tests/test_environment.rst index b55bfad602..7982a085a1 100644 --- a/docs/report/honeycomb_functional_tests/test_environment.rst +++ b/docs/report/honeycomb_functional_tests/test_environment.rst @@ -7,10 +7,10 @@ each host being a Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz, 18c, 512GB RAM) running Ubuntu 14.04.3 and the following VIRL software versions: - STD server version 0.10.24.7 - UWM server version 0.10.24.7 + - STD server version 0.10.32.16 + - UWM server version 0.10.32.16 -Current VPP tests have been executed on a single VM operating system and +Current Honeycomb tests have been executed on a single VM operating system and version only, as described in the following paragraphs. In CSIT terminology, the VM operating system for both DUTs and TG that diff --git a/docs/report/honeycomb_performance_tests/csit_release_notes.rst b/docs/report/honeycomb_performance_tests/csit_release_notes.rst index c9a0444597..d79907d4ca 100644 --- a/docs/report/honeycomb_performance_tests/csit_release_notes.rst +++ b/docs/report/honeycomb_performance_tests/csit_release_notes.rst @@ -4,7 +4,7 @@ CSIT Release Notes Changes in CSIT |release| ------------------------- -#. First release with Honeycomb performance testing. +#. No changes since previous release. Known Issues ------------ diff --git a/docs/report/honeycomb_performance_tests/test_result_data.rst b/docs/report/honeycomb_performance_tests/test_result_data.rst index 2e365369d8..9e791d6e88 100644 --- a/docs/report/honeycomb_performance_tests/test_result_data.rst +++ b/docs/report/honeycomb_performance_tests/test_result_data.rst @@ -15,83 +15,5 @@ read operation targets. Honeycomb + Netconf =================== -Performs read operations from Honeycomb's operational data store. Honeycomb -does not contact VPP to obtain the most up-to-date value. Operations are -performed synchronously per client, varying the number of clients from 1 to 16. - -netconf-netty-threads: 16 - -+----------------+----------------------------------+----------------+ -| # clients | TCP performance (reads/sec) | Total requests | -+================+==================================+================+ -| 1 | 5179 | 100K | -+----------------+----------------------------------+----------------+ -| 2 | 10601 | 100K | -+----------------+----------------------------------+----------------+ -| 4 | 21488 | 100K | -+----------------+----------------------------------+----------------+ -| 8 | 35772 | 100K | -+----------------+----------------------------------+----------------+ -| 16 | 73544 | 100K | -+----------------+----------------------------------+----------------+ - -netconf-netty-threads: 1 - -+----------------+----------------------------------+----------------+ -| # clients | TCP performance (reads/sec) | Total requests | -+================+==================================+================+ -| 1 | 5179 | 100K | -+----------------+----------------------------------+----------------+ -| 2 | 5557 | 100K | -+----------------+----------------------------------+----------------+ -| 4 | 5888 | 100K | -+----------------+----------------------------------+----------------+ -| 8 | 5977 | 100K | -+----------------+----------------------------------+----------------+ -| 16 | 6119 | 100K | -+----------------+----------------------------------+----------------+ - -Data source: -https://jenkins.fd.io/view/hc2vpp/job/hc2vpp-csit-perf-master-ubuntu1604/8/ - -Honeycomb + Netconf + VPP -========================= - -Performs read operations from Honeycomb's operational data store. Honeycomb -uses VPP API to obtain the value from VPP before responding. Operations are -performed synchronously per client, varying the number of clients from 1 to 16. - -netconf-netty-threads: 16 - -+----------------+----------------------------------+----------------+ -| # clients | TCP performance (reads/sec) | Total requests | -+================+==================================+================+ -| 1 | 4766 | 100K | -+----------------+----------------------------------+----------------+ -| 2 | 9906 | 100K | -+----------------+----------------------------------+----------------+ -| 4 | 17594 | 100K | -+----------------+----------------------------------+----------------+ -| 8 | 29933 | 100K | -+----------------+----------------------------------+----------------+ -| 16 | 45254 | 100K | -+----------------+----------------------------------+----------------+ - -netconf-netty-threads: 1 - -+----------------+----------------------------------+----------------+ -| # clients | TCP performance (reads/sec) | Total requests | -+================+==================================+================+ -| 1 | 4600 | 100K | -+----------------+----------------------------------+----------------+ -| 2 | 5375 | 100K | -+----------------+----------------------------------+----------------+ -| 4 | 5404 | 100K | -+----------------+----------------------------------+----------------+ -| 8 | 5488 | 100K | -+----------------+----------------------------------+----------------+ -| 16 | 6000 | 100K | -+----------------+----------------------------------+----------------+ - -Data source: -https://jenkins.fd.io/view/hc2vpp/job/hc2vpp-csit-perf-master-ubuntu1604/9/ \ No newline at end of file +No new data has been generated in the course of this release. Results from +the last successful run are available in `17.07 release report `_. \ No newline at end of file -- 2.16.6