Update static report content - Honeycomb 48/9048/3
authorselias <samelias@cisco.com>
Thu, 26 Oct 2017 10:14:48 +0000 (12:14 +0200)
committerTibor Frank <tifrank@cisco.com>
Thu, 26 Oct 2017 11:00:41 +0000 (11:00 +0000)
Change-Id: I9f6612a4495df62f3a43c19457864a4eeff64a3d
Signed-off-by: selias <samelias@cisco.com>
docs/report/honeycomb_functional_tests/csit_release_notes.rst
docs/report/honeycomb_functional_tests/overview.rst
docs/report/honeycomb_functional_tests/test_environment.rst
docs/report/honeycomb_performance_tests/csit_release_notes.rst
docs/report/honeycomb_performance_tests/test_result_data.rst

index 38ec001..7509ca5 100644 (file)
@@ -4,15 +4,15 @@ CSIT Release Notes
 Changes in CSIT |release|\r
 -------------------------\r
 \r
-#. Added Honeycomb functional tests for the following features:\r
+#. Added Honeycomb functional tests for the following areas:\r
 \r
-   - Policer\r
+   - LISP GPE\r
+   - Honeycomb northbound interfaces over IPv6\r
+   - Honeycomb implementation of ODL BGP\r
 \r
 #. Improved test coverage for the following features:\r
 \r
-   - Interface Management\r
-   - Vlan\r
-   - Port Mirroring\r
+   - Vlan sub-interfaces\r
 \r
 Known Issues\r
 ------------\r
@@ -27,17 +27,7 @@ tests in VIRL:
 |   |                                            |            | VPP refuses the configuration but returns code 200:OK. This can cause      |\r
 |   |                                            |            | desync between Honeycomb's config and operational data.                    |\r
 +---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
-| 2 | Removal of ACP-plugin interface assignment | HC2VPP-173 | Attempting to remove all ACLs from an interface responds with OK but does  |\r
-|   |                                            |            | not remove the assignments.                                                |\r
+| 2 | VxLAN GPE configuration crashes VPP        | VPP-875    | Specific VxLAN GPE configurations cause VPP to crash and restart.          |\r
 +---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
-| 3 | VxLAN GPE configuration crashes VPP        | VPP-875    | Specific VxLAN GPE configurations cause VPP to crash and restart.          |\r
-+---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
-| 4 | Policer traffic test failure               | CSIT-      | Traffic test has begun to fail, likely due to VPP changes. There is  more  |\r
-|   |                                            |            | information available yet.                                                 |\r
-+---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
-| 5 | SPAN traffic test failure                  | CSIT-      | Traffic test has begun to fail, likely due to VPP changes. There is  more  |\r
-|   |                                            |            | information available yet.                                                 |\r
-+---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
-| 6 | Unnumbered interface VIRL issue            | CSIT-      | CRUD for unnumbered interface appears to fail in VIRL, but not in local    |\r
-|   |                                            |            | test runs. Investigation pending.                                          |\r
+| 3 | Operational data for IPv6 special routes   | HC2VPP-228 | Special hop routes are misidentified as regular routes in operational data.|\r
 +---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
index a0491b1..493499a 100644 (file)
@@ -105,7 +105,12 @@ with results listed in this report:
   - Configure and delete Lisp mapping as local and remote.
   - Configure and delete Lisp adjacency mapping
   - Configure and delete Lisp map resolver, proxy ITR.
-  - Test case count: 16
+  - Test case count: 18
+- **LISP GPE** - CRUD for LISP GPE mappings.
+  - Toggle Lisp GPE feature status.
+  - Configure Lisp GPE mappings.
+  - Traffic test verifying encapsulation.
+  - Test case count: 12
 - **NAT** - CRD for NAT entries, interface assignment.
   - Configure and delete up to two NAT entries.
   - Assign NAT entries to a physical interface.
@@ -146,11 +151,16 @@ with results listed in this report:
   - Configure Policing rules.
   - Assign to interface.
   - Test case count: 6
+- **Border Gateway Protocol** - CRUD and functional tests for BGP.
+  - Configure peers and routes
+  - Check interactions with another BGP peer.
+  - Test case count: 9
 - **Honeycomb Infractructure** - configuration persistence,
   - Netconf notifications for interface events,
   - Netconf negative tests aimed at specific issues
+  - Netconf/Restconf northbound over IPv6
 
-Total 173 Honeycomb functional tests in the CSIT |release|.
+Total 217 Honeycomb functional tests in the CSIT |release|.
 
 Operational data in Honeycomb should mirror configuration data at all times.
 Because of this, test cases follow this general pattern:
index b55bfad..7982a08 100644 (file)
@@ -7,10 +7,10 @@ each host being a Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @
 2.30GHz, 18c, 512GB RAM) running Ubuntu 14.04.3 and the following VIRL software\r
 versions:\r
 \r
-  STD server version 0.10.24.7\r
-  UWM server version 0.10.24.7\r
+  - STD server version 0.10.32.16\r
+  - UWM server version 0.10.32.16\r
 \r
-Current VPP tests have been executed on a single VM operating system and\r
+Current Honeycomb tests have been executed on a single VM operating system and\r
 version only, as described in the following paragraphs.\r
 \r
 In CSIT terminology, the VM operating system for both DUTs and TG that\r
index c9a0444..d79907d 100644 (file)
@@ -4,7 +4,7 @@ CSIT Release Notes
 Changes in CSIT |release|\r
 -------------------------\r
 \r
-#. First release with Honeycomb performance testing.\r
+#. No changes since previous release.\r
 \r
 Known Issues\r
 ------------\r
index 2e36536..9e791d6 100644 (file)
@@ -15,83 +15,5 @@ read operation targets.
 Honeycomb + Netconf
 ===================
 
-Performs read operations from Honeycomb's operational data store. Honeycomb
-does not contact VPP to obtain the most up-to-date value. Operations are
-performed synchronously per client, varying the number of clients from 1 to 16.
-
-netconf-netty-threads: 16
-
-+----------------+----------------------------------+----------------+
-| # clients      | TCP performance (reads/sec)      | Total requests |
-+================+==================================+================+
-| 1              | 5179                             | 100K           |
-+----------------+----------------------------------+----------------+
-| 2              | 10601                            | 100K           |
-+----------------+----------------------------------+----------------+
-| 4              | 21488                            | 100K           |
-+----------------+----------------------------------+----------------+
-| 8              | 35772                            | 100K           |
-+----------------+----------------------------------+----------------+
-| 16             | 73544                            | 100K           |
-+----------------+----------------------------------+----------------+
-
-netconf-netty-threads: 1
-
-+----------------+----------------------------------+----------------+
-| # clients      | TCP performance (reads/sec)      | Total requests |
-+================+==================================+================+
-| 1              | 5179                             | 100K           |
-+----------------+----------------------------------+----------------+
-| 2              | 5557                             | 100K           |
-+----------------+----------------------------------+----------------+
-| 4              | 5888                             | 100K           |
-+----------------+----------------------------------+----------------+
-| 8              | 5977                             | 100K           |
-+----------------+----------------------------------+----------------+
-| 16             | 6119                             | 100K           |
-+----------------+----------------------------------+----------------+
-
-Data source:
-https://jenkins.fd.io/view/hc2vpp/job/hc2vpp-csit-perf-master-ubuntu1604/8/
-
-Honeycomb + Netconf + VPP
-=========================
-
-Performs read operations from Honeycomb's operational data store. Honeycomb
-uses VPP API to obtain the value from VPP before responding. Operations are
-performed synchronously per client, varying the number of clients from 1 to 16.
-
-netconf-netty-threads: 16
-
-+----------------+----------------------------------+----------------+
-| # clients      | TCP performance (reads/sec)      | Total requests |
-+================+==================================+================+
-| 1              | 4766                             | 100K           |
-+----------------+----------------------------------+----------------+
-| 2              | 9906                             | 100K           |
-+----------------+----------------------------------+----------------+
-| 4              | 17594                            | 100K           |
-+----------------+----------------------------------+----------------+
-| 8              | 29933                            | 100K           |
-+----------------+----------------------------------+----------------+
-| 16             | 45254                            | 100K           |
-+----------------+----------------------------------+----------------+
-
-netconf-netty-threads: 1
-
-+----------------+----------------------------------+----------------+
-| # clients      | TCP performance (reads/sec)      | Total requests |
-+================+==================================+================+
-| 1              | 4600                             | 100K           |
-+----------------+----------------------------------+----------------+
-| 2              | 5375                             | 100K           |
-+----------------+----------------------------------+----------------+
-| 4              | 5404                             | 100K           |
-+----------------+----------------------------------+----------------+
-| 8              | 5488                             | 100K           |
-+----------------+----------------------------------+----------------+
-| 16             | 6000                             | 100K           |
-+----------------+----------------------------------+----------------+
-
-Data source:
-https://jenkins.fd.io/view/hc2vpp/job/hc2vpp-csit-perf-master-ubuntu1604/9/
\ No newline at end of file
+No new data has been generated in the course of this release. Results from
+the last successful run are available in `17.07 release report <https://docs.fd.io/csit/rls1707/report/honeycomb_performance_tests/test_result_data.html>`_.
\ No newline at end of file

©2016 FD.io a Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation. Linux is a registered trademark of Linus Torvalds.
Please see our privacy policy and terms of use.