1704 Report - Honeycomb section

Change-Id: I8006cf8187f943b8d880c8193b6b1bde25ad343c
Signed-off-by: selias <samelias@cisco.com>

diff --git a/docs/report/honeycomb_functional_tests/csit_release_notes.rst b/docs/report/honeycomb_functional_tests/csit_release_notes.rst
index 68b5eb6..5252542 100644 (file)
--- a/docs/report/honeycomb_functional_tests/csit_release_notes.rst
+++ b/docs/report/honeycomb_functional_tests/csit_release_notes.rst
@@ -4,32 +4,38 @@ CSIT Release Notes
 Changes in CSIT |release|\r
 -------------------------\r
 \r
 Changes in CSIT |release|\r
 -------------------------\r
 \r

-#. Naming change for all Honeycomb functional test suites in VIRL\r
-\r
-    - Honeycomb functional test case names stayed unchanged\r
-\r

 #. Added Honeycomb functional tests\r
 \r
 #. Added Honeycomb functional tests\r
 \r

-    - NSH_SFC\r
-    - LISP\r
-    - NAT\r
-    - SPAN\r
+    - ACL plugin\r
+    - Routing\r
+    - SLAAC\r
+    - Proxy ARP\r
+    - DHCP Relay\r
+    - Neighbor Discovery Proxy\r
+\r
+#. Changed execution environment from Ubuntu14.04 to Ubuntu16.04\r

 \r
 Known Issues\r
 ------------\r
 \r
 Here is the list of known issues in CSIT |release| for Honeycomb functional\r
 tests in VIRL:\r
 \r
 Known Issues\r
 ------------\r
 \r
 Here is the list of known issues in CSIT |release| for Honeycomb functional\r
 tests in VIRL:\r

++---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
+| # | Issue                                      | Jira ID    | Description                                                                |\r
++---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
+| 1 | IP address subnet validation               | VPP-649    | When configuring two IP addresses from the same subnet on an interface,    |\r
+|   |                                            |            | VPP refuses the configuration but returns OK. This can cause desync        |\r
+|   |                                            |            | between Honeycomb's config and operational data.                           |\r
++---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
+| 2 | Persistence of VxLAN tunnel naming context | HC2VPP-47  | When VPP restarts with Honeycomb running and a VxLan interface configured, |\r
+|   |                                            |            | the interface is sometimes renamed to "vxlan_tunnel0".                     |\r
+|   |                                            |            | It is otherwise configured correctly.                                      |\r
++---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
+| 3 | Classifier plugin for IPv6 cases           | VPP-687    | Classifier ignores IPv6 packets with less than 8 bytes after last header.  |\r
+|   |                                            |            | Fixed in VPP 17.07.                                                        |\r
++---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
+| 4 | Batch disable Lisp features                | HC2VPP-131 | When removing complex Lisp configurations in a single request,             |\r
+|   |                                            |            | the operation fails due to a write ordering issue.                         |\r
++---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r

 \r
 \r

-+---+-------------------------------------------------+-----------------------------------------------------------------+\r
-| # | Issue                                           | Description                                                     |\r
-+---+-------------------------------------------------+-----------------------------------------------------------------+\r
-| 1 | Operational data for Vhost-user interfaces      | Honeycomb Operational data reports Vhost-user interfaces        |\r
-|   | "server" flag                                   | as client, even if they are server.                             |\r
-+---+-------------------------------------------------+-----------------------------------------------------------------+\r
-| 2 | Persistence of VxLAN tunnels                    | Configuration persistence often fails to restore                |\r
-|   |                                                 | Honeycomb's internal naming context for VxLAN interfaces.       |\r
-|   |                                                 | The interface is renamed to "vxlan_tunnel0" but is otherwise    |\r
-|   |                                                 | configured correctly.                                           |\r
-+---+-------------------------------------------------+-----------------------------------------------------------------+\r

 \r
 \r

diff --git a/docs/report/honeycomb_functional_tests/overview.rst b/docs/report/honeycomb_functional_tests/overview.rst
index d93de13..c643f3e 100644 (file)
--- a/docs/report/honeycomb_functional_tests/overview.rst
+++ b/docs/report/honeycomb_functional_tests/overview.rst
@@ -117,6 +117,33 @@ with results listed in this report:
   - up to 2 interfaces.\r
   - Remove SPAN configuration from interfaces.\r
   - Test case count: 3\r
   - up to 2 interfaces.\r
   - Remove SPAN configuration from interfaces.\r
   - Test case count: 3\r

+- **ACL-PLUGIN** - CRD for high-level classifier\r
+  - MAC + IP address classification.\r
+  - IPv4, IPv6 address classification.\r
+  - TCP, UDP, ICMP, ICMPv6 protocol/next-header classification.\r
+  - port number classification.\r
+  - ICMP, ICMPv6 code and type classification.\r
+  - Test case count: 15\r
+- **ProxyARP** - CRD for proxyARP feature.\r
+  - Configure proxyARP.\r
+  - Assign to interface.\r
+  - Test case count: 3\r
+- **ProxyND6** - CRD for Neighbor Discovery Proxy.\r
+  - Configure ProxyND6 feature on interface.\r
+  - Test case count: 4\r
+- **DHCP Relay** - CRD for DHCP relay feature.\r
+  - Configure DHCP Relays.\r
+  - IPv4 and IPv6 variants.\r
+  - Test case count: 4\r
+- **SLAAC** - CRD for Stateless Address AutoConfiguration.\r
+  - Configure SLAAC feature on interfaces.\r
+  - Test case count: 7\r
+- **Routing** - CRD for routing.\r
+  - Configure single-hop route.\r
+  - Configure multi-hop routes.\r
+  - Configure blackhole route.\r
+  - IPv4 and IPv6 variants.\r
+    - Test case count: 6\r

 - **Honeycomb Infractructure** - configuration persistence,\r
   - Netconf notifications for interface events,\r
   - Netconf negative tests aimed at specific issues\r
 - **Honeycomb Infractructure** - configuration persistence,\r
   - Netconf notifications for interface events,\r
   - Netconf negative tests aimed at specific issues\r


@@ -130,7 +157,8 @@ Because of this, test cases follow this general pattern:
 #. read status of the feature using VPP API dump.\r
 #. modify configuration of the feature using restconf.\r
 #. verify changes to operational data using restconf.\r
 #. read status of the feature using VPP API dump.\r
 #. modify configuration of the feature using restconf.\r
 #. verify changes to operational data using restconf.\r

-#. verify changes using VPP API dump.\r
+#. verify changes using VPP API dump, OR\r
+#. send a packet to VPP node and observe behaviour to verify configuration\r

 \r
 Test cases involving network interfaces utilize the first two interfaces on\r
 the DUT node.\r
 \r
 Test cases involving network interfaces utilize the first two interfaces on\r
 the DUT node.\r

diff --git a/docs/report/honeycomb_functional_tests/test_environment.rst b/docs/report/honeycomb_functional_tests/test_environment.rst
index e706aa5..028911f 100644 (file)
--- a/docs/report/honeycomb_functional_tests/test_environment.rst
+++ b/docs/report/honeycomb_functional_tests/test_environment.rst
@@ -13,21 +13,21 @@ versions:
 Current VPP tests have been executed on a single VM operating system and\r
 version only, as described in the following paragraphs.\r
 \r
 Current VPP tests have been executed on a single VM operating system and\r
 version only, as described in the following paragraphs.\r
 \r

-In CSIT terminology, the VM operating system for both SUTs and TG that VPP 17.01\r
+In CSIT terminology, the VM operating system for both SUTs and TG that VPP 17.04\r

 has been tested with, is the following:\r
 \r
 has been tested with, is the following:\r
 \r

-  ubuntu-14.04.4_2016-10-07_1.3\r
+  ubuntu-16.04.1_2017-02-23_1.8\r

 \r
 \r

-which implies Ubuntu 14.04.3 LTS, current as of 2016/10/07 (that is, package\r
-versions are those that would have been installed by a "apt-get update",\r
-"apt-get upgrade" on October 7), produced by CSIT disk image build scripts\r
-version 1.6.\r
+This image implies Ubuntu 16.04.1 LTS, current as of 2017/02/23 (that is,\r
+package versions are those that would have been installed by a "apt-get update",\r
+"apt-get upgrade" on February 23), produced by CSIT disk image build scripts\r
+version 1.8.\r

 \r
 The exact list of installed packages and their versions (including the Linux\r
 kernel package version) are included in CSIT source repository:\r
 \r
 \r
 The exact list of installed packages and their versions (including the Linux\r
 kernel package version) are included in CSIT source repository:\r
 \r

-  resources/tools/disk-image-builder/ubuntu/lists/ubuntu-14.04.4_2016-10-07_1.3\r
+  resources/tools/disk-image-builder/ubuntu/lists/ubuntu-16.04.1_2017-02-23_1.8\r

 \r
 A replica of this VM image can be built by running the "build.sh" script in CSIT\r
 \r
 A replica of this VM image can be built by running the "build.sh" script in CSIT\r

-repository resources/tools/disk-image-builder/.\r
+repository resources/tools/disk-image-builder/ubuntu.\r

 \r
 \r