Apply minor improvements to methodology docs
[csit.git] / docs / report / vpp_functional_tests / csit_release_notes.rst
index fd71d78..8b0ae9e 100644 (file)
@@ -1,73 +1,60 @@
-CSIT Release Notes
-==================
+Release Notes
+=============
 
-Changes in CSIT |release|
+Changes in |csit-release|
 -------------------------
 
-#. VPP functional test framework changes:
+#. TEST FRAMEWORK
 
-   - Upgrade of tb4-virl1 and tb4-virl2 hosts;
+   - **VPP_Device**: Added container based functional VPP device tests
+     integrated into LFN CI/CD infrastructure. VPP_Device tests run on
+     1-Node testbeds (1n-skx, 1n-arm) and rely on Linux SRIOV Virtual
+     Function (VF), dot1q VLAN tagging and external loopback cables to
+     facilitate packet passing over exernal physical links. Initial
+     focus is on few baseline tests. Existing CSIT VIRL tests can be
+     moved to VPP_Device framework by changing L1 KW(s). L2 KWs and RF
+     test definition code stay unchanged. CSIT VIRL to VPP_Device
+     migration is expected in the next CSIT release.
 
-     - OS version to Ubuntu 16.04.02;
+   - **Bug fixes**.
 
-     - VIRL version to Mitaka;
+#. CSIT TEST MIMGRATION
 
-   - Implemented load balance mechanism for VIRL hosts;
-
-   - Implemented waiting mechanism on VIRL host if there is not enough resources
-     available to successfully start simulation;
-
-#. Code updates and optimizations in CSIT functional framework:
-
-   - Complete CSIT framework code revision and optimizations as descried
-     on CSIT wiki page
-     _`Design_Optimizations <https://wiki.fd.io/view/CSIT/Design_Optimizations>`_;
-
-   - For more detail see the CSIT Framework Design section in this
-     report;
+   - **VPP_Path**: Continuing migration of the original FD.io CSIT VIRL
+     tests to VPP-make_test VPP integration tests for functional
+     acceptance of VPP feature path(s) driven by use case(s). See P1
+     and P2 markup in `CSIT_VIRL migration progress
+     <https://docs.google.com/spreadsheets/d/1PciV8XN9v1qHbIRUpFJoqyES29_vik7lcFDl73G1usc/edit?usp=sharing>`_
 
 Known Issues
 ------------
 
-Here is the list of known issues in CSIT |release| for VPP functional tests in VIRL:
+List of known issues in |csit-release| for VPP functional tests in VIRL:
+
++---+----------------------------------------+-------------------------------------------------------------------------------------------------------------------------+
+| # | JiraID                                 | Issue Description                                                                                                       |
++===+========================================+=========================================================================================================================+
+| 1 | `CSIT-129                              | DHCPv4 client: Client responses to DHCPv4 OFFER sent with different XID.                                                |
+|   | <https://jira.fd.io/browse/CSIT-129>`_ | Client replies with DHCPv4 REQUEST message when received DHCPv4 OFFER message with different (wrong) XID.               |
+|   | `VPP-99                                |                                                                                                                         |
+|   | <https://jira.fd.io/browse/VPP-99>`_   |                                                                                                                         |
++---+----------------------------------------+-------------------------------------------------------------------------------------------------------------------------+
+| 2 | `CSIT-398                              | Softwire - MAP-E: Incorrect calculation of IPv6 destination address when IPv4 prefix is 0.                              |
+|   | <https://jira.fd.io/browse/CSIT-398>`_ | IPv6 destination address is wrongly calculated in  case that IPv4 prefix is equal to 0 and IPv6 prefix is less than 40. |
+|   | `VPP-380                               |                                                                                                                         |
+|   | <https://jira.fd.io/browse/VPP-380>`_  |                                                                                                                         |
++---+----------------------------------------+-------------------------------------------------------------------------------------------------------------------------+
+| 3 | `CSIT-399                              | Softwire - MAP-E: Map domain is created when incorrect parameters provided.                                             |
+|   | <https://jira.fd.io/browse/CSIT-399>`_ | Map domain is created in case that the sum of suffix length of IPv4 prefix and PSID length is greater than EA bits      |
+|   | `VPP-435                               | length. IPv6 destination address contains bits writen with PSID over the EA-bit length when IPv4 packet is sent.        |
+|   | <https://jira.fd.io/browse/VPP-435>`_  |                                                                                                                         |
++---+----------------------------------------+-------------------------------------------------------------------------------------------------------------------------+
+| 4 | `CSIT-409                              | IPv6 RA: Incorrect IPv6 destination address in response to ICMPv6 Router Solicitation.                                  |
+|   | <https://jira.fd.io/browse/CSIT-409>`_ | Wrong IPv6 destination address (ff02::1) is used in ICMPv6 Router Advertisement packet sent as a response to received   |
+|   | `VPP-406                               | ICMPv6 Router Solicitation packet.                                                                                      |
+|   | <https://jira.fd.io/browse/VPP-406>`_  |                                                                                                                         |
++---+----------------------------------------+-------------------------------------------------------------------------------------------------------------------------+
+| 5 | `CSIT-565                              | Vhost-user: QEMU reconnect does not work.                                                                               |
+|   | <https://jira.fd.io/browse/CSIT-565>`_ | QEMU 2.5.0 used in CSIT does not support vhost-user reconnect. Requires upgrading CSIT VIRL environment to QEMU 2.7.0.  |
++---+----------------------------------------+-------------------------------------------------------------------------------------------------------------------------+
 
-+---+-------------------------------------------------+----------+------------------------------------------------------+
-| # | Issue                                           | Jira ID  | Description                                          |
-+---+-------------------------------------------------+----------+------------------------------------------------------+
-| 1 | DHCPv4 client: Client responses to DHCPv4 OFFER | CSIT-129 | Client replies with DHCPv4 REQUEST message when      |
-|   | sent with different XID.                        |          | received DHCPv4 OFFER message with different (wrong) |
-|   |                                                 |          | XID.                                                 |
-+---+-------------------------------------------------+----------+------------------------------------------------------+
-| 2 | Softwire - MAP-E: Incorrect calculation of IPv6 | CSIT-398 | IPv6 destination address is wrongly calculated in    |
-|   | destination address when IPv4 prefix is 0.      |          | case that IPv4 prefix is equal to 0 and IPv6 prefix  |
-|   |                                                 |          | is less than 40.                                     |
-+---+-------------------------------------------------+----------+------------------------------------------------------+
-| 3 | Softwire - MAP-E: Map domain is created when    | CSIT-399 | Map domain is created in case that the sum of suffix |
-|   | incorrect parameters provided.                  |          | length of IPv4 prefix and PSID length is greater     |
-|   |                                                 |          | than EA bits length. IPv6 destination address        |
-|   |                                                 |          | contains bits writen with PSID over the EA-bit       |
-|   |                                                 |          | length when IPv4 packet is sent.                     |
-+---+-------------------------------------------------+----------+------------------------------------------------------+
-| 4 | IPv6 RA: Incorrect IPv6 destination address in  | CSIT-409 | Wrong IPv6 destination address (ff02::1) is used in  |
-|   | response to ICMPv6 Router Solicitation.         |          | ICMPv6 Router Advertisement packet sent as a         |
-|   |                                                 |          | response to received  ICMPv6 Router Solicitation     |
-|   |                                                 |          | packet.                                              |
-+---+-------------------------------------------------+----------+------------------------------------------------------+
-| 5 | Vhost-user: QEMU reconnect does not work.       | CSIT-565 | Using QEMU 2.5.0 that does not support vhost-user    |
-|   |                                                 |          | reconnect. It requires moving CSIT VIRL environment  |
-|   |                                                 |          | to QEMU 2.7.0.                                       |
-+---+-------------------------------------------------+----------+------------------------------------------------------+
-| 6 | Centos7: LISP and VXLAN sporadic test failures. | CSIT-566 | Observing sporadic crashes of DUT1 VM during LISP    |
-|   |                                                 |          | and VXLAN test executions, what leads to failure of  |
-|   |                                                 |          | all other tests in the suite. NOTE: After upgrading  |
-|   |                                                 |          | one of the VIRL servers host from Ubuntu14.04 to     |
-|   |                                                 |          | 16.04, DUT1 VM stops crashing, but tests keep        |
-|   |                                                 |          | failing sporadically 1-in-100. Possible root cause   |
-|   |                                                 |          | identified - unexpected IPv6 RA packets upsetting    |
-|   |                                                 |          | some test component. Currently suspecting            |
-|   |                                                 |          | environment or CSIT issue, but can not exclude VPP,  |
-|   |                                                 |          | further troubleshooting in progress.                 |
-+---+-------------------------------------------------+----------+------------------------------------------------------+
-| 7 | IPSEC: Tests cover old crypto code path         |    --    | There are used default conf settings so IPSEC tests  |
-|   |                                                 |          | use old security code not the new Crypto SW code.    |
-+---+-------------------------------------------------+----------+------------------------------------------------------+