CSIT rls1704 report - small updates 72/6472/2
authorJan Gelety <jgelety@cisco.com>
Wed, 26 Apr 2017 15:17:02 +0000 (17:17 +0200)
committerMaciek Konstantynowicz <mkonstan@cisco.com>
Wed, 26 Apr 2017 15:48:01 +0000 (16:48 +0100)
Change-Id: I80b5aaf2bab77a6131fca8cf4e903ef6aecd1195
Signed-off-by: Jan Gelety <jgelety@cisco.com>
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
docs/report/testpmd_performance_tests/csit_release_notes.rst
docs/report/vpp_functional_tests/csit_release_notes.rst
docs/report/vpp_performance_tests/csit_release_notes.rst
docs/report/vpp_unit_tests/overview.rst
resources/tools/report_gen/run_report.sh

index 7f87d24..72bef87 100644 (file)
@@ -20,6 +20,6 @@ Here is the list of known issues in CSIT |release| for Testpmd performance tests
 |   | of discovered NDR throughput values across        |            | throughput. Applies to XL710 and X710 NICs, no issues observed  |
 |   | multiple test runs with xl710 and x710 NICs.      |            | on x520 NICs.                                                   |
 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
-| 2 | Lower than expected NDR throughput with xl710     | CSIT-?     | Suspected NIC firmware or DPDK driver issue affecting NDR       |
+| 2 | Lower than expected NDR throughput with xl710     | CSIT-571   | Suspected NIC firmware or DPDK driver issue affecting NDR       |
 |   | and x710 NICs, compared to x520 NICs.             |            | throughput. Applies to XL710 and X710 NICs.                     |
 +---+---------------------------------------------------+------------+-----------------------------------------------------------------+
\ No newline at end of file
index 8f63b8e..25ca27c 100644 (file)
@@ -6,11 +6,10 @@ Changes in CSIT |release|
 
 #. VPP functional test environment changes
 
-#. Implemented VAT command history collection for every test case as part of teardown
-
+    - Implemented VAT command history collection for every test case as part of teardown.
     - Introduction of Centos7 tests in VIRL environment.
 
-#. VPP performance test framework changes
+#. VPP functional test framework changes
 
     - Added VAT command history collection for every test case as part of teardown.
 
@@ -21,7 +20,6 @@ Changes in CSIT |release|
     - IPv4 routed-forwarding with vhost-user interfaces to VM.
     - Vhost-user interface re-connect tests.
 
->>>>>>> csit rls1704 report - updated csit_release_notes.rst and overview.rst files.
 Known Issues
 ------------
 
@@ -55,17 +53,21 @@ Here is the list of known issues in CSIT |release| for VPP functional tests in V
 |   |                                                 |          | destination address is reported.                     |
 +---+-------------------------------------------------+----------+------------------------------------------------------+
 | 6 | IPv4 + VLAN: Processing of tagged frame doesn't | CSIT-564 | Dot1q tagged packets are thrown away in case of IPv4 |
-|   | work with virtio driver                         |          | routing on interface binded to virtio driver.        |
+|   | work with virtio driver                         |          | routing on interface binded to virtio driver. Issue  |
+|   |                                                 |          | with VIRL test simulation environment                |
 +---+-------------------------------------------------+----------+------------------------------------------------------+
-| 7 | VHOST-user: QEMU re-connect doesn't work        | CSIT-565 | Used QEMU 2.5.0 doesn't support re-connection. Usage |
-|   |                                                 |          | of QEMU 2.7.0 is not recommended by vpp-dev team at  |
-|   |                                                 |          | the moment.                                          |
+| 7 | Vhost-user: QEMU reconnect doesn't work         | CSIT-565 | Using QEMU 2.5.0 that does not support vhost-user    |
+|   |                                                 |          | reconnect. It requires moving CSIT VIRL environment  |
+|   |                                                 |          | to QEMU 2.7.0.                                       |
 +---+-------------------------------------------------+----------+------------------------------------------------------+
-| 8 | Centos7: LISP and VXLAN test failures           | CSIT-566 | We can observe crash of SUT1 VM during LISP tests    |
-|   |                                                 |          | execution, that leads to failure of all other tests  |
-|   |                                                 |          | (sometimes observed  during VXLAN tests execution    |
-|   |                                                 |          | too). NOTE: Possible root cause identified after     |
-|   |                                                 |          | upgrade of one of VIRL servers. When fix is          |
-|   |                                                 |          | confirmed in a short time it will be merged to CSIT  |
-|   |                                                 |          | rls1704 branch.                                      |
+| 8 | Centos7: LISP and VXLAN sporadic test failures  | CSIT-566 | Observing sporadic crashes of DUT1 VM during LISP    |
+|   |                                                 |          | and VXLAN test executions, what leads to failure of  |
+|   |                                                 |          | all other tests in the suite. NOTE: After upgrading  |
+|   |                                                 |          | one of the VIRL servers host from Ubuntu14.04 to     |
+|   |                                                 |          | 16.04, DUT1 VM stops crashing, but tests keep        |
+|   |                                                 |          | failing sporadically 1-in-100. Possible root cause   |
+|   |                                                 |          | identified - unexpected IPv6 RA packets upsetting    |
+|   |                                                 |          | some test component. Currently suspecting            |
+|   |                                                 |          | environment or CSIT issue, but can not exclude VPP,  |
+|   |                                                 |          | further troubleshooting in progress.                 |
 +---+-------------------------------------------------+----------+------------------------------------------------------+
index 7c17e0f..b4fc9de 100644 (file)
@@ -215,14 +215,14 @@ Here is the list of known issues in CSIT |release| for VPP performance tests:
 |   | routes - 2M in this case. ip6scale2kk works.    |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 | 3 | Vic1385 and Vic1227 low performance             | VPP-664    | Low NDR performance.                                            |\r
-|   |                                                 |            |                                       .                         |\r
+|   |                                                 |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 | 4 | Sporadic NDR discovery test failures on x520    | CSIT-750   | Suspected issue with HW settings (BIOS, FW) in LF               |\r
 |   |                                                 |            | infrastructure. Issue can't be replicated outside LF.           |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 | 5 | VPP in 2t2c setups - large variation            | CSIT-568   | Suspected NIC firmware or DPDK driver issue affecting NDR       |\r
 |   | of discovered NDR throughput values across      |            | throughput. Applies to XL710 and X710 NICs, x520 NICs are fine. |\r
-|   | multiple test runs with xl710 and x710 NICs.    |            |                                       .                         |\r
+|   | multiple test runs with xl710 and x710 NICs.    |            |                                                                 |\r
 +---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
 | 6 | Lower than expected NDR and PDR throughput with | CSIT-569   | Suspected NIC firmware or DPDK driver issue affecting NDR and   |\r
 |   | xl710 and x710 NICs, compared to x520 NICs.     |            | PDR throughput. Applies to XL710 and X710 NICs.                 |\r
index 2848db7..4ef50f5 100644 (file)
@@ -33,7 +33,7 @@ Unit Tests Coverage
 -------------------\r
 \r
 Following VPP functional test areas are covered in VPP unit test code included\r
-in VPP rls1701 with results listed in this report:\r
+in VPP rls1704 with results listed in this report:\r
 \r
 - CRUD Loopback - create, read, update, delete Loopback interfaces.\r
 - Flow-per-packet plugin - collect and report L2 and IP4 flow statistics.\r
index 00fbdfd..4aa15b7 100755 (executable)
@@ -238,7 +238,7 @@ sed -i -e "s@###LINK###@${JEN_URL}\/${JEN_JOB}\/${JEN_BUILD[-1]}@g" \
 ### FUNCTIONAL SOURCE DATA
 
 JEN_JOB='csit-vpp-functional-1704-ubuntu1604-virl'
-JEN_BUILD=57
+JEN_BUILD=63
 
 if [[ ${DEBUG} -eq 1 ]] ;
 then

©2016 FD.io a Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation. Linux is a registered trademark of Linus Torvalds.
Please see our privacy policy and terms of use.