\r
- **isolcpus=<cpu number>-<cpu number>** used for all cpu cores apart from\r
first core of each socket used for running VPP worker threads and Qemu/LXC\r
- processes https://www.kernel.org/doc/Documentation/kernel-parameters.txt\r
+ processes\r
+ https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt\r
- **intel_pstate=disable** - [X86] Do not enable intel_pstate as the default\r
scaling driver for the supported processors. Intel P-State driver decide what\r
P-state (CPU core power state) to use based on requesting policy from the\r
- **rcu_nocbs** - [KNL] In kernels built with CONFIG_RCU_NOCB_CPU=y, set the\r
specified list of CPUs to be no-callback CPUs, that never queue RCU callbacks\r
(read-copy update).\r
- https://www.kernel.org/doc/Documentation/kernel-parameters.txt\r
+ https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt\r
\r
**Applied command line boot parameters:**\r
\r
Current VPP tests have been executed on a single VM operating system and\r
version only, as described in the following paragraphs.\r
\r
-In CSIT terminology, the VM operating system for both SUTs and TG that\r
+In CSIT terminology, the VM operating system for both DUTs and TG that\r
|vpp-release| has been tested with, is the following:\r
\r
|virl-image-ubuntu|\r
"apt-get upgrade" on that day), produced by CSIT disk image build scripts.\r
\r
The exact list of installed packages and their versions (including the Linux\r
-kernel package version) are included in CSIT source repository:\r
+kernel package version) are included in `VIRL images lists`_.\r
\r
- resources/tools/disk-image-builder/ubuntu/lists/|virl-image-ubuntu|\r
-\r
-A replica of this VM image can be built by running the "build.sh" script in CSIT\r
-repository resources/tools/disk-image-builder/ubuntu.\r
+A replica of this VM image can be built by running the "build.sh" script in\r
+`VIRL nested`_.\r
\r
Changes in CSIT |release|\r
-------------------------\r
\r
-#. First release with honeycomb performance testing\r
+#. First release with Honeycomb performance testing.\r
\r
Known Issues\r
------------\r
================
To execute performance tests, there are three identical testbeds, each testbed
-consists of two SUTs and one TG.
+consists of two DUTs and one TG.
Server HW Configuration
-----------------------
+.. _csit-design:\r
+\r
CSIT Design\r
===========\r
\r
Following NSH_SFC functional test areas are covered in the CSIT |release| with\r
results listed in this report:\r
\r
-- **NSH SFC Classifier** - TG send some TCP packets to test NSH SFC\r
+- **NSH SFC Classifier** - TG sends some TCP packets to test NSH SFC\r
Classifier functional. DUT1 will receive these packets from one NIC and loopback\r
the VXLAN-GPE-NSH encapsulated packets to the TG from other NIC.\r
- Test case count: 7\r
-- **NSH SFC Proxy Inbound** - TG send some VXLAN-GPE-NSH encapsulated packets\r
+- **NSH SFC Proxy Inbound** - TG sends some VXLAN-GPE-NSH encapsulated packets\r
to test NSH SFC Proxy Inbound functional. DUT1 will receive these packets from one\r
NIC and loopback the VXLAN encapsulated packets to the TG from other NIC.\r
- Test case count: 6\r
-- **NSH SFC Proxy Outbound** - TG send some VXLAN encapsulated packets to test\r
+- **NSH SFC Proxy Outbound** - TG sends some VXLAN encapsulated packets to test\r
NSH SFC Proxy Outbound functional. DUT1 will receive these packets from one NIC\r
and loopback the VXLAN-GPE-NSH encapsulated packets to the TG from other NIC.\r
- Test case count: 6\r
-- **NSH SFC Service Function Forward** - TG send some VXLAN-GPE-NSH encapsulated\r
+- **NSH SFC Service Function Forward** - TG sends some VXLAN-GPE-NSH encapsulated\r
packets to test NSH SFC Service Function Forward functional. DUT1 will receive these\r
packets from one NIC and swap the VXLAN-GPE-NSH header, after that DUT1 loopback the\r
VXLAN-GPE-NSH encapsulated packtes to the TG from other NIC.\r
on CSIT wiki page
`Design_Optimizations <https://wiki.fd.io/view/CSIT/Design_Optimizations>`_;
- - For more detail see the CSIT Framework Design section in this
- report;
+ - For more detail see the :ref:`CSIT Framework Design <csit-design>` section
+ in this report;
Known Issues
------------
| | | | environment or CSIT issue, but can not exclude VPP, |
| | | | further troubleshooting in progress. |
+---+-------------------------------------------------+----------+------------------------------------------------------+
-| 7 | IPSEC: Tests cover old crypto code path | -- | There are used default conf settings so IPSEC tests |
+| 7 | IPSEC: Tests cover old crypto code path | CSIT-733 | There are used default conf settings so IPSEC tests |
| | | | use old security code not the new Crypto SW code. |
+---+-------------------------------------------------+----------+------------------------------------------------------+
SUT Configuration - VIRL Guest VM\r
---------------------------------\r
\r
-Configuration of the SUT VMs is defined in file\r
-\r
- /csit/resources/tools/virl/topologies/double-ring-nested.xenial.virl\r
+Configurations of the SUT VMs is defined in `VIRL topologies directory`_\r
\r
- List of SUT VM interfaces:::\r
\r
In CSIT terminology, the VM operating system for both SUTs that |vpp-release| has\r
been tested with, is the following:\r
\r
-**#. Ubuntu VIRL image**\r
-\r
-This image implies Ubuntu 16.04.1 LTS, current as of yyyy-mm-dd (that is,\r
-package versions are those that would have been installed by a "apt-get update",\r
-"apt-get upgrade" on that day), produced by CSIT disk image build scripts.\r
-\r
-The exact list of installed packages and their versions (including the Linux\r
-kernel package version) are included in CSIT source repository:\r
-\r
- resources/tools/disk-image-builder/ubuntu/lists/|virl-image-ubuntu|\r
-\r
-A replica of this VM image can be built by running the "build.sh" script in CSIT\r
-repository resources/tools/disk-image-builder/ubuntu.\r
-\r
-**#. CentOS VIRL image**\r
-\r
-The Centos7.3 image is ready to be used but no tests running on it now.\r
-Corresponding Jenkins jobs are under preparation.\r
-\r
-The exact list of installed packages and their versions (including the Linux\r
-kernel package version) are included in CSIT source repository:\r
-\r
- resources/tools/disk-image-builder/ubuntu/lists/|virl-image-centos|\r
-\r
-A replica of this VM image can be built by running the "build.sh" script in CSIT\r
-repository resources/tools/disk-image-builder/centos.\r
-\r
-**#. Nested VM image**\r
-\r
-In addition to the "main" VM image, tests which require VPP to communicate to a\r
-VM over a vhost-user interface, utilize a "nested" VM image.\r
-\r
-This "nested" VM is dynamically created and destroyed as part of a test case,\r
-and therefore the "nested" VM image is optimized to be small, lightweight and\r
-have a short boot time. The "nested" VM image is not built around any\r
-established Linux distribution, but is based on `BuildRoot\r
-<https://buildroot.org/>`_, a tool for building embedded Linux systems. Just as\r
-for the "main" image, scripts to produce an identical replica of the "nested"\r
-image are included in CSIT GIT repository, and the image can be rebuilt using\r
-the "build.sh" script at:\r
-\r
- resources/tools/disk-image-builder/ubuntu/lists/nested\r
+#. **Ubuntu VIRL image**\r
+\r
+ This image implies Ubuntu 16.04.1 LTS, current as of yyyy-mm-dd (that is,\r
+ package versions are those that would have been installed by a "apt-get\r
+ update", "apt-get upgrade" on that day), produced by CSIT disk image build\r
+ scripts.\r
+ \r
+ The exact list of installed packages and their versions (including the Linux\r
+ kernel package version) are included in `VIRL images lists`_.\r
+ \r
+ A replica of this VM image can be built by running the "build.sh" script in\r
+ CSIT repository.\r
+\r
+#. **CentOS VIRL image**\r
+\r
+ The Centos7.3 image is ready to be used but no tests running on it now.\r
+ Corresponding Jenkins jobs are under preparation.\r
+ \r
+ The exact list of installed packages and their versions (including the Linux\r
+ kernel package version) are included in `VIRL images lists`_.\r
+ \r
+ A replica of this VM image can be built by running the "build.sh" script in\r
+ CSIT repository.\r
+\r
+#. **Nested VM image**\r
+\r
+ In addition to the "main" VM image, tests which require VPP to communicate to\r
+ a VM over a vhost-user interface, utilize a "nested" VM image.\r
+\r
+ This "nested" VM is dynamically created and destroyed as part of a test case,\r
+ and therefore the "nested" VM image is optimized to be small, lightweight and\r
+ have a short boot time. The "nested" VM image is not built around any\r
+ established Linux distribution, but is based on `BuildRoot\r
+ <https://buildroot.org/>`_, a tool for building embedded Linux systems. Just\r
+ as for the "main" image, scripts to produce an identical replica of the\r
+ "nested" image are included in CSIT GIT repository, and the image can be\r
+ rebuilt using the "build.sh" script at `VIRL nested`_.\r
\r
DUT Configuration - VPP\r
-----------------------\r
**DUT port configuration**\r
\r
Port configuration of DUTs is defined in topology file that is generated per\r
-VIRL simulation based on the definition stored in file\r
-\r
- /csit/resources/tools/virl/topologies/double-ring-nested.xenial.yaml\r
+VIRL simulation based on the definition stored in `VIRL topologies directory`_.\r
\r
Example of DUT nodes configuration:::\r
\r
\r
**TG VM configuration**\r
\r
-Configuration of the TG VMs is defined in file\r
+Configuration of the TG VMs is defined in `VIRL topologies directory`_.\r
\r
/csit/resources/tools/virl/topologies/double-ring-nested.xenial.virl\r
\r
**TG node port configuration**\r
\r
Port configuration of TG is defined in topology file that is generated per VIRL\r
-simulation based on the definition stored in file\r
-\r
- /csit/resources/tools/virl/topologies/double-ring-nested.xenial.yaml\r
+simulation based on the definition stored in `VIRL topologies directory`_.\r
\r
Example of TG node configuration:::\r
\r
on CSIT wiki page\r
`Design_Optimizations <https://wiki.fd.io/view/CSIT/Design_Optimizations>`_.\r
\r
- - For more detail see the CSIT Framework Design section in this\r
- report;\r
+ - For more detail see the :ref:`CSIT Framework Design <csit-design>` section\r
+ in this report;\r
\r
#. Changes to CSIT driver for TRex Traffic Generator:\r
\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
| # | Issue | Jira ID | Description |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 1 | Security-groups acl-plugin scale tests failure | CSIT-xxx | VPP with 2 worker threads crashes during security-groups |\r
+| 1 | Security-groups acl-plugin scale tests failure | CSIT-731 | VPP with 2 worker threads crashes during security-groups |\r
| | with stateful acls if VPP with 2 worker threads | VPP-912 | iaclNsf and oaclNsf tests with 100k flows. |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 2 | VPP fails memif tests in 4 worker 2 core setup | CSIT-xxx | VPP with 4 worker threads running on 2 physical cores crashes |\r
-| | | VPP-xxx | during memif tests. Initial debugging points to DPDK code |\r
+| 2 | VPP fails memif tests in 4 worker 2 core setup | CSIT-732 | VPP with 4 worker threads running on 2 physical cores crashes |\r
+| | | VPP-920 | during memif tests. Initial debugging points to DPDK code |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| X | NDR discovery test failures 1518B frame size | VPP-663 | VPP reporting errors: dpdk-input Rx ip checksum errors. |\r
+| 3 | NDR discovery test failures 1518B frame size | VPP-663 | VPP reporting errors: dpdk-input Rx ip checksum errors. |\r
| | for ip4scale200k, ip4scale2m scale IPv4 routed- | | Observed frequency: all test runs. |\r
| | forwarding tests. ip4scale20k tests are fine. | | |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| X | Vic1385 and Vic1227 low performance. | VPP-664 | Low NDR performance. |\r
+| 4 | Vic1385 and Vic1227 low performance. | VPP-664 | Low NDR performance. |\r
| | | | |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| X | Sporadic NDR discovery test failures on x520. | CSIT-750 | Suspected issue with HW settings (BIOS, FW) in LF |\r
+| 5 | Sporadic NDR discovery test failures on x520. | CSIT-750 | Suspected issue with HW settings (BIOS, FW) in LF |\r
| | | | infrastructure. Issue can't be replicated outside LF. |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| X | VPP in 2t2c setups - large variation | CSIT-568 | Suspected NIC firmware or DPDK driver issue affecting NDR |\r
+| 6 | VPP in 2t2c setups - large variation | CSIT-568 | Suspected NIC firmware or DPDK driver issue affecting NDR |\r
| | of discovered NDR throughput values across | | throughput. Applies to XL710 and X710 NICs, x520 NICs are fine. |\r
| | multiple test runs with xl710 and x710 NICs. | | |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| X | Lower than expected NDR and PDR throughput with | CSIT-569 | Suspected NIC firmware or DPDK driver issue affecting NDR and |\r
+| 7 | Lower than expected NDR and PDR throughput with | CSIT-569 | Suspected NIC firmware or DPDK driver issue affecting NDR and |\r
| | xl710 and x710 NICs, compared to x520 NICs. | | PDR throughput. Applies to XL710 and X710 NICs. |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
\r
$ sh -c 'cd /opt/trex-core-2.25/scripts/ && sudo nohup ./t-rex-64 -i -c 7 --iom 0 > /dev/null 2>&1 &' > /dev/null
-- There are traffic streams dynamically prepared for each test. The traffic
- is sent and the statistics obtained using trex_stl_lib.api.STLClient.
+- There are traffic streams dynamically prepared for each test, based on traffic
+ profiles. The traffic is sent and the statistics obtained using
+ trex_stl_lib.api.STLClient.
**Measuring packet loss**
\r
- **isolcpus=<cpu number>-<cpu number>** used for all cpu cores apart from\r
first core of each socket used for running VPP worker threads and Qemu/LXC\r
- processes https://www.kernel.org/doc/Documentation/kernel-parameters.txt\r
+ processes\r
+ https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt\r
- **intel_pstate=disable** - [X86] Do not enable intel_pstate as the default\r
scaling driver for the supported processors. Intel P-State driver decide what\r
P-state (CPU core power state) to use based on requesting policy from the\r
- **rcu_nocbs** - [KNL] In kernels built with CONFIG_RCU_NOCB_CPU=y, set the\r
specified list of CPUs to be no-callback CPUs, that never queue RCU callbacks\r
(read-copy update).\r
- https://www.kernel.org/doc/Documentation/kernel-parameters.txt\r
+ https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt\r
\r
**Applied command line boot parameters:**\r
\r
.. _tag documentation rst file: https://git.fd.io/csit/tree/docs/tag_documentation.rst?h=rls1707
.. _TRex intallation: https://git.fd.io/csit/tree/resources/tools/trex/trex_installer.sh?h=rls1707
.. _TRex driver: https://git.fd.io/csit/tree/resources/tools/trex/trex_stateless_profile.py?h=rls1707
-.. _CSIT Honeycomb Functional Tests Documentation: https://docs.fd.io/csit/rls1707/doc/tests.vpp.func.html
-.. _CSIT Honeycomb Performance Tests Documentation: https://docs.fd.io/csit/rls1707/doc/tests.vpp.perf.html
+.. _VIRL topologies directory: https://git.fd.io/csit/tree/resources/tools/virl/topologies/?h=rls1707
+.. _VIRL images lists: https://git.fd.io/csit/tree/resources/tools/disk-image-builder/ubuntu/lists/?h=rls1707
+.. _VIRL nested: https://git.fd.io/csit/tree/resources/tools/disk-image-builder/nested/?h=rls1707
+.. _CSIT Honeycomb Functional Tests Documentation: https://docs.fd.io/csit/rls1707/doc/tests.vpp.func.honeycomb.html
+.. _CSIT Honeycomb Performance Tests Documentation: https://docs.fd.io/csit/rls1707/doc/tests.vpp.perf.honeycomb.html
.. _CSIT DPDK Performance Tests Documentation: https://docs.fd.io/csit/rls1707/doc/tests.dpdk.perf.html
.. _CSIT VPP Functional Tests Documentation: https://docs.fd.io/csit/rls1707/doc/tests.vpp.func.html
.. _CSIT VPP Performance Tests Documentation: https://docs.fd.io/csit/rls1707/doc/tests.vpp.perf.html
.. _FD.io test executor honeycomb performance jobs: https://jenkins.fd.io/view/hc2vpp/job/hc2vpp-csit-perf-master-ubuntu1604/lastSuccessfulBuild
.. _FD.io test executor NSH_SFC functional jobs: https://jenkins.fd.io/view/csit/job/csit-nsh_sfc-verify-func-1707-ubuntu1604-virl/lastSuccessfulBuild
.. _FD.io VPP compile job: https://jenkins.fd.io/view/vpp/job/vpp-merge-1707-ubuntu1604/
-.. _FD.io VPP compile job: https://jenkins.fd.io/view/vpp/job/vpp-merge-1707-ubuntu1604/
.. _CSIT Testbed Setup: https://git.fd.io/csit/tree/resources/tools/testbed-setup/README.md?h=rls1707
"""
JOB[PERF,HC,BLD]="8 9"
JOB[FUNC,HC]=hc2vpp-csit-integration-1707-ubuntu1604
JOB[FUNC,HC,BLD]=lastSuccessfulBuild
-JOB[FUNC,NSH]=hc2vpp-csit-integration-1707-ubuntu1604
-JOB[FUNC,NSH,BLD]=lastSuccessfulBuild
+JOB[FUNC,NSH]=csit-nsh_sfc-verify-func-1707-ubuntu1604-virl
+JOB[FUNC,NSH,BLD]=2
JOB[1704,PERF,VPP]=csit-vpp-perf-1704-all
JOB[1704,VPP,BLD]="6 7 8 9 10 12 14 15 16 17"