CSIT-1052: Remove HC Performance from 1804 report

Change-Id: I924028fd6d56fff1859bc1fd27d7a5273d54435f
Signed-off-by: Tibor Frank <tifrank@cisco.com>

diff --git a/docs/report/honeycomb_performance_tests/csit_release_notes.rst b/docs/report/honeycomb_performance_tests/csit_release_notes.rst
deleted file mode 100644 (file)
index d79907d..0000000
--- a/docs/report/honeycomb_performance_tests/csit_release_notes.rst
+++ /dev/null
@@ -1,20 +0,0 @@
-CSIT Release Notes\r
-==================\r
-\r
-Changes in CSIT |release|\r
--------------------------\r
-\r
-#. No changes since previous release.\r
-\r
-Known Issues\r
-------------\r
-\r
-Here is the list of known issues in CSIT |release| for Honeycomb performance\r
-tests in VIRL:\r
-\r
-+---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
-| # | Issue                                      | Jira ID    | Description                                                                |\r
-+---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r
-| 1 | Intermittent failures in Honeycomb startup | HC2VPP-199 | During test setup Honeycomb sometimes fails to start. Does not appear to   |\r
-|   |                                            |            | affect measured results, only fails test execution ocassionally.           |\r
-+---+--------------------------------------------+------------+----------------------------------------------------------------------------+\r

diff --git a/docs/report/honeycomb_performance_tests/documentation.rst b/docs/report/honeycomb_performance_tests/documentation.rst
deleted file mode 100644 (file)
index 6b15bde..0000000
--- a/docs/report/honeycomb_performance_tests/documentation.rst
+++ /dev/null
@@ -1,5 +0,0 @@
-Documentation\r
-=============\r
-\r
-`CSIT Honeycomb Performance Tests Documentation`_ contains detailed\r
-functional description and input parameters for each test case.\r

diff --git a/docs/report/honeycomb_performance_tests/index.rst b/docs/report/honeycomb_performance_tests/index.rst
deleted file mode 100644 (file)
index 3177494..0000000
--- a/docs/report/honeycomb_performance_tests/index.rst
+++ /dev/null
@@ -1,11 +0,0 @@
-Honeycomb Performance Tests
-===========================
-
-.. toctree::
-
-    overview
-    csit_release_notes
-    test_environment
-    documentation
-    test_result_data
-

diff --git a/docs/report/honeycomb_performance_tests/overview.rst b/docs/report/honeycomb_performance_tests/overview.rst
deleted file mode 100644 (file)
index ee9788e..0000000
--- a/docs/report/honeycomb_performance_tests/overview.rst
+++ /dev/null
@@ -1,122 +0,0 @@
-Overview
-========
-
-Tested Physical Topologies
---------------------------
-
-CSIT VPP performance tests are executed on physical baremetal servers hosted by
-LF FD.io project. Testbed physical topology is shown in the figure below.::
-
-    +------------------------+           +------------------------+
-    |                        |           |                        |
-    |  +------------------+  |           |  +------------------+  |
-    |  |                  |  |           |  |                  |  |
-    |  |                  <----------------->                  |  |
-    |  |       DUT1       |  |           |  |       DUT2       |  |
-    |  +--^---------------+  |           |  +---------------^--+  |
-    |     |                  |           |                  |     |
-    |     |            SUT1  |           |  SUT2            |     |
-    +------------------------+           +------------------^-----+
-          |                                                 |
-          |                                                 |
-          |                  +-----------+                  |
-          |                  |           |                  |
-          +------------------>    TG     <------------------+
-                             |           |
-                             +-----------+
-
-SUT1 runs VPP SW application in Linux user-mode as a
-Device Under Test (DUT), and a python script to generate traffic. SUT2 and TG
-are unused.
-sical connectivity between SUTs and to TG is provided using
-different NIC model. Currently installed NIC models include:
-
-Performance tests involve sending Netconf requests over localhost to the
-Honeycomb listener port, and measuring response time.
-
-Note that reported performance results are specific to the SUTs tested.
-Current LF FD.io SUTs are based on Intel XEON E5-2699v3 2.3GHz CPUs. SUTs with
-other CPUs are likely to yield different results.
-
-For detailed LF FD.io test bed specification and physical topology please refer
-to `LF FDio CSIT testbed wiki page
-<https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
-
-Performance Tests Coverage
---------------------------
-
-As of right now, there is only a single Honeycomb performance test. Measuring
-response time for a simple read operation, performed synchronously and using
-single (not batch) requests.
-
-Currently the tests do not trigger automatically, but can be run on-demand from
-the hc2vpp project.
-
-Performance Tests Naming
-------------------------
-
-CSIT |release| follows a common structured naming convention for all
-performance and system functional tests, introduced in CSIT |release-1|.
-
-The naming should be intuitive for majority of the tests. Complete
-description of CSIT test naming convention is provided on `CSIT test naming wiki
-<https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
-
-Here few illustrative examples of the new naming usage for performance test
-suites:
-
-#. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P**
-
-   - *PortNICConfig-WireEncapsulation-PacketForwardingFunction-
-     PacketProcessingFunction1-...-PacketProcessingFunctionN-TestType*
-   - *10ge2p1x520-dot1q-l2bdbasemaclrn-ndrdisc.robot* => 2 ports of 10GE on
-     Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain baseline switching
-     with MAC learning, NDR throughput discovery.
-   - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrchk.robot* => 2 ports of 10GE
-     on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain baseline
-     switching with MAC learning, NDR throughput discovery.
-   - *10ge2p1x520-ethip4-ip4base-ndrdisc.robot* => 2 ports of 10GE on Intel
-     x520 NIC, IPv4 baseline routed forwarding, NDR throughput discovery.
-   - *10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot* => 2 ports of 10GE on
-     Intel x520 NIC, IPv6 scaled up routed forwarding, NDR throughput
-     discovery.
-
-#. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC,
-   P2V2P, NIC2VMchain2NIC, P2V2V2P**
-
-   - *PortNICConfig-WireEncapsulation-PacketForwardingFunction-
-     PacketProcessingFunction1-...-PacketProcessingFunctionN-VirtEncapsulation-
-     VirtPortConfig-VMconfig-TestType*
-   - *10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 ports
-     of 10GE on Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain
-     switching to/from two vhost interfaces and one VM, NDR throughput
-     discovery.
-   - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2
-     ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain
-     switching to/from two vhost interfaces and one VM, NDR throughput
-     discovery.
-   - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc.robot* => 2
-     ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain
-     switching to/from four vhost interfaces and two VMs, NDR throughput
-     discovery.
-
-Methodology: Multi-Core
------------------------
-
-**Multi-core Test** - CSIT |release| multi-core tests are executed in the
-following thread and core configurations:
-
-#. 1t - 1 Honeycomb Netconf thread on 1 CPU physical core.
-#. 8t - 8 Honeycomb Netconf thread on 8 CPU physical core.
-#. 16t - 16 Honeycomb Netconf thread on 16 CPU physical core.
-
-Traffic generator also uses multiple threads/cores, to simulate multiple
-Netconf clients accessing the Honeycomb server.
-
-Methodology: Performance measurement
-------------------------------------
-
-The following values are measured and reported in tests:
-
-- Average request rate. Averaged over the entire test duration, over all client
-  threads. Negative replies (if any) are not counted and are reported separately.

diff --git a/docs/report/honeycomb_performance_tests/test_environment.rst b/docs/report/honeycomb_performance_tests/test_environment.rst
deleted file mode 100644 (file)
index db187b2..0000000
--- a/docs/report/honeycomb_performance_tests/test_environment.rst
+++ /dev/null
@@ -1,22 +0,0 @@
-Test Environment
-================
-
-To execute performance tests, there are three identical testbeds, each testbed
-consists of two DUTs and one TG.
-
-Server HW Configuration
------------------------
-
-See `Performance HW Configuration <../vpp_performance_tests/test_environment.html>`_
-
-Additionally, configuration for the Honeycomb client:
-
-
-**Honeycomb Startup Command**
-
-Use the server mode JIT compiler, increase the default memory size,
-metaspace size, and enable NUMA optimizations for the JVM.
-
-::
-
-    $ java -server -Xms128m -Xmx512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=512m -XX:+UseNUMA -XX:+UseParallelGC

diff --git a/docs/report/honeycomb_performance_tests/test_result_data.rst b/docs/report/honeycomb_performance_tests/test_result_data.rst
deleted file mode 100644 (file)
index 9e791d6..0000000
--- a/docs/report/honeycomb_performance_tests/test_result_data.rst
+++ /dev/null
@@ -1,19 +0,0 @@
-Test Result Data
-================
-
-This section includes summary of Netconf read operation performance.
-Performance is reported for Honeycomb running in multiple configurations of
-netconf thread(s) and their physical CPU core(s) placement, and for different
-read operation targets.
-
-.. note::
-
-    Test results have been generated by
-    `FD.io test executor honeycomb performance jobs`_ with Robot Framework
-    result files csit-vpp-perf-\*.zip `archived here <../../_static/archive/>`_.
-
-Honeycomb + Netconf
-===================
-
-No new data has been generated in the course of this release. Results from
-the last successful run are available in `17.07 release report <https://docs.fd.io/csit/rls1707/report/honeycomb_performance_tests/test_result_data.html>`_.
\ No newline at end of file

diff --git a/docs/report/index.rst b/docs/report/index.rst
index bbcd5cb..158259f 100644 (file)
--- a/docs/report/index.rst
+++ b/docs/report/index.rst
@@ -1,4 +1,4 @@
-CSIT 18.01
+CSIT 18.04

 ==========
 
 .. toctree::
 ==========
 
 .. toctree::


@@ -7,7 +7,6 @@ CSIT 18.01
     introduction/index
     vpp_performance_tests/index
     dpdk_performance_tests/index
     introduction/index
     vpp_performance_tests/index
     dpdk_performance_tests/index

-    honeycomb_performance_tests/index

     vpp_functional_tests/index
     honeycomb_functional_tests/index
     vpp_unit_tests/index
     vpp_functional_tests/index
     honeycomb_functional_tests/index
     vpp_unit_tests/index

diff --git a/docs/report/introduction/general_notes.rst b/docs/report/introduction/general_notes.rst
index 994a53e..b51ed15 100644 (file)
--- a/docs/report/introduction/general_notes.rst
+++ b/docs/report/introduction/general_notes.rst
@@ -49,11 +49,6 @@ is listed separately, as follows:
    Testpmd/L3FWD application runs in host user-mode. TRex is used as a traffic
    generator.
 
    Testpmd/L3FWD application runs in host user-mode. TRex is used as a traffic
    generator.
 

-#. **Honeycomb Performance** - Honeycomb performance tests are executed in
-   physical FD.io testbeds, focusing on the performance of Honeycomb management
-   and programming functionality of VPP. Tests cover a range of CRUD operations
-   executed against VPP.
-

 #. **VPP Functional** - VPP functional tests are executed in virtual
    FD.io testbeds focusing on VPP packet processing functionality, including
    network data plane and in -line control plane. Tests cover vNIC-to-vNIC
 #. **VPP Functional** - VPP functional tests are executed in virtual
    FD.io testbeds focusing on VPP packet processing functionality, including
    network data plane and in -line control plane. Tests cover vNIC-to-vNIC

diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst
index 22bc29f..536d5d3 100644 (file)
--- a/docs/report/introduction/overview.rst
+++ b/docs/report/introduction/overview.rst
@@ -44,13 +44,6 @@ CSIT |release| report contains following main sections and sub-sections:
    executions; *Test Environment* - environment description; *Documentation* -
    CSIT source code documentation for DPDK performance tests.
 
    executions; *Test Environment* - environment description; *Documentation* -
    CSIT source code documentation for DPDK performance tests.
 

-#. **Honeycomb Performance Tests** - Honeycomb performance tests executed in
-   physical FD.io testbeds; *Overview* - tested topologies, test coverage
-   and naming specifics; *CSIT Release Notes* - changes in CSIT |release|,
-   added tests, environment or methodology changes, known CSIT issues;
-   *Test Environment* - environment description; *Documentation* - source
-   code documentation for Honeycomb performance tests.
-

 #. **VPP Functional Tests** - VPP functional tests executed in virtual
    FD.io testbeds; *Overview* - tested virtual topologies, test coverage and
    naming specifics; *CSIT Release Notes* - changes in CSIT |release|, added
 #. **VPP Functional Tests** - VPP functional tests executed in virtual
    FD.io testbeds; *Overview* - tested virtual topologies, test coverage and
    naming specifics; *CSIT Release Notes* - changes in CSIT |release|, added

diff --git a/resources/tools/presentation/conf.py b/resources/tools/presentation/conf.py
index c2e2ea0..58dd561 100644 (file)
--- a/resources/tools/presentation/conf.py
+++ b/resources/tools/presentation/conf.py
@@ -14,7 +14,8 @@
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.

-#
+
+

 import os
 import sys
 
 import os
 import sys
 


@@ -35,7 +36,7 @@ extensions = ['sphinxcontrib.programoutput',
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 

-# The suffix(es) of source filenames.
+# The suffix(es) of source file names.

 # You can specify multiple suffix as a list of string:
 #
 source_suffix = ['.rst', '.md']
 # You can specify multiple suffix as a list of string:
 #
 source_suffix = ['.rst', '.md']


@@ -53,19 +54,19 @@ author = u'FD.io CSIT'
 # built documents.
 #
 # The short X.Y version.
 # built documents.
 #
 # The short X.Y version.

-#version = u''
+# version = u''

 # The full version, including alpha/beta/rc tags.
 # The full version, including alpha/beta/rc tags.

-#release = u''
+# release = u''

 
 rst_epilog = """
 
 rst_epilog = """

-.. |release-1| replace:: rls1710
+.. |release-1| replace:: {prev_release}

 .. |srelease| replace:: {srelease}
 .. |vpp-release| replace:: VPP-{vpprelease} release
 .. |srelease| replace:: {srelease}
 .. |vpp-release| replace:: VPP-{vpprelease} release

-.. |vpp-release-1| replace:: VPP-17.10 release
+.. |vpp-release-1| replace:: VPP-{vpp_prev_release} release

 .. |dpdk-release| replace:: DPDK {dpdkrelease}
 .. |dpdk-release| replace:: DPDK {dpdkrelease}

-.. |trex-release| replace:: TRex v2.35
-.. |virl-image-ubuntu| replace:: ubuntu-16.04.1_2017-10-21_2.0
-.. |virl-image-centos| replace:: centos-7.3-1611_2017-02-23_1.4
+.. |trex-release| replace:: TRex {trex_version}
+.. |virl-image-ubuntu| replace:: {csit_ubuntu_ver}
+.. |virl-image-centos| replace:: {csit_centos_ver}

 
 .. _pdf version of this report: https://docs.fd.io/csit/{release}/report/_static/archive/csit_{release}.pdf
 .. _tag documentation rst file: https://git.fd.io/csit/tree/docs/tag_documentation.rst?h={release}
 
 .. _pdf version of this report: https://docs.fd.io/csit/{release}/report/_static/archive/csit_{release}.pdf
 .. _tag documentation rst file: https://git.fd.io/csit/tree/docs/tag_documentation.rst?h={release}


@@ -91,7 +92,16 @@ rst_epilog = """
 .. _FD.io VPP compile job: https://jenkins.fd.io/view/vpp/job/vpp-merge-{srelease}-ubuntu1604/
 .. _FD.io DPDK compile job: https://jenkins.fd.io/view/deb-dpdk/job/deb_dpdk-merge-{sdpdkrelease}-ubuntu1604/
 .. _CSIT Testbed Setup: https://git.fd.io/csit/tree/resources/tools/testbed-setup/README.md?h={release}
 .. _FD.io VPP compile job: https://jenkins.fd.io/view/vpp/job/vpp-merge-{srelease}-ubuntu1604/
 .. _FD.io DPDK compile job: https://jenkins.fd.io/view/deb-dpdk/job/deb_dpdk-merge-{sdpdkrelease}-ubuntu1604/
 .. _CSIT Testbed Setup: https://git.fd.io/csit/tree/resources/tools/testbed-setup/README.md?h={release}

-""".format(release='rls1801', srelease='1801', vpprelease='18.01', dpdkrelease='17.11', sdpdkrelease='1711')
+""".format(release='rls1804',
+           prev_release='rls1801',
+           srelease='1804',
+           vpprelease='18.04',
+           vpp_prev_release='18.01',
+           dpdkrelease='18.02',
+           sdpdkrelease='1802',
+           trex_version='v2.35',
+           csit_ubuntu_ver='csit-ubuntu-16.04.1_2018-03-07_2.1',
+           csit_centos_ver='csit-centos-7.4-1711_2018-03-20_1.9')

 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.