Report: Remove VNF

- add comparisons for dnv

Change-Id: I123a925a3c91fba782f152b3cd73f1b63fe50c59
Signed-off-by: Tibor Frank <tifrank@cisco.com>

diff --git a/docs/report/index.html.template b/docs/report/index.html.template
index e29d16e..a74a7f1 100644 (file)
--- a/docs/report/index.html.template
+++ b/docs/report/index.html.template
@@ -21,7 +21,6 @@ CSIT-1901.1
     vpp_performance_tests/throughput_speedup_multi_core/index
     vpp_performance_tests/packet_latency_graphs/index
     vpp_performance_tests/soak_tests/index
-    vpp_performance_tests/nf_service_density/index
     vpp_performance_tests/comparisons/index
     vpp_performance_tests/throughput_trending
     vpp_performance_tests/test_environment

diff --git a/docs/report/introduction/introduction.rst b/docs/report/introduction/introduction.rst
index e9e6935..d789774 100644 (file)
--- a/docs/report/introduction/introduction.rst
+++ b/docs/report/introduction/introduction.rst
@@ -32,9 +32,6 @@ available for download.
      executions.
    - **Soak Tests**: Long duration soak tests are executed using PLRsearch
      algorithm.
-   - **NFV Service Density**: Network Function Virtualization (NFV) service
-     density tests focus on measuring total per server throughput at varied NFV
-     service "packing" densities with vswitch providing host dataplane.
    - **Comparisons**: Performance comparisons between VPP releases and
      between different testbed types.
    - **Throughput Trending**: References to continuous VPP performance

diff --git a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
index f15f693..42f39a8 100644 (file)
--- a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
+++ b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
@@ -134,3 +134,38 @@ Comparison tables in ASCII and CSV formats:
   - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-mrr.csv>`_
   - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr.csv>`_
   - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr.csv>`_
+
+2n-dnv
+~~~~~~
+
+NDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.txt>`_
+  - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.txt>`_
+  - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.csv>`_
+  - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.csv>`_
+
+PDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.txt>`_
+  - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.txt>`_
+  - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.csv>`_
+  - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.csv>`_
+
+MRR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.txt>`_
+  - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.txt>`_
+  - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.txt>`_
+  - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.csv>`_
+  - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.csv>`_
+  - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.csv>`_

diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 705e3ad..c1e6fba 100644 (file)
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -6,14 +6,6 @@ Changes in |csit-release|
 
 #. VPP PERFORMANCE TESTS
 
-   - **Service density 2n-skx tests**: Network Function Virtualization (NFV)
-     service density tests focus on measuring total per server throughput at
-     varied NFV service *packing* densities with vswitch providing host
-     dataplane. The goal is to compare and contrast performance of a shared
-     vswitch for different network topologies and virtualization technologies,
-     and their impact on vswitch performance and efficiency in a range of NFV
-     service configurations.
-
    - **Experimental Soak Tests**: Added performamce soak tests framework
      code for extended time duration tests and throughput discovery
      at given PLR and at give total test time e.g. minutes, hours,
@@ -73,7 +65,7 @@ List of known issues in |csit-release| for VPP performance tests:
 | 4  | `CSIT-1431                              | AVF 4 cores tests are sporadically failing. Under investigation.                                                                |
 |    | <https://jira.fd.io/browse/CSIT-1431>`_ |                                                                                                                                 |
 +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
-| 5  | `CSIT-1465                              | 4c VPP VM vhost tests failing on 3n-skx. Under investigation.                                                                  |
+| 5  | `CSIT-1465                              | 4c VPP VM vhost tests failing on 3n-skx. Under investigation.                                                                   |
 |    | <https://jira.fd.io/browse/CSIT-1465>`_ |                                                                                                                                 |
 +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+
 | 6  | `CSIT-1466                              | IPSecHW scale tests failing due to VPP reset. Under investigation.                                                              |

diff --git a/docs/report/vpp_performance_tests/index.rst b/docs/report/vpp_performance_tests/index.rst
index e33c37d..29770e7 100644 (file)
--- a/docs/report/vpp_performance_tests/index.rst
+++ b/docs/report/vpp_performance_tests/index.rst
@@ -9,11 +9,11 @@ VPP Performance
     throughput_speedup_multi_core/index
     packet_latency_graphs/index
     soak_tests/index
-    nf_service_density/index
     comparisons/index
     throughput_trending
     test_environment
     documentation/index
 
 ..
+    nf_service_density/index
     http_server_performance/index

diff --git a/resources/tools/presentation/input_data_files.py b/resources/tools/presentation/input_data_files.py
index a1ab243..262c6a0 100644 (file)
--- a/resources/tools/presentation/input_data_files.py
+++ b/resources/tools/presentation/input_data_files.py
@@ -167,7 +167,7 @@ def download_and_unzip_data_file(spec, job, build, pid, log):
     elif job.startswith("hc2vpp-"):
         url = spec.environment["urls"]["URL[JENKINS,HC]"]
     elif job.startswith("intel-dnv-"):
-        url = spec.environment["urls"]["URL[VIRL,DNV]"]
+        url = spec.environment["urls"]["URL[VIRL,DNV]"].format(release=job[-4:])
     else:
         raise PresentationError("No url defined for the job '{}'.".
                                 format(job))

diff --git a/resources/tools/presentation/specification.yaml b/resources/tools/presentation/specification.yaml
index 105c281..4e341db 100644 (file)
--- a/resources/tools/presentation/specification.yaml
+++ b/resources/tools/presentation/specification.yaml
@@ -75,7 +75,7 @@
   urls:
     URL[JENKINS,CSIT]: "https://jenkins.fd.io/view/csit/job"
     URL[JENKINS,HC]: "https://jenkins.fd.io/view/hc2vpp/job"
-    URL[VIRL,DNV]: "http://10.30.51.28/download/denverton/1901/robot-plugin.zip"
+    URL[VIRL,DNV]: "http://10.30.51.28/download/denverton/{release}/robot-plugin.zip"
     # URL[VIRL,DNV]: "http://download.it/from/nexus"
     URL[NEXUS]: "https://docs.fd.io/csit"
     DIR[NEXUS]: "report/_static/archive"
@@ -365,7 +365,6 @@
 
     # Denverton:
     intel-dnv-vpp:
-      # Not used
       intel-dnv-vpp-release-1901:
         - 1
 
@@ -546,6 +545,16 @@
 
     # Performance changes
 
+    intel-dnv-vpp-changes-2n-dnv-ref:
+      intel-dnv-vpp-release-1810:
+        - 1
+    intel-dnv-vpp-changes-2n-dnv-cmp:
+      intel-dnv-vpp-release-1901:
+        - 1
+    intel-dnv-vpp-changes-2n-dnv:
+      - "intel-dnv-vpp-changes-2n-dnv-ref"
+      - "intel-dnv-vpp-changes-2n-dnv-cmp"
+
     vpp-performance-changes-3n-hsw-h1:
       csit-vpp-perf-verify-1807-3n-hsw:
         - 11  # NDRPDR full 1
@@ -1464,10 +1473,12 @@
 
   builds:
 
-    # Not used
     intel-dnv-vpp-release-1901:
       - 1
 
+    intel-dnv-vpp-release-1810:
+      - 1
+
     csit-vpp-perf-verify-1807-3n-hsw:
       - 8   # MRR full
       - 9   # MRR sel
@@ -2256,6 +2267,147 @@
 ################################################################################
 # VPP
 
+# VPP Performance Changes 2n-dnv 1t1c pdr
+-
+  type: "table"
+  title: "VPP Performance Changes 2n-dnv 1t1c pdr"
+  algorithm: "table_performance_comparison"
+  output-file: "{DIR[STATIC,VPP]}/performance-changes-2n-dnv-1t1c-pdr"
+  reference:
+    title: "rls1810"
+    data: "intel-dnv-vpp-changes-2n-dnv-ref"
+  compare:
+    title: "rls1901.1"
+    data: "intel-dnv-vpp-changes-2n-dnv-cmp"
+  data: "intel-dnv-vpp-changes-2n-dnv"
+  include-tests: "PDR"
+  filter: "'1T1C' and ('NDRPDR' or 'NDRPDRDISC')"
+  parameters:
+  - "name"
+  - "parent"
+  - "throughput"
+
+# VPP Performance Changes 2n-dnv 2t2c pdr
+-
+  type: "table"
+  title: "VPP Performance Changes 2n-dnv 2t2c pdr"
+  algorithm: "table_performance_comparison"
+  output-file: "{DIR[STATIC,VPP]}/performance-changes-2n-dnv-2t2c-pdr"
+  reference:
+    title: "rls1810"
+    data: "intel-dnv-vpp-changes-2n-dnv-ref"
+  compare:
+    title: "rls1901.1"
+    data: "intel-dnv-vpp-changes-2n-dnv-cmp"
+  data: "intel-dnv-vpp-changes-2n-dnv"
+  include-tests: "PDR"
+  filter: "'2T2C' and ('NDRPDR' or 'NDRPDRDISC')"
+  parameters:
+  - "name"
+  - "parent"
+  - "throughput"
+
+# VPP Performance Changes 2n-dnv 1t1c ndr
+-
+  type: "table"
+  title: "VPP Performance Changes 2n-dnv 1t1c ndr"
+  algorithm: "table_performance_comparison"
+  output-file: "{DIR[STATIC,VPP]}/performance-changes-2n-dnv-1t1c-ndr"
+  reference:
+    title: "rls1810"
+    data: "intel-dnv-vpp-changes-2n-dnv-ref"
+  compare:
+    title: "rls1901.1"
+    data: "intel-dnv-vpp-changes-2n-dnv-cmp"
+  data: "intel-dnv-vpp-changes-2n-dnv"
+  include-tests: "NDR"
+  filter: "'1T1C' and ('NDRPDR' or 'NDRPDRDISC')"
+  parameters:
+  - "name"
+  - "parent"
+  - "throughput"
+
+# VPP Performance Changes 2n-dnv 2t2c ndr
+-
+  type: "table"
+  title: "VPP Performance Changes 2n-dnv 2t2c ndr"
+  algorithm: "table_performance_comparison"
+  output-file: "{DIR[STATIC,VPP]}/performance-changes-2n-dnv-2t2c-ndr"
+  reference:
+    title: "rls1810"
+    data: "intel-dnv-vpp-changes-2n-dnv-ref"
+  compare:
+    title: "rls1901.1"
+    data: "intel-dnv-vpp-changes-2n-dnv-cmp"
+  data: "intel-dnv-vpp-changes-2n-dnv"
+  include-tests: "NDR"
+  filter: "'2T2C' and ('NDRPDR' or 'NDRPDRDISC')"
+  parameters:
+  - "name"
+  - "parent"
+  - "throughput"
+  
+
+# VPP Performance Changes 2n-dnv 1t1c MRR
+-
+  type: "table"
+  title: "VPP Performance Changes 2n-dnv 1t1c MRR"
+  algorithm: "table_performance_comparison"
+  output-file: "{DIR[STATIC,VPP]}/performance-changes-2n-dnv-1t1c-mrr"
+  reference:
+    title: "rls1810"
+    data: "intel-dnv-vpp-changes-2n-dnv-ref"
+  compare:
+    title: "rls1901.1"
+    data: "intel-dnv-vpp-changes-2n-dnv-cmp"
+  data: "intel-dnv-vpp-changes-2n-dnv"
+  include-tests: "MRR"
+  filter: "'1T1C' and 'MRR'"
+  parameters:
+  - "name"
+  - "parent"
+  - "result"
+
+# VPP Performance Changes 2n-dnv 2t2c MRR
+-
+  type: "table"
+  title: "VPP Performance Changes 2n-dnv 2t2c MRR"
+  algorithm: "table_performance_comparison"
+  output-file: "{DIR[STATIC,VPP]}/performance-changes-2n-dnv-2t2c-mrr"
+  reference:
+    title: "rls1810"
+    data: "intel-dnv-vpp-changes-2n-dnv-ref"
+  compare:
+    title: "rls1901.1"
+    data: "intel-dnv-vpp-changes-2n-dnv-cmp"
+  data: "intel-dnv-vpp-changes-2n-dnv"
+  include-tests: "MRR"
+  filter: "'2T2C' and 'MRR'"
+  parameters:
+  - "name"
+  - "parent"
+  - "result"
+
+# VPP Performance Changes 2n-dnv 4t4c MRR
+-
+  type: "table"
+  title: "VPP Performance Changes 2n-dnv 4t4c MRR"
+  algorithm: "table_performance_comparison"
+  output-file: "{DIR[STATIC,VPP]}/performance-changes-2n-dnv-4t4c-mrr"
+  reference:
+    title: "rls1810"
+    data: "intel-dnv-vpp-changes-2n-dnv-ref"
+  compare:
+    title: "rls1901.1"
+    data: "intel-dnv-vpp-changes-2n-dnv-cmp"
+  data: "intel-dnv-vpp-changes-2n-dnv"
+  include-tests: "MRR"
+  filter: "'4T4C' and 'MRR'"
+  parameters:
+  - "name"
+  - "parent"
+  - "result"
+
 # VPP Performance Changes 3n-hsw 1t1c pdr
 -
   type: "table"

diff --git a/resources/tools/presentation/specification_parser.py b/resources/tools/presentation/specification_parser.py
index bf53883..b459bd3 100644 (file)
--- a/resources/tools/presentation/specification_parser.py
+++ b/resources/tools/presentation/specification_parser.py
@@ -370,7 +370,7 @@ class Specification(object):
 
         try:
             self._specification["environment"]["urls"] = \
-                self._replace_tags(self._cfg_yaml[idx]["urls"])
+                self._cfg_yaml[idx]["urls"]
         except KeyError:
             self._specification["environment"]["urls"] = None