CSIT-1397: Create Heatmap graphs

 - colorscales

Change-Id: I86e72432ffa850e76aed6d061461519c5f095a1f
Signed-off-by: Tibor Frank <tifrank@cisco.com>

diff --git a/docs/report/index.html.template b/docs/report/index.html.template
index e4fc829..7737f49 100644 (file)
--- a/docs/report/index.html.template
+++ b/docs/report/index.html.template
@@ -20,6 +20,8 @@ CSIT-1901
     vpp_performance_tests/packet_throughput_graphs/index
     vpp_performance_tests/throughput_speedup_multi_core/index
     vpp_performance_tests/packet_latency_graphs/index
     vpp_performance_tests/packet_throughput_graphs/index
     vpp_performance_tests/throughput_speedup_multi_core/index
     vpp_performance_tests/packet_latency_graphs/index

+    vpp_performance_tests/soak_tests/index
+    vpp_performance_tests/nf_service_density/index

     vpp_performance_tests/http_server_performance/index
     vpp_performance_tests/comparisons/index
     vpp_performance_tests/throughput_trending
     vpp_performance_tests/http_server_performance/index
     vpp_performance_tests/comparisons/index
     vpp_performance_tests/throughput_trending

diff --git a/docs/report/introduction/report_history.rst b/docs/report/introduction/report_history.rst
index 77bf937..cec1e6e 100644 (file)
--- a/docs/report/introduction/report_history.rst
+++ b/docs/report/introduction/report_history.rst
@@ -3,12 +3,12 @@ Report History
 
 FD.io CSIT-1901 Report history and per .[ww] revision changes are listed below.
 
 
 FD.io CSIT-1901 Report history and per .[ww] revision changes are listed below.
 

-+----------------+----------------------------------------------------------------+
-| .[ww] Revision | Changes                                                        |
-+================+================================================================+
-| .04            | Initial version                                                |
-|                |                                                                |
-+----------------+----------------------------------------------------------------+
++----------------+------------------------------------------------------------+
+| .[ww] Revision | Changes                                                    |
++================+============================================================+
+| .06            | Initial version                                            |
+|                |                                                            |
++----------------+------------------------------------------------------------+

 
 FD.io CSIT Reports follow CSIT-[yy][mm].[ww] numbering format, with version
 denoted by concatenation of two digit year [yy] and two digit month [mm], and
 
 FD.io CSIT Reports follow CSIT-[yy][mm].[ww] numbering format, with version
 denoted by concatenation of two digit year [yy] and two digit month [mm], and

diff --git a/docs/report/introduction/test_environment_intro.rst b/docs/report/introduction/test_environment_intro.rst
index 7763e9d..1965645 100644 (file)
--- a/docs/report/introduction/test_environment_intro.rst
+++ b/docs/report/introduction/test_environment_intro.rst
@@ -1,5 +1,3 @@
-.. _test_environment:
-

 Test Environment
 ================
 
 Test Environment
 ================
 

diff --git a/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst b/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst
index c749d4c..9288ed7 100644 (file)
--- a/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst
+++ b/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst
@@ -4,8 +4,8 @@ CentOS
 .. note::
 
     Data sources for reported test configurations: i) `FD.io test executor vpp
 .. note::
 
     Data sources for reported test configurations: i) `FD.io test executor vpp

-    functional jobs`_, ii) archived FD.io jobs test result `output files
-    <../../_static/archive/>`_.
+    functional jobs using CentOs`_, ii) archived FD.io jobs test result
+    `output files <../../_static/archive/>`_.

 
 .. toctree::
 
 
 .. toctree::
 

diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index d54a096..5d87b85 100644 (file)
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -10,7 +10,7 @@ Changes in |csit-release|
      updated with a configurable burst MRR parameters: trial duration
      and number of trials in a single burst. Enables a new Burst MRR
      (BMRR for short) methodology for more precise performance
      updated with a configurable burst MRR parameters: trial duration
      and number of trials in a single burst. Enables a new Burst MRR
      (BMRR for short) methodology for more precise performance

-     trending. See updated :ref:`performance_test_methodology` section
+     trending. See updated :ref:`test_methodology` section

      for more details.
 
    - **2n-skx tests**: Added performamce tests for 2n-skx (2-Node Xeon
      for more details.
 
    - **2n-skx tests**: Added performamce tests for 2n-skx (2-Node Xeon


@@ -41,7 +41,7 @@ Changes in |csit-release|
      code for extended time duration tests and throughput discovery
      at given PLR and at give total test time e.g. minutes, hours,
      days, weeks, months, years. See updated
      code for extended time duration tests and throughput discovery
      at given PLR and at give total test time e.g. minutes, hours,
      days, weeks, months, years. See updated

-     :ref:`performance_test_methodology` section for more details.
+     :ref:`test_methodology` section for more details.

 
    - **Trending Tests BMRR**: Used new Burst MRR (BMRR) tests for daily
      trending.
 
    - **Trending Tests BMRR**: Used new Burst MRR (BMRR) tests for daily
      trending.

diff --git a/docs/vpp-device.rst b/docs/vpp-device.rst
index c071771..96d6d75 100644 (file)
--- a/docs/vpp-device.rst
+++ b/docs/vpp-device.rst
@@ -199,8 +199,10 @@ that only Y number of simulations are spawned on any given HW node.
   *root* user account as primary. From the jenkins slave it will be able to
   connect via env variable, since the jenkins slave doesn't actually know what
   host its running on.
   *root* user account as primary. From the jenkins slave it will be able to
   connect via env variable, since the jenkins slave doesn't actually know what
   host its running on.

+

   ::
   ::

-    ssh -p 6022 root@10.30.51.node
+
+      ssh -p 6022 root@10.30.51.node

 
 csit-sut-dcr
 ~~~~~~~~~~~~
 
 csit-sut-dcr
 ~~~~~~~~~~~~

diff --git a/resources/tools/presentation/conf.py b/resources/tools/presentation/conf.py
index 2ef414f..58d4443 100644 (file)
--- a/resources/tools/presentation/conf.py
+++ b/resources/tools/presentation/conf.py
@@ -45,7 +45,8 @@ source_suffix = ['.rst', '.md']
 master_doc = 'index'
 
 # General information about the project.
 master_doc = 'index'
 
 # General information about the project.

-project = u'FD.io CSIT-1901.04'
+report_week = '06'
+project = u'FD.io CSIT-1901.{week}'.format(week=report_week)

 copyright = u'2019, FD.io'
 author = u'FD.io CSIT'
 
 copyright = u'2019, FD.io'
 author = u'FD.io CSIT'
 


@@ -105,7 +106,7 @@ rst_epilog = """
 .. _CSIT Testbed Setup: https://git.fd.io/csit/tree/resources/tools/testbed-setup/README.md?h={release}
 .. _K8s configuration files: https://github.com/FDio/csit/tree/{release}/resources/templates/kubernetes
 """.format(release='rls1901',
 .. _CSIT Testbed Setup: https://git.fd.io/csit/tree/resources/tools/testbed-setup/README.md?h={release}
 .. _K8s configuration files: https://github.com/FDio/csit/tree/{release}/resources/templates/kubernetes
 """.format(release='rls1901',

-           report_week='04',
+           report_week=report_week,

            prev_release='rls1810',
            srelease='1901',
            csitrelease='1901',
            prev_release='rls1810',
            srelease='1901',
            csitrelease='1901',

diff --git a/resources/tools/presentation/generator_plots.py b/resources/tools/presentation/generator_plots.py
index 7cdcb62..747c3a2 100644 (file)
--- a/resources/tools/presentation/generator_plots.py
+++ b/resources/tools/presentation/generator_plots.py
@@ -35,6 +35,8 @@ COLORS = ["SkyBlue", "Olive", "Purple", "Coral", "Indigo", "Pink",
           "LightGreen", "LightSeaGreen", "LightSkyBlue", "Maroon",
           "MediumSeaGreen", "SeaGreen", "LightSlateGrey"]
 
           "LightGreen", "LightSeaGreen", "LightSkyBlue", "Maroon",
           "MediumSeaGreen", "SeaGreen", "LightSlateGrey"]
 

+REGEX_NIC = re.compile(r'\d*ge\dp\d\D*\d*-')
+

 
 def generate_plots(spec, data):
     """Generate all plots specified in the specification file.
 
 def generate_plots(spec, data):
     """Generate all plots specified in the specification file.


@@ -145,21 +147,14 @@ def plot_performance_box(plot, input_data):
     df.head()
     y_max = list()
     for i, col in enumerate(df.columns):
     df.head()
     y_max = list()
     for i, col in enumerate(df.columns):

+        tst_name = re.sub(REGEX_NIC, "",
+                          col.lower().replace('-ndrpdr', '').
+                          replace('2n1l-', ''))

         name = "{nr}. ({samples:02d} run{plural}) {name}".\
             format(nr=(i + 1),
                    samples=nr_of_samples[i],
                    plural='s' if nr_of_samples[i] > 1 else '',
         name = "{nr}. ({samples:02d} run{plural}) {name}".\
             format(nr=(i + 1),
                    samples=nr_of_samples[i],
                    plural='s' if nr_of_samples[i] > 1 else '',

-                   name=col.lower().replace('-ndrpdr', ''))
-        if len(name) > 50:
-            name_lst = name.split('-')
-            name = ""
-            split_name = True
-            for segment in name_lst:
-                if (len(name) + len(segment) + 1) > 50 and split_name:
-                    name += "<br>    "
-                    split_name = False
-                name += segment + '-'
-            name = name[:-1]
+                   name=tst_name)

 
         logging.debug(name)
         traces.append(plgo.Box(x=[str(i + 1) + '.'] * len(df[col]),
 
         logging.debug(name)
         traces.append(plgo.Box(x=[str(i + 1) + '.'] * len(df[col]),


@@ -178,7 +173,7 @@ def plot_performance_box(plot, input_data):
         # Create plot
         layout = deepcopy(plot["layout"])
         if layout.get("title", None):
         # Create plot
         layout = deepcopy(plot["layout"])
         if layout.get("title", None):

-            layout["title"] = "<b>Packet Throughput:</b> {0}". \
+            layout["title"] = "<b>Throughput:</b> {0}". \

                 format(layout["title"])
         if y_max:
             layout["yaxis"]["range"] = [0, max(y_max)]
                 format(layout["title"])
         if y_max:
             layout["yaxis"]["range"] = [0, max(y_max)]


@@ -383,13 +378,14 @@ def plot_soak_boxes(plot, input_data):
     for test_name, test_data in y_sorted.items():
         idx += 1
         name = "{nr}. {name}".\
     for test_name, test_data in y_sorted.items():
         idx += 1
         name = "{nr}. {name}".\

-            format(nr=idx, name=test_name.lower().replace('-soak', ''))
-        if len(name) > 50:
+            format(nr=idx, name=test_name.lower().replace('-soak', '').
+                   replace('2n1l-', ''))
+        if len(name) > 55:

             name_lst = name.split('-')
             name = ""
             split_name = True
             for segment in name_lst:
             name_lst = name.split('-')
             name = ""
             split_name = True
             for segment in name_lst:

-                if (len(name) + len(segment) + 1) > 50 and split_name:
+                if (len(name) + len(segment) + 1) > 55 and split_name:

                     name += "<br>    "
                     split_name = False
                 name += segment + '-'
                     name += "<br>    "
                     split_name = False
                 name += segment + '-'


@@ -405,10 +401,8 @@ def plot_soak_boxes(plot, input_data):
         if y_base:
             y_base /= 1000000
 
         if y_base:
             y_base /= 1000000
 

-        hovertext = ("{name}<br>"
-                     "Upper bound: {upper:.2f}Mpps<br>"
-                     "Lower bound: {lower:.2f}Mpps".format(name=name,
-                                                           upper=y_val,
+        hovertext = ("Upper bound: {upper:.2f}<br>"
+                     "Lower bound: {lower:.2f}".format(upper=y_val,

                                                            lower=y_base))
         traces.append(plgo.Bar(x=[str(idx) + '.', ],
                                # +0.05 to see the value in case lower == upper
                                                            lower=y_base))
         traces.append(plgo.Bar(x=[str(idx) + '.', ],
                                # +0.05 to see the value in case lower == upper


@@ -421,7 +415,7 @@ def plot_soak_boxes(plot, input_data):
         # Create plot
         layout = deepcopy(plot["layout"])
         if layout.get("title", None):
         # Create plot
         layout = deepcopy(plot["layout"])
         if layout.get("title", None):

-            layout["title"] = "<b>Soak Tests:</b> {0}". \
+            layout["title"] = "<b>Throughput:</b> {0}". \

                 format(layout["title"])
         if y_max:
             layout["yaxis"]["range"] = [0, y_max + 1]
                 format(layout["title"])
         if y_max:
             layout["yaxis"]["range"] = [0, y_max + 1]


@@ -541,17 +535,8 @@ def plot_latency_error_bars(plot, input_data):
     y_maxs = list()
     nr_of_samples = list()
     for key, val in y_sorted.items():
     y_maxs = list()
     nr_of_samples = list()
     for key, val in y_sorted.items():

-        name = "-".join(key.split("-")[1:-1])
-        if len(name) > 50:
-            name_lst = name.split('-')
-            name = ""
-            split_name = True
-            for segment in name_lst:
-                if (len(name) + len(segment) + 1) > 50 and split_name:
-                    name += "<br>"
-                    split_name = False
-                name += segment + '-'
-            name = name[:-1]
+        name = re.sub(REGEX_NIC, "", key.replace('-ndrpdr', '').
+                      replace('2n1l-', ''))

         x_vals.append(name)  # dir 1
         y_vals.append(mean(val[1]) if val[1] else None)
         y_mins.append(mean(val[0]) if val[0] else None)
         x_vals.append(name)  # dir 1
         y_vals.append(mean(val[1]) if val[1] else None)
         y_mins.append(mean(val[0]) if val[0] else None)


@@ -641,7 +626,7 @@ def plot_latency_error_bars(plot, input_data):
                      format(plot["output-file"], plot["output-file-type"]))
         layout = deepcopy(plot["layout"])
         if layout.get("title", None):
                      format(plot["output-file"], plot["output-file-type"]))
         layout = deepcopy(plot["layout"])
         if layout.get("title", None):

-            layout["title"] = "<b>Packet Latency:</b> {0}".\
+            layout["title"] = "<b>Latency:</b> {0}".\

                 format(layout["title"])
         layout["annotations"] = annotations
         plpl = plgo.Figure(data=traces, layout=layout)
                 format(layout["title"])
         layout["annotations"] = annotations
         plpl = plgo.Figure(data=traces, layout=layout)


@@ -730,18 +715,8 @@ def plot_throughput_speedup_analysis(plot, input_data):
     for test_name, test_vals in y_vals.items():
         try:
             if test_vals["1"][1]:
     for test_name, test_vals in y_vals.items():
         try:
             if test_vals["1"][1]:

-                name = "-".join(test_name.split('-')[1:-1])
-                if len(name) > 50:
-                    name_lst = name.split('-')
-                    name = ""
-                    split_name = True
-                    for segment in name_lst:
-                        if (len(name) + len(segment) + 1) > 50 and split_name:
-                            name += "<br>"
-                            split_name = False
-                        name += segment + '-'
-                    name = name[:-1]
-
+                name = re.sub(REGEX_NIC, "", test_name.replace('-ndrpdr', '').
+                              replace('2n1l-', ''))

                 vals[name] = dict()
                 y_val_1 = test_vals["1"][0] / 1000000.0
                 y_val_2 = test_vals["2"][0] / 1000000.0 if test_vals["2"][0] \
                 vals[name] = dict()
                 y_val_1 = test_vals["1"][0] / 1000000.0
                 y_val_2 = test_vals["2"][0] / 1000000.0 if test_vals["2"][0] \


@@ -761,12 +736,14 @@ def plot_throughput_speedup_analysis(plot, input_data):
                                        test_vals["4"][1]]
 
                 try:
                                        test_vals["4"][1]]
 
                 try:

-                    val_max = max(max(vals[name]["val"], vals[name]["ideal"]))
+                    # val_max = max(max(vals[name]["val"], vals[name]["ideal"]))
+                    val_max = max(vals[name]["val"])

                 except ValueError as err:
                     logging.error(err)
                     continue
                 if val_max:
                 except ValueError as err:
                     logging.error(err)
                     continue
                 if val_max:

-                    y_max.append(int((val_max / 10) + 1) * 10)
+                    # y_max.append(int((val_max / 10) + 1) * 10)
+                    y_max.append(val_max)

 
                 if y_val_2:
                     vals[name]["rel"][1] = round(y_val_2 / y_val_1, 2)
 
                 if y_val_2:
                     vals[name]["rel"][1] = round(y_val_2 / y_val_1, 2)


@@ -818,7 +795,9 @@ def plot_throughput_speedup_analysis(plot, input_data):
         for tag in order:
             for test, tags in y_tags_l.items():
                 if tag.lower() in tags:
         for tag in order:
             for test, tags in y_tags_l.items():
                 if tag.lower() in tags:

-                    name = "-".join(test.split('-')[1:-1])
+                    name = re.sub(REGEX_NIC, "",
+                                  test.replace('-ndrpdr', '').
+                                  replace('2n1l-', ''))

                     try:
                         y_sorted[name] = vals.pop(name)
                         y_tags_l.pop(test)
                     try:
                         y_sorted[name] = vals.pop(name)
                         y_tags_l.pop(test)


@@ -840,35 +819,36 @@ def plot_throughput_speedup_analysis(plot, input_data):
         logging.error(err)
         return
     nic_limit /= 1000000.0
         logging.error(err)
         return
     nic_limit /= 1000000.0

-    if nic_limit < threshold:
-        traces.append(plgo.Scatter(
-            x=x_vals,
-            y=[nic_limit, ] * len(x_vals),
-            name="NIC: {0:.2f}Mpps".format(nic_limit),
-            showlegend=False,
-            mode="lines",
-            line=dict(
-                dash="dot",
-                color=COLORS[-1],
-                width=1),
-            hoverinfo="none"
-        ))
-        annotations.append(dict(
-            x=1,
-            y=nic_limit,
-            xref="x",
-            yref="y",
-            xanchor="left",
-            yanchor="bottom",
-            text="NIC: {0:.2f}Mpps".format(nic_limit),
-            font=dict(
-                size=14,
-                color=COLORS[-1],
-            ),
-            align="left",
-            showarrow=False
-        ))
-        y_max.append(int((nic_limit / 10) + 1) * 10)
+    # if nic_limit < threshold:
+    traces.append(plgo.Scatter(
+        x=x_vals,
+        y=[nic_limit, ] * len(x_vals),
+        name="NIC: {0:.2f}Mpps".format(nic_limit),
+        showlegend=False,
+        mode="lines",
+        line=dict(
+            dash="dot",
+            color=COLORS[-1],
+            width=1),
+        hoverinfo="none"
+    ))
+    annotations.append(dict(
+        x=1,
+        y=nic_limit,
+        xref="x",
+        yref="y",
+        xanchor="left",
+        yanchor="bottom",
+        text="NIC: {0:.2f}Mpps".format(nic_limit),
+        font=dict(
+            size=14,
+            color=COLORS[-1],
+        ),
+        align="left",
+        showarrow=False
+    ))
+    # y_max.append(int((nic_limit / 10) + 1) * 10)
+    y_max.append(nic_limit)

 
     lnk_limit /= 1000000.0
     if lnk_limit < threshold:
 
     lnk_limit /= 1000000.0
     if lnk_limit < threshold:


@@ -899,10 +879,12 @@ def plot_throughput_speedup_analysis(plot, input_data):
             align="left",
             showarrow=False
         ))
             align="left",
             showarrow=False
         ))

-        y_max.append(int((lnk_limit / 10) + 1) * 10)
+        # y_max.append(int((lnk_limit / 10) + 1) * 10)
+        y_max.append(lnk_limit)

 
     pci_limit /= 1000000.0
 
     pci_limit /= 1000000.0

-    if pci_limit < threshold:
+    if (pci_limit < threshold and
+        (pci_limit < lnk_limit * 0.95 or lnk_limit > lnk_limit * 1.05)):

         traces.append(plgo.Scatter(
             x=x_vals,
             y=[pci_limit, ] * len(x_vals),
         traces.append(plgo.Scatter(
             x=x_vals,
             y=[pci_limit, ] * len(x_vals),


@@ -930,7 +912,8 @@ def plot_throughput_speedup_analysis(plot, input_data):
             align="left",
             showarrow=False
         ))
             align="left",
             showarrow=False
         ))

-        y_max.append(int((pci_limit / 10) + 1) * 10)
+        # y_max.append(int((pci_limit / 10) + 1) * 10)
+        y_max.append(pci_limit)

 
     # Perfect and measured:
     cidx = 0
 
     # Perfect and measured:
     cidx = 0


@@ -990,6 +973,8 @@ def plot_throughput_speedup_analysis(plot, input_data):
         if layout.get("title", None):
             layout["title"] = "<b>Speedup Multi-core:</b> {0}". \
                 format(layout["title"])
         if layout.get("title", None):
             layout["title"] = "<b>Speedup Multi-core:</b> {0}". \
                 format(layout["title"])

+        # layout["yaxis"]["range"] = [0, int((max(y_max) / 10) + 1) * 10]
+        layout["yaxis"]["range"] = [0, int(max(y_max) * 1.1)]

         layout["annotations"].extend(annotations)
         plpl = plgo.Figure(data=traces, layout=layout)
 
         layout["annotations"].extend(annotations)
         plpl = plgo.Figure(data=traces, layout=layout)
 


@@ -1097,6 +1082,9 @@ def plot_service_density_heatmap(plot, input_data):
     """
 
     REGEX_CN = re.compile(r'^(\d*)R(\d*)C$')
     """
 
     REGEX_CN = re.compile(r'^(\d*)R(\d*)C$')

+    REGEX_TEST_NAME = re.compile(r'^.*-(\d+vhost|\d+memif)-'
+                                 r'(\d+chain|\d+pipe)-'
+                                 r'(\d+vm|\d+dcr|\d+drc).*$')

 
     txt_chains = list()
     txt_nodes = list()
 
     txt_chains = list()
     txt_nodes = list()


@@ -1106,7 +1094,7 @@ def plot_service_density_heatmap(plot, input_data):
     logging.info("    Creating the data set for the {0} '{1}'.".
                  format(plot.get("type", ""), plot.get("title", "")))
     data = input_data.filter_data(plot, continue_on_error=True)
     logging.info("    Creating the data set for the {0} '{1}'.".
                  format(plot.get("type", ""), plot.get("title", "")))
     data = input_data.filter_data(plot, continue_on_error=True)

-    if data is None:
+    if data is None or data.empty:

         logging.error("No data.")
         return
 
         logging.error("No data.")
         return
 


@@ -1121,10 +1109,18 @@ def plot_service_density_heatmap(plot, input_data):
                         break
                 else:
                     continue
                         break
                 else:
                     continue

+                groups = re.search(REGEX_TEST_NAME, test["name"])
+                if groups and len(groups.groups()) == 3:
+                    hover_name = "{vhost}-{chain}-{vm}".format(
+                        vhost=str(groups.group(1)),
+                        chain=str(groups.group(2)),
+                        vm=str(groups.group(3)))
+                else:
+                    hover_name = ""

                 if vals.get(c, None) is None:
                     vals[c] = dict()
                 if vals[c].get(n, None) is None:
                 if vals.get(c, None) is None:
                     vals[c] = dict()
                 if vals[c].get(n, None) is None:

-                    vals[c][n] = dict(name=test["name"],
+                    vals[c][n] = dict(name=hover_name,

                                       vals=list(),
                                       nr=None,
                                       mean=None,
                                       vals=list(),
                                       nr=None,
                                       mean=None,


@@ -1141,6 +1137,10 @@ def plot_service_density_heatmap(plot, input_data):
                 if result:
                     vals[c][n]["vals"].append(result)
 
                 if result:
                     vals[c][n]["vals"].append(result)
 

+    if not vals:
+        logging.error("No data.")
+        return
+

     for key_c in vals.keys():
         txt_chains.append(key_c)
         for key_n in vals[key_c].keys():
     for key_c in vals.keys():
         txt_chains.append(key_c)
         for key_n in vals[key_c].keys():


@@ -1148,9 +1148,9 @@ def plot_service_density_heatmap(plot, input_data):
             if vals[key_c][key_n]["vals"]:
                 vals[key_c][key_n]["nr"] = len(vals[key_c][key_n]["vals"])
                 vals[key_c][key_n]["mean"] = \
             if vals[key_c][key_n]["vals"]:
                 vals[key_c][key_n]["nr"] = len(vals[key_c][key_n]["vals"])
                 vals[key_c][key_n]["mean"] = \

-                    round(mean(vals[key_c][key_n]["vals"]) / 1000000, 2)
+                    round(mean(vals[key_c][key_n]["vals"]) / 1000000, 1)

                 vals[key_c][key_n]["stdev"] = \
                 vals[key_c][key_n]["stdev"] = \

-                    round(stdev(vals[key_c][key_n]["vals"]) / 1000000, 2)
+                    round(stdev(vals[key_c][key_n]["vals"]) / 1000000, 1)

     txt_nodes = list(set(txt_nodes))
 
     txt_chains = sorted(txt_chains, key=lambda chain: int(chain))
     txt_nodes = list(set(txt_nodes))
 
     txt_chains = sorted(txt_chains, key=lambda chain: int(chain))


@@ -1168,13 +1168,23 @@ def plot_service_density_heatmap(plot, input_data):
                 val = None
             data[c - 1].append(val)
 
                 val = None
             data[c - 1].append(val)
 

+    # Colorscales:
+    my_green = [[0.0, 'rgb(235, 249, 242)'],
+                [1.0, 'rgb(45, 134, 89)']]
+
+    my_blue = [[0.0, 'rgb(236, 242, 248)'],
+               [1.0, 'rgb(57, 115, 172)']]
+
+    my_grey = [[0.0, 'rgb(230, 230, 230)'],
+               [1.0, 'rgb(102, 102, 102)']]
+

     hovertext = list()
     annotations = list()
 
     hovertext = list()
     annotations = list()
 

-    text = ("{name}<br>"
-            "No. of Samples: {nr}<br>"
-            "Throughput: {val}<br>"
-            "Stdev: {stdev}")
+    text = ("Test: {name}<br>"
+            "Runs: {nr}<br>"
+            "Thput: {val}<br>"
+            "StDev: {stdev}")

 
     for c in range(len(txt_chains)):
         hover_line = list()
 
     for c in range(len(txt_chains)):
         hover_line = list()


@@ -1206,22 +1216,30 @@ def plot_service_density_heatmap(plot, input_data):
                      y=chains,
                      z=data,
                      colorbar=dict(
                      y=chains,
                      z=data,
                      colorbar=dict(

-                         title="Packet Throughput [Mpps]",
+                         title=plot.get("z-axis", ""),

                          titleside="right",
                          titlefont=dict(
                          titleside="right",
                          titlefont=dict(

-                            size=14
+                            size=16

                          ),
                          ),

+                         tickfont=dict(
+                             size=16,
+                         ),
+                         tickformat=".1f",
+                         yanchor="bottom",
+                         y=-0.02,
+                         len=0.925,

                      ),
                      showscale=True,
                      ),
                      showscale=True,

-                     colorscale="Reds",
+                     colorscale=my_green,

                      text=hovertext,
                      hoverinfo="text")
     ]
 
     for idx, item in enumerate(txt_nodes):
                      text=hovertext,
                      hoverinfo="text")
     ]
 
     for idx, item in enumerate(txt_nodes):

+        # X-axis, numbers:

         annotations.append(dict(
             x=idx+1,
         annotations.append(dict(
             x=idx+1,

-            y=0,
+            y=0.05,

             xref="x",
             yref="y",
             xanchor="center",
             xref="x",
             yref="y",
             xanchor="center",


@@ -1234,8 +1252,9 @@ def plot_service_density_heatmap(plot, input_data):
             showarrow=False
         ))
     for idx, item in enumerate(txt_chains):
             showarrow=False
         ))
     for idx, item in enumerate(txt_chains):

+        # Y-axis, numbers:

         annotations.append(dict(
         annotations.append(dict(

-            x=0.3,
+            x=0.35,

             y=idx+1,
             xref="x",
             yref="y",
             y=idx+1,
             xref="x",
             yref="y",


@@ -1248,30 +1267,30 @@ def plot_service_density_heatmap(plot, input_data):
             align="center",
             showarrow=False
         ))
             align="center",
             showarrow=False
         ))

-    # X-axis:
+    # X-axis, title:

     annotations.append(dict(
         x=0.55,
     annotations.append(dict(
         x=0.55,

-        y=1.05,
+        y=-0.15,

         xref="paper",
         xref="paper",

-        yref="paper",
+        yref="y",

         xanchor="center",
         xanchor="center",

-        yanchor="middle",
-        text="<b>No. of Network Functions per Service Instance</b>",
+        yanchor="bottom",
+        text=plot.get("x-axis", ""),

         font=dict(
             size=16,
         ),
         align="center",
         showarrow=False
     ))
         font=dict(
             size=16,
         ),
         align="center",
         showarrow=False
     ))

-    # Y-axis:
+    # Y-axis, title:

     annotations.append(dict(
     annotations.append(dict(

-        x=-0.04,
+        x=-0.1,

         y=0.5,
         y=0.5,

-        xref="paper",
+        xref="x",

         yref="paper",
         xanchor="center",
         yanchor="middle",
         yref="paper",
         xanchor="center",
         yanchor="middle",

-        text="<b>No. of Service Instances</b>",
+        text=plot.get("y-axis", ""),

         font=dict(
             size=16,
         ),
         font=dict(
             size=16,
         ),


@@ -1288,80 +1307,20 @@ def plot_service_density_heatmap(plot, input_data):
             direction='up',
             buttons=list([
                 dict(
             direction='up',
             buttons=list([
                 dict(

-                    args=[{"colorscale": "Reds", "reversescale": False}],
-                    label="Red",
+                    args=[{"colorscale": [my_green, ], "reversescale": False}],
+                    label="Green",

                     method="update"
                 ),
                 dict(
                     method="update"
                 ),
                 dict(

-                    args=[{"colorscale": "Blues", "reversescale": True}],
+                    args=[{"colorscale": [my_blue, ], "reversescale": False}],

                     label="Blue",
                     method="update"
                 ),
                 dict(
                     label="Blue",
                     method="update"
                 ),
                 dict(

-                    args=[{"colorscale": "Greys", "reversescale": True}],
+                    args=[{"colorscale": [my_grey, ], "reversescale": False}],

                     label="Grey",
                     method="update"
                     label="Grey",
                     method="update"

-                ),
-                dict(
-                    args=[{"colorscale": "Greens", "reversescale": True}],
-                    label="Green",
-                    method="update"
-                ),
-                dict(
-                    args=[{"colorscale": "RdBu", "reversescale": False}],
-                    label="RedBlue",
-                    method="update"
-                ),
-                dict(
-                    args=[{"colorscale": "Picnic", "reversescale": False}],
-                    label="Picnic",
-                    method="update"
-                ),
-                dict(
-                    args=[{"colorscale": "Rainbow", "reversescale": False}],
-                    label="Rainbow",
-                    method="update"
-                ),
-                dict(
-                    args=[{"colorscale": "Portland", "reversescale": False}],
-                    label="Portland",
-                    method="update"
-                ),
-                dict(
-                    args=[{"colorscale": "Jet", "reversescale": False}],
-                    label="Jet",
-                    method="update"
-                ),
-                dict(
-                    args=[{"colorscale": "Hot", "reversescale": True}],
-                    label="Hot",
-                    method="update"
-                ),
-                dict(
-                    args=[{"colorscale": "Blackbody", "reversescale": True}],
-                    label="Blackbody",
-                    method="update"
-                ),
-                dict(
-                    args=[{"colorscale": "Earth", "reversescale": True}],
-                    label="Earth",
-                    method="update"
-                ),
-                dict(
-                    args=[{"colorscale": "Electric", "reversescale": True}],
-                    label="Electric",
-                    method="update"
-                ),
-                dict(
-                    args=[{"colorscale": "Viridis", "reversescale": True}],
-                    label="Viridis",
-                    method="update"
-                ),
-                dict(
-                    args=[{"colorscale": "Cividis", "reversescale": True}],
-                    label="Cividis",
-                    method="update"
-                ),
+                )

             ])
         )
     ])
             ])
         )
     ])

diff --git a/resources/tools/presentation/run_report.sh b/resources/tools/presentation/run_report.sh
index 26e4b9c..54eee64 100755 (executable)
--- a/resources/tools/presentation/run_report.sh
+++ b/resources/tools/presentation/run_report.sh
@@ -38,7 +38,7 @@ export PYTHONPATH=`pwd`
 python pal.py \
     --specification specification.yaml \
     --release ${RELEASE} \
 python pal.py \
     --specification specification.yaml \
     --release ${RELEASE} \

-    --week "04" \
+    --week "06" \

     --logging INFO \
     --force
 
     --logging INFO \
     --force
 

diff --git a/resources/tools/presentation/specification.yaml b/resources/tools/presentation/specification.yaml
index bd1ebcb..9f98810 100644 (file)
--- a/resources/tools/presentation/specification.yaml
+++ b/resources/tools/presentation/specification.yaml
@@ -490,11 +490,13 @@
 
     plot-vpp-soak-2n-skx:
       csit-vpp-perf-verify-master-2n-skx:
 
     plot-vpp-soak-2n-skx:
       csit-vpp-perf-verify-master-2n-skx:

-      - 207  # SOAK sel
+      - 248  # SOAK sel
+      - 251  # SOAK sel

 
     plot-vpp-vsc-2n-skx:
       csit-vpp-perf-verify-master-2n-skx:
       - 237
 
     plot-vpp-vsc-2n-skx:
       csit-vpp-perf-verify-master-2n-skx:
       - 237

+      - 243

 
     plot-vpp-csc-2n-skx:
       csit-vpp-perf-verify-master-2n-skx:
 
     plot-vpp-csc-2n-skx:
       csit-vpp-perf-verify-master-2n-skx:


@@ -502,6 +504,10 @@
       - 231
       - 233
 
       - 231
       - 233
 

+    plot-vpp-csp-2n-skx:
+      csit-vpp-perf-verify-master-2n-skx:
+        - 247
+

     plot-vpp-throughput-latency-3n-skx:
       csit-vpp-perf-verify-1810-3n-skx:
       - 24  # NDRPDR full
     plot-vpp-throughput-latency-3n-skx:
       csit-vpp-perf-verify-1810-3n-skx:
       - 24  # NDRPDR full


@@ -1040,7 +1046,7 @@
           size: 16
         zeroline: True
         rangemode: "tozero"
           size: 16
         zeroline: True
         rangemode: "tozero"

-        # range: [0,100]
+        range: [0,100]

       legend:
         orientation: "h"
         font:
       legend:
         orientation: "h"
         font:


@@ -1321,11 +1327,14 @@
     - 24  # NDRPDR sel
     - 27  # NDRPDR sel
     csit-vpp-perf-verify-master-2n-skx:
     - 24  # NDRPDR sel
     - 27  # NDRPDR sel
     csit-vpp-perf-verify-master-2n-skx:

-    - 207  # SOAK sel
+    - 248  # SOAK sel
+    - 251  # SOAK sel

     - 237  # VSC
     - 226  # CSC
     - 231  # CSC
     - 233  # CSC
     - 237  # VSC
     - 226  # CSC
     - 231  # CSC
     - 233  # CSC

+    - 243  # VSC
+    - 247  # CSP

     csit-vpp-perf-check-1804:
     - 5   # mrr - full
     - 6   # mrr - sel
     csit-vpp-perf-check-1804:
     - 5   # mrr - full
     - 6   # mrr - sel


@@ -5298,10 +5307,10 @@
 ###                                P L O T S                                 ###
 ################################################################################
 
 ###                                P L O T S                                 ###
 ################################################################################
 

-# Soak test - example
+# Soak test - 30min Soak Test (PLRsearch), boxes

 -
   type: "plot"
 -
   type: "plot"

-  title: "VPP Throughput Soak: Example"
+  title: "VPP Throughput: 30min Soak Test (PLRsearch) boxes"

   algorithm: "plot_soak_boxes"
   output-file-type: ".html"
   output-file: "{DIR[STATIC,VPP]}/example-soak-boxes"
   algorithm: "plot_soak_boxes"
   output-file-type: ".html"
   output-file: "{DIR[STATIC,VPP]}/example-soak-boxes"


@@ -5317,14 +5326,15 @@
   - "L2BDMACLRN"
   - "IP4FWD"
   - "IP6FWD"
   - "L2BDMACLRN"
   - "IP4FWD"
   - "IP6FWD"

+  - "VHOST"

   layout:
   layout:

-    title: "example-30min-boxes"
+    title: "30min Soak Test (PLRsearch)"

     layout: "plot-soak-throughput"
 
     layout: "plot-soak-throughput"
 

-# Soak test - example
+# Soak test - example, not used

 -
   type: "plot"
 -
   type: "plot"

-  title: "VPP Throughput Soak: Example"
+  title: "VPP Throughput: 30min Soak Test (PLRsearch) bars"

   algorithm: "plot_soak_bars"
   output-file-type: ".html"
   output-file: "{DIR[STATIC,VPP]}/example-soak-bars"
   algorithm: "plot_soak_bars"
   output-file-type: ".html"
   output-file: "{DIR[STATIC,VPP]}/example-soak-bars"


@@ -5340,23 +5350,78 @@
   - "L2BDMACLRN"
   - "IP4FWD"
   - "IP6FWD"
   - "L2BDMACLRN"
   - "IP4FWD"
   - "IP6FWD"

+  - "VHOST"

   layout:
   layout:

-    title: "example-30min-bars"
+    title: "30min Soak Test (PLRsearch)"

     layout: "plot-soak-throughput"
 
     layout: "plot-soak-throughput"
 

-# Heatmap - example
+################################################################################
+# Plots - NF Density
+
+-
+  type: "plot"
+  title: "Packet Throughput: 64B-1c-eth-l2bd-{a}vhost-{b}chain-{c}vm-l3fwdip4-mrr"
+  algorithm: "plot_service_density_heatmap"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/l2bd-2n-skx-x710-64b-2t1c-base-vsc-mrr"
+  data: "plot-vpp-vsc-2n-skx"
+  filter: "'NIC_Intel-X710' and
+           'NF_DENSITY' and
+           'VM' and
+           'CHAIN' and
+           '64B' and
+           '2T1C' and
+           'MRR'"
+  parameters:
+  - "result"
+  - "name"
+  - "tags"
+  include-tests: "MRR" # "PDR" | "NDR" | "MRR"
+  x-axis: "X = VNFs per Service Chain"
+  y-axis: "Y = Number of Service Chains"
+  z-axis: "64B Packet Throughput (mean MRR) [Mpps]"
+  layout:
+    title: "<b>Throughput:</b> eth-l2bd-{2*X*Y}vhost-{Y}chain-{X*Y}vm-l3fwdip4-mrr"
+    layout: "plot-service-density"
+

 -
   type: "plot"
 -
   type: "plot"

-  title: "Service Density: l2sw-2n-skx-x710-64b-2t1c-base-vsc-mrr"
+  title: "Packet Throughput: 64B-2c-eth-l2bd-{a}vhost-{b}chain-{c}vm-l3fwdip4-mrr"

   algorithm: "plot_service_density_heatmap"
   output-file-type: ".html"
   algorithm: "plot_service_density_heatmap"
   output-file-type: ".html"

-  output-file: "{DIR[STATIC,VPP]}/l2sw-2n-skx-x710-64b-2t1c-base-vsc-mrr"
+  output-file: "{DIR[STATIC,VPP]}/l2bd-2n-skx-x710-64b-4t2c-base-vsc-mrr"

   data: "plot-vpp-vsc-2n-skx"
   filter: "'NIC_Intel-X710' and
            'NF_DENSITY' and
            'VM' and
            'CHAIN' and
            '64B' and
   data: "plot-vpp-vsc-2n-skx"
   filter: "'NIC_Intel-X710' and
            'NF_DENSITY' and
            'VM' and
            'CHAIN' and
            '64B' and

+           '4T2C' and
+           'MRR'"
+  parameters:
+  - "result"
+  - "name"
+  - "tags"
+  include-tests: "MRR" # "PDR" | "NDR" | "MRR"
+  x-axis: "X = VNFs per Service Chain"
+  y-axis: "Y = Number of Service Chains"
+  z-axis: "64B Packet Throughput (mean MRR) [Mpps]"
+  layout:
+    title: "<b>Throughput:</b> eth-l2bd-{2*X*Y}vhost-{Y}chain-{X*Y}vm-l3fwdip4-mrr"
+    layout: "plot-service-density"
+
+-
+  type: "plot"
+  title: "Packet Throughput: 64B-1c-eth-l2bd-{a}memif-{b}chain-{c}drc-vppip4-mrr"
+  algorithm: "plot_service_density_heatmap"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/l2bd-2n-skx-x710-64b-2t1c-base-csc-mrr"
+  data: "plot-vpp-csc-2n-skx"
+  filter: "'NIC_Intel-X710' and
+           'NF_DENSITY' and
+           'DOCKER' and
+           'CHAIN' and
+           '64B' and

            '2T1C' and
            'MRR'"
   parameters:
            '2T1C' and
            'MRR'"
   parameters:


@@ -5364,22 +5429,51 @@
   - "name"
   - "tags"
   include-tests: "MRR" # "PDR" | "NDR" | "MRR"
   - "name"
   - "tags"
   include-tests: "MRR" # "PDR" | "NDR" | "MRR"

+  x-axis: "X = CNFs per Service Chain"
+  y-axis: "Y = Number of Service Chains"
+  z-axis: "64B Packet Throughput (mean MRR) [Mpps]"

   layout:
   layout:

-    title: "<b>Network Service Density Matrix: Packet Throughput View</b>"
+    title: "<b>Throughput:</b> eth-l2bd-{2*X*Y}memif-{Y}chain-{X*Y}drc-vppip4-mrr"

     layout: "plot-service-density"
 
 -
   type: "plot"
     layout: "plot-service-density"
 
 -
   type: "plot"

-  title: "Service Density: l2sw-2n-skx-x710-64b-2t1c-base-csc-mrr"
+  title: "Packet Throughput: 64B-2c-eth-l2bd-{a}memif-{b}chain-{c}drc-vppip4-mrr"

   algorithm: "plot_service_density_heatmap"
   output-file-type: ".html"
   algorithm: "plot_service_density_heatmap"
   output-file-type: ".html"

-  output-file: "{DIR[STATIC,VPP]}/l2sw-2n-skx-x710-64b-2t1c-base-csc-mrr"
+  output-file: "{DIR[STATIC,VPP]}/l2bd-2n-skx-x710-64b-4t2c-base-csc-mrr"

   data: "plot-vpp-csc-2n-skx"
   filter: "'NIC_Intel-X710' and
            'NF_DENSITY' and
            'DOCKER' and
            'CHAIN' and
            '64B' and
   data: "plot-vpp-csc-2n-skx"
   filter: "'NIC_Intel-X710' and
            'NF_DENSITY' and
            'DOCKER' and
            'CHAIN' and
            '64B' and

+           '4T2C' and
+           'MRR'"
+  parameters:
+  - "result"
+  - "name"
+  - "tags"
+  include-tests: "MRR" # "PDR" | "NDR" | "MRR"
+  x-axis: "X = CNFs per Service Chain"
+  y-axis: "Y = Number of Service Chains"
+  z-axis: "64B Packet Throughput (mean MRR) [Mpps]"
+  layout:
+    title: "<b>Throughput:</b> eth-l2bd-{2*X*Y}memif-{Y}chain-{X*Y}drc-vppip4-mrr"
+    layout: "plot-service-density"
+
+-
+  type: "plot"
+  title: "Packet Throughput: 64B-1c-eth-l2bd-{a}memif-{b}pipe-{c}drc-vppip4-mrr"
+  algorithm: "plot_service_density_heatmap"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/l2bd-2n-skx-x710-64b-2t1c-base-csp-mrr"
+  data: "plot-vpp-csp-2n-skx"
+  filter: "'NIC_Intel-X710' and
+           'NF_DENSITY' and
+           'DOCKER' and
+           'PIPELINE' and
+           '64B' and

            '2T1C' and
            'MRR'"
   parameters:
            '2T1C' and
            'MRR'"
   parameters:


@@ -5387,8 +5481,37 @@
   - "name"
   - "tags"
   include-tests: "MRR" # "PDR" | "NDR" | "MRR"
   - "name"
   - "tags"
   include-tests: "MRR" # "PDR" | "NDR" | "MRR"

+  x-axis: "X = CNFs per Service Pipeline"
+  y-axis: "Y = Number of Service Pipelines"
+  z-axis: "64B Packet Throughput (mean MRR) [Mpps]"
+  layout:
+    title: "<b>Throughput:</b> eth-l2bd-{2*Y}memif-{Y}pipe-{X*Y}drc-vppip4-mrr"
+    layout: "plot-service-density"
+
+-
+  type: "plot"
+  title: "Packet Throughput: 64B-2c-eth-l2bd-{a}memif-{b}pipe-{c}drc-vppip4-mrr"
+  algorithm: "plot_service_density_heatmap"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/l2bd-2n-skx-x710-64b-4t2c-base-csp-mrr"
+  data: "plot-vpp-csp-2n-skx"
+  filter: "'NIC_Intel-X710' and
+           'NF_DENSITY' and
+           'DOCKER' and
+           'PIPELINE' and
+           '64B' and
+           '4T2C' and
+           'MRR'"
+  parameters:
+  - "result"
+  - "name"
+  - "tags"
+  include-tests: "MRR" # "PDR" | "NDR" | "MRR"
+  x-axis: "X = CNFs per Service Pipeline"
+  y-axis: "Y = Number of Service Pipelines"
+  z-axis: "64B Packet Throughput (mean MRR) [Mpps]"

   layout:
   layout:

-    title: "<b>Network Service Density Matrix: Packet Throughput View</b>"
+    title: "<b>Throughput:</b> eth-l2bd-{2*Y}memif-{Y}pipe-{X*Y}drc-vppip4-mrr"

     layout: "plot-service-density"
 
 ################################################################################
     layout: "plot-service-density"
 
 ################################################################################