-
type: "environment"
+ -
+ type: "configuration"
-
type: "debug"
-
- build-dirs - a list of the directories where the results are stored.
The structure of the section "Environment" is as follows (example):
+
::
-
DIR[WORKING,DATA]: "_tmp/data"
+Section: Configuration
+''''''''''''''''''''''
+
+This section specifies the groups of parameters which are repeatedly used in the
+elements defined later in the specification file. It has the following parts:
+
+ - data sets - Specification of data sets used later in element's specifications
+ to define the input data.
+ - plot layouts - Specification of plot layouts used later in plots'
+ specifications to define the plot layout.
+
+The structure of the section "Configuration" is as follows (example):
+
+::
+
+ -
+ type: "configuration"
+ data-sets:
+ plot-vpp-throughput-latency:
+ csit-vpp-perf-1710-all:
+ - 11
+ - 12
+ - 13
+ - 14
+ - 15
+ - 16
+ - 17
+ - 18
+ - 19
+ - 20
+ vpp-perf-results:
+ csit-vpp-perf-1710-all:
+ - 20
+ - 23
+ plot-layouts:
+ plot-throughput:
+ xaxis:
+ autorange: True
+ autotick: False
+ fixedrange: False
+ gridcolor: "rgb(238, 238, 238)"
+ linecolor: "rgb(238, 238, 238)"
+ linewidth: 1
+ showgrid: True
+ showline: True
+ showticklabels: True
+ tickcolor: "rgb(238, 238, 238)"
+ tickmode: "linear"
+ title: "Indexed Test Cases"
+ zeroline: False
+ yaxis:
+ gridcolor: "rgb(238, 238, 238)'"
+ hoverformat: ".4s"
+ linecolor: "rgb(238, 238, 238)"
+ linewidth: 1
+ range: []
+ showgrid: True
+ showline: True
+ showticklabels: True
+ tickcolor: "rgb(238, 238, 238)"
+ title: "Packets Per Second [pps]"
+ zeroline: False
+ boxmode: "group"
+ boxgroupgap: 0.5
+ autosize: False
+ margin:
+ t: 50
+ b: 20
+ l: 50
+ r: 20
+ showlegend: True
+ legend:
+ orientation: "h"
+ width: 700
+ height: 1000
+
+The definitions from this sections are used in the elements, e.g.:
+
+::
+
+ -
+ type: "plot"
+ title: "VPP Performance 64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc"
+ algorithm: "plot_performance_box"
+ output-file-type: ".html"
+ output-file: "{DIR[STATIC,VPP]}/64B-1t1c-l2-sel1-ndrdisc"
+ data:
+ "plot-vpp-throughput-latency"
+ filter: "'64B' and ('BASE' or 'SCALE') and 'NDRDISC' and '1T1C' and ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and not 'VHOST'"
+ parameters:
+ - "throughput"
+ - "parent"
+ traces:
+ hoverinfo: "x+y"
+ boxpoints: "outliers"
+ whiskerwidth: 0
+ layout:
+ title: "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc"
+ layout:
+ "plot-throughput"
+
+
Section: Debug mode
'''''''''''''''''''
- methods to compute statistical data from the filtered input data.
- trending.
- - etc.
+
+Throughput Speedup Analysis - Multi-Core with Multi-Threading
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+Throughput Speedup Analysis (TSA) calculates throughput speedup ratios
+for tested 1-, 2- and 4-core multi-threaded VPP configurations using the
+following formula:
+
+::
+
+ N_core_throughput
+ N_core_throughput_speedup = -----------------
+ 1_core_throughput
+
+Multi-core throughput speedup ratios are plotted in grouped bar graphs
+for throughput tests with 64B/78B frame size, with number of cores on
+X-axis and speedup ratio on Y-axis.
+
+For better comparison multiple test results' data sets are plotted per
+each graph:
+
+ - graph type: grouped bars;
+ - graph X-axis: (testcase index, number of cores);
+ - graph Y-axis: speedup factor.
+
+Subset of existing performance tests is covered by TSA graphs.
+
+**Model for TSA:**
+
+::
+
+ -
+ type: "plot"
+ title: "TSA: 64B-*-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc"
+ algorithm: "plot_throughput_speedup_analysis"
+ output-file-type: ".html"
+ output-file: "{DIR[STATIC,VPP]}/10ge2p1x520-64B-l2-tsa-ndrdisc"
+ data:
+ "plot-throughput-speedup-analysis"
+ filter: "'NIC_Intel-X520-DA2' and '64B' and 'BASE' and 'NDRDISC' and ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and not 'VHOST'"
+ parameters:
+ - "throughput"
+ - "parent"
+ - "tags"
+ layout:
+ title: "64B-*-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc"
+ layout:
+ "plot-throughput-speedup-analysis"
Advanced data analytics
```````````````````````
-As the next steps, advanced data analytics (ADA) will be implemented using
-machine learning (ML) and artificial intelligence (AI).
+In the future advanced data analytics (ADA) will be added to analyze the
+telemetry data collected from SUT telemetry sources and correlate it to
+performance test results.
-TODO:
+:TODO:
- - describe the concept of ADA.
- - add specification.
+ - describe the concept of ADA.
+ - add specification.
Data presentation