csit.git
3 years agoTrending: Fixes after review 26/28426/1
Tibor Frank [Mon, 17 Aug 2020 10:16:57 +0000 (12:16 +0200)]
Trending: Fixes after review

Change-Id: Ia2fdd4d222c723c063f267a20e1a4f18b4e03d69
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoTrending: NFV Tests 41/28341/11
Tibor Frank [Thu, 13 Aug 2020 06:56:46 +0000 (08:56 +0200)]
Trending: NFV Tests

- 2n-skx
- 2n-clx

Change-Id: I6dd55ed167dc4267b2a4e4c52ec0874ad0e4a4b6
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoTrending: NFV Tests 75/28275/2
Tibor Frank [Wed, 12 Aug 2020 13:15:29 +0000 (15:15 +0200)]
Trending: NFV Tests

Change-Id: I882c0af852a9e556df3c748164360cdbc336d654
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoUpdate of VPP_STABLE_VER files 98/28198/1
Jan Gelety [Wed, 5 Aug 2020 17:53:48 +0000 (19:53 +0200)]
Update of VPP_STABLE_VER files

 - use new vpp ref build - ubuntu 18.04:
   20.09-rc0~277-g6042d2815

 - use new vpp ref build - centos7:
   20.09-rc0~277_g6042d28~b9760

Change-Id: I1da941f54a6338846a3478923dbb5988ffbf842a
Signed-off-by: Jan Gelety <jgelety@cisco.com>
3 years agoLimit user triggers: no OR 83/28183/1
Vratko Polak [Thu, 6 Aug 2020 08:32:46 +0000 (10:32 +0200)]
Limit user triggers: no OR

To avoid runs such as:
https://jenkins.fd.io/job/vpp-csit-verify-perf-master-3n-hsw/340/

Change-Id: I1b30d5f440ddf8ff32b11265b2ac2176f4b9a665
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agojob-specs: align mrr-weekly NFV tests with report-iter 66/28166/2
Tibor Frank [Tue, 4 Aug 2020 09:10:41 +0000 (11:10 +0200)]
job-specs: align mrr-weekly NFV tests with report-iter

- 2n-skx
- 2n-clx

Change-Id: I8e96d335be2c26ffb03e210f6f658fba1825c597
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoAdd Denverton testbeds BIOS configuration doc 08/28108/2
Yulong Pei [Wed, 29 Jul 2020 20:41:32 +0000 (13:41 -0700)]
Add Denverton testbeds BIOS configuration doc

Signed-off-by: Xinfeng Zhao <xinfengx.zhao@intel.com>
Signed-off-by: Yulong Pei <yulong.pei@intel.com>
Change-Id: I531b380277a885f45af0749d048f44ba12285dd2

3 years agoReport 2005: Add data, configure rls2005.32 72/28172/1
Tibor Frank [Wed, 5 Aug 2020 04:52:06 +0000 (06:52 +0200)]
Report 2005: Add data, configure rls2005.32

Change-Id: I79129cac48c60a959e4dc1ebfc9d63b9b230e933
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoTrending: Set longer time period 70/28170/1
Tibor Frank [Tue, 4 Aug 2020 11:17:00 +0000 (13:17 +0200)]
Trending: Set longer time period

- daily: up to 180 sets, max 180 days
- weekly: up to 26 sets, max 180 days

Change-Id: I59c550e5728fcb83f247b0c958f21209d927ee98
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoTrending: Implement the latest changes in job specs 02/28102/11
Tibor Frank [Wed, 29 Jul 2020 13:35:41 +0000 (15:35 +0200)]
Trending: Implement the latest changes in job specs

Graphs:
 - MRR
   - 2n-clx
   - 2n-skx
   - 3n-skx
   - 3n-hsw
   - 3n-tsh
 - NDRPDR
   - 2n-skx
   - 3n-skx
   - 3n-hsw
   - 2n-clx

Change-Id: Ieab26ba1d2ee56e66c5c8598c4dba240a14e53e9
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoUpdate of VPP_STABLE_VER files 40/28140/1
Jan Gelety [Thu, 23 Jul 2020 21:00:29 +0000 (23:00 +0200)]
Update of VPP_STABLE_VER files

 - use new vpp ref build - ubuntu 18.04:
   20.09-rc0~266-ge6c3e8f0e

 - use new vpp ref build - centos7:
   20.09-rc0~266_ge6c3e8f~b9748

Change-Id: I3d173d9b67657b238f7a65e24d81f0ac8f9d0093
Signed-off-by: Jan Gelety <jgelety@cisco.com>
3 years agoT-Rex: Add advanced stateful mode 68/27168/46
Jan Gelety [Thu, 7 May 2020 02:37:19 +0000 (04:37 +0200)]
T-Rex: Add advanced stateful mode

- provide base routines to run T-Rex in advanced stateful mode

Change-Id: Ib0dc5f2919c370753335f6446860683dc4b12d93
Signed-off-by: Jan Gelety <jgelety@cisco.com>
3 years agoFramework: Perf stat capturing 16/28116/17
Peter Mikus [Thu, 5 Mar 2020 11:12:18 +0000 (11:12 +0000)]
Framework: Perf stat capturing

Change-Id: I3bbe1fe0073ddeead5219993675f24955e8c3dfd
Signed-off-by: Peter Mikus <pmikus@cisco.com>
3 years agoIntroduce per DUT configurable stats 59/28059/12
pmikus [Fri, 24 Jul 2020 12:57:43 +0000 (12:57 +0000)]
Introduce per DUT configurable stats

- test/suite/global
- binary logic is not working

Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: Ia3d81cbf2c5f04d1093a0a408c84a9ffc6f3eef0

3 years agoTRex stateless: Fix unidirectional tests 11/28111/2
Vratko Polak [Thu, 30 Jul 2020 11:25:33 +0000 (13:25 +0200)]
TRex stateless: Fix unidirectional tests

Change-Id: I5e101f012ae4a4383e61b274499817d146479993
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoUpdate RCA files with new findings 53/28053/5
Vratko Polak [Wed, 29 Jul 2020 12:33:10 +0000 (14:33 +0200)]
Update RCA files with new findings

Change-Id: I0db4d513cdc84c6f8c1e2dffa46cd41e13df5b21
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoAnsible: Nomad 0.12.0 upgrade 00/28100/1
pmikus [Wed, 29 Jul 2020 09:09:00 +0000 (09:09 +0000)]
Ansible: Nomad 0.12.0 upgrade

+ random fixes

Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: Icb320d1ba4bbe3b5f99956cf8e89c2a29e114e41

3 years agoSoak: Avoid a possible deadlock. 81/28081/3
Vratko Polak [Tue, 28 Jul 2020 14:38:55 +0000 (16:38 +0200)]
Soak: Avoid a possible deadlock.

Change-Id: I31c2d7744b5cd3021132fb188480b8edec74986c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agovsap: install ab using ansible 30/28030/2
Xiaolong Jiang [Wed, 22 Jul 2020 09:28:27 +0000 (17:28 +0800)]
vsap: install ab using ansible

Signed-off-by: Xiaolong Jiang <xiaolongx.jiang@intel.com>
Change-Id: I275ad9dccc02e6b1f1c791fabf8896fb03acb3fb

3 years agoDocs testing_in_vagrant: add header sections and make command line codes more readable 58/28058/3
Martin Balaz [Fri, 24 Jul 2020 10:08:07 +0000 (12:08 +0200)]
Docs testing_in_vagrant: add header sections and make command line codes more readable

Signed-off-by: Martin Balaz <martin.balaz@pantheon.tech>
Change-Id: I0b8fdaad1e68e41e4a436d9f2fc566d8896adbd6
Signed-off-by: Martin Balaz <martin.balaz@pantheon.tech>
3 years agoDoc: Update durations of test runs 56/27756/11
Tibor Frank [Wed, 1 Jul 2020 12:31:42 +0000 (14:31 +0200)]
Doc: Update durations of test runs

Change-Id: I0652f4483a83954096649fbd92af1fecc4bf0a7d
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoUpdate of VPP_STABLE_VER files 69/28069/1
Jan Gelety [Tue, 21 Jul 2020 21:44:26 +0000 (23:44 +0200)]
Update of VPP_STABLE_VER files

 - use new vpp ref build - ubuntu 18.04:
   20.09-rc0~243-gc72995dd7

 - use new vpp ref build - centos7:
   20.09-rc0~243_gc72995d~b9727

Change-Id: Ib37d4a51bee9782077c537531d67e91ac52a358d
Signed-off-by: Jan Gelety <jgelety@cisco.com>
3 years agoNDRPDR: Allow smaller min_rate, 9001 pps 37/28037/5
Vratko Polak [Fri, 24 Jul 2020 11:55:53 +0000 (13:55 +0200)]
NDRPDR: Allow smaller min_rate, 9001 pps

+ Latency measurements need more than 9000 pps.
- Previously 0% measurement used 9500 pps.

Change-Id: Ic0841de096dfa8a61329f98aa1ba6d3f0ce60c66
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agojob-specs: Fix tag l2bdsbasemaclrn --> l2bdbasemaclrn 18/28018/3
Tibor Frank [Tue, 21 Jul 2020 11:57:31 +0000 (13:57 +0200)]
job-specs: Fix tag l2bdsbasemaclrn --> l2bdbasemaclrn

Change-Id: Icfecc2b9576d1be32c78f6b0019b64d6f401745e
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data 55/28055/1
Tibor Frank [Fri, 24 Jul 2020 05:14:07 +0000 (07:14 +0200)]
Report 2005: Add data

Change-Id: Id91ac52634fbf4c6b931a61946eb7a9d8019dd76
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoansible nomad: install htop 40/28040/2
Dave Wallace [Wed, 22 Jul 2020 17:22:13 +0000 (17:22 +0000)]
ansible nomad: install htop

Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: I0549e9b5ac6769e3fc760ab2047d89a188baa36a

3 years agoFIX: Log required cli_cmd not "cli_inband" 20/28020/3
Jan Gelety [Mon, 20 Jul 2020 22:13:53 +0000 (00:13 +0200)]
FIX: Log required cli_cmd not "cli_inband"

Change-Id: If0975b1d54882390c5be418927e2961d0f4c8429
Signed-off-by: Jan Gelety <jgelety@cisco.com>
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoperpatch: Echo MAKE_PARALLEL_* var before build 21/27821/5
Juraj Linkeš [Mon, 1 Jun 2020 06:22:42 +0000 (08:22 +0200)]
perpatch: Echo MAKE_PARALLEL_* var before build

VPP uses MAKE_PARALLEL_FLAGS or MAKE_PARALLEL_JOBS to limit the number
of cpus to use during build, so emit a line on stdout if it's used.

Change-Id: I669398d474d172abb6c848a45f24f1bdd56990d8
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
3 years agoReport 2005: Configure rls2005.31, add data 45/28045/1
Tibor Frank [Thu, 23 Jul 2020 05:54:18 +0000 (07:54 +0200)]
Report 2005: Configure rls2005.31, add data

Change-Id: I87e07e334783e10dc863194f240e5dbcbf141915
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data 24/28024/1
Tibor Frank [Wed, 22 Jul 2020 05:43:36 +0000 (07:43 +0200)]
Report 2005: Add data

Change-Id: Ic6574e2bb24ce4004ed8c917d9deeff9a45b37ab
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoUpdate of VPP_STABLE_VER files 16/28016/1
Jan Gelety [Mon, 20 Jul 2020 18:23:18 +0000 (20:23 +0200)]
Update of VPP_STABLE_VER files

 - use new vpp ref build - ubuntu 18.04:
   20.09-rc0~223-g459d17bb7

 - use new vpp ref build - centos7:
   20.09-rc0~223_g459d17b~b9700

Change-Id: I369f708dc4423f6897562e2597d6296a473f77b1
Signed-off-by: Jan Gelety <jgelety@cisco.com>
3 years agoReport 2005: Add data 78/27978/1
Tibor Frank [Tue, 21 Jul 2020 07:54:24 +0000 (09:54 +0200)]
Report 2005: Add data

Change-Id: I677f2aa67bcd51a31509aefacbe9c428578beec5
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data 75/27975/1
Tibor Frank [Mon, 20 Jul 2020 17:58:24 +0000 (19:58 +0200)]
Report 2005: Add data

Change-Id: I312d707aeb5feccacb393009e6cd02959ced4145
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data 71/27971/1
Tibor Frank [Sun, 19 Jul 2020 19:23:08 +0000 (21:23 +0200)]
Report 2005: Add data

Change-Id: I4c674a7821a83b523a9406515849d10341a0d2ec
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agojob-specs: Align 2n-skx nfv tests with 2n-clx 15/27915/3
Tibor Frank [Wed, 15 Jul 2020 13:33:35 +0000 (15:33 +0200)]
job-specs: Align 2n-skx nfv tests with 2n-clx

Change-Id: I6c828ee146bab8ebdb24be6b03369f6e772d25c8
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data 56/27956/1
Tibor Frank [Fri, 17 Jul 2020 06:59:45 +0000 (08:59 +0200)]
Report 2005: Add data

Change-Id: I1d1b802481449c29056e541752228c47a6e3ea72
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoRemove remains of WRK tests 75/27875/3
Tibor Frank [Mon, 13 Jul 2020 12:08:13 +0000 (14:08 +0200)]
Remove remains of WRK tests

Change-Id: If61783fb717757c6189f06924412bd079e15a08f
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoFIX: CSIT annoyances II 49/27949/2
pmikus [Thu, 16 Jul 2020 14:21:35 +0000 (14:21 +0000)]
FIX: CSIT annoyances II

- useless code

Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I65dd0f4e1744e4ff84661cd38a67d4490c9733e7

3 years agoFIX: CSIT annoyances I. 48/27948/2
pmikus [Thu, 16 Jul 2020 14:18:20 +0000 (14:18 +0000)]
FIX: CSIT annoyances I.

- As time flies

Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: If4cc9575b3fa828ad598b524c8cbf943c623b485

3 years agoAdd ADL plugin to cop tests 47/27947/2
Vratko Polak [Thu, 16 Jul 2020 12:54:37 +0000 (14:54 +0200)]
Add ADL plugin to cop tests

The following VPP change renames and moves
COP functionality into a brand new plugin:
https://gerrit.fd.io/r/c/vpp/+/27916

In a subsequent change we should rename tests
and start using the new API messages.
This change is just a minimal edit to avoid test failures.

Change-Id: I1c4b81060dd533ef447f3cc44443b71c5e524d59
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoReport 2005: Add data 44/27944/1
Tibor Frank [Thu, 16 Jul 2020 11:47:35 +0000 (13:47 +0200)]
Report 2005: Add data

Change-Id: I24215fa78d3ee9affb301d22c655b925dbb5bc14
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Configure 2005.30, add data 14/27914/2
Tibor Frank [Wed, 15 Jul 2020 12:04:11 +0000 (14:04 +0200)]
Report 2005: Configure 2005.30, add data

Change-Id: I33908f6967e89e894560fd5f63c57a76c110e9ab
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add pdf version 11/27911/1
Tibor Frank [Wed, 15 Jul 2020 08:31:13 +0000 (10:31 +0200)]
Report 2005: Add pdf version

Change-Id: I020559c4782b98700dfe5606a80695a1cfecbdcf
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoreport: 2n-clx, 2n-skx, 3n-skx RCA edits in comparison tables 79/27879/3
Maciek Konstantynowicz [Mon, 13 Jul 2020 16:14:16 +0000 (18:14 +0200)]
report: 2n-clx, 2n-skx, 3n-skx RCA edits in comparison tables

Change-Id: I9810b0b33355d880b606d7402564a3150d12397c
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
3 years agoreport: Updated vpp performance tests release notes 06/27906/2
Maciek Konstantynowicz [Tue, 14 Jul 2020 10:09:14 +0000 (12:09 +0200)]
report: Updated vpp performance tests release notes

Change-Id: I1028f4fc9d0f38d70ba8b838a66f74274e534a40
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
3 years agoReport 2005: Fix legend in comparison tables 05/27905/2
Tibor Frank [Tue, 14 Jul 2020 09:31:09 +0000 (11:31 +0200)]
Report 2005: Fix legend in comparison tables

Change-Id: I79265bb14c6daf75944ed596c21b9929f8524333
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add soak tests 81/27881/1
Tibor Frank [Tue, 14 Jul 2020 06:02:38 +0000 (08:02 +0200)]
Report 2005: Add soak tests

Change-Id: Ib56215e906afba34d52b390ebc5a2d58bab4bf71
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add NFV tests 73/27873/6
Tibor Frank [Mon, 13 Jul 2020 11:42:13 +0000 (13:42 +0200)]
Report 2005: Add NFV tests

- 2n-skx - ndrpdr, mrr
- 2n-clx - ndrpdr, mrr

Change-Id: I75f78a3986b17bb03187a4cc91260c4a60a7d515
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport: remove wrk tests 80/27880/1
Tibor Frank [Mon, 13 Jul 2020 19:15:50 +0000 (21:15 +0200)]
Report: remove wrk tests

Change-Id: Ie3f6ad2e4b5542e915f310c7b9ebeddf88b6fd8a
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoreport: updated environment versioning section 74/27874/4
Maciek Konstantynowicz [Mon, 13 Jul 2020 12:02:24 +0000 (14:02 +0200)]
report: updated environment versioning section

Change-Id: I95adc8730d379286511398754042ad88b8c7d681
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
3 years agoReport: Improve vnf_service_chains_reconf.rst 41/27541/7
Vratko Polak [Thu, 25 Jun 2020 12:50:16 +0000 (14:50 +0200)]
Report: Improve vnf_service_chains_reconf.rst

+ Use the quantity name from methodology (not from test implementation).

Change-Id: Ic46f0447af840b392fe5dbbfec2bd11a847b06ca
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoReport: vpp_performance_tests/overview.rst 38/27538/6
Vratko Polak [Thu, 25 Jun 2020 12:26:45 +0000 (14:26 +0200)]
Report: vpp_performance_tests/overview.rst

Change-Id: I1c5c1a26574861e22485e7b40e19304bb635bc4a
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoDocs: Update TRex information 76/27876/2
Vratko Polak [Mon, 13 Jul 2020 12:32:08 +0000 (14:32 +0200)]
Docs: Update TRex information

The previous was describing CSIT version 3.

+ Stateless profile is now under GPL directory.

Change-Id: I9810ca224159053386557493eaf90b4aa682710e
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoRCA 2005: Update status in RCA files 57/27857/4
Vratko Polak [Fri, 10 Jul 2020 19:06:04 +0000 (21:06 +0200)]
RCA 2005: Update status in RCA files

CSIT-1738, CSIT-1740 and CSIT-1741 are now DONE.

Change-Id: Id9b93ace9464a7e6af64f7860ad1a15c7a287a53
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoRCA: Improve footnotes for ACL VPP. 54/27854/3
Vratko Polak [Fri, 10 Jul 2020 11:58:40 +0000 (13:58 +0200)]
RCA: Improve footnotes for ACL VPP.

+ Stateful ends with single l.

Change-Id: I7be89b3f536d05ed9b21ba8849a4eef33259d312
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoSoak: Do not allow negative loss 97/27297/5
Vratko Polak [Mon, 13 Jul 2020 10:17:44 +0000 (12:17 +0200)]
Soak: Do not allow negative loss

Change-Id: I6da359d25edc415e44263d3f85f166369e564987
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoReport 2005: Add reconf tests 70/27870/2
Tibor Frank [Mon, 13 Jul 2020 10:48:02 +0000 (12:48 +0200)]
Report 2005: Add reconf tests

Change-Id: Ic9466d4c0e6fb757e7d7c01d1aa83a59c1389371
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoSoak: Fix worker exception passing 96/27296/4
Vratko Polak [Wed, 27 May 2020 18:45:26 +0000 (20:45 +0200)]
Soak: Fix worker exception passing

Change-Id: I636f020e97df1b37ac8b6a30af511eebe611b56f
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoSoak: Fix failure condition 67/27267/4
Vratko Polak [Tue, 26 May 2020 11:57:08 +0000 (13:57 +0200)]
Soak: Fix failure condition

Previously, number of directions was not taken into account.
Also, ideally PLRsearch never reports value under the hard minimum,
so successful results are now required to be more than 10% better.

Change-Id: I8622726b97bd1da3e139c8044a2932837fc268b7
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoUpdate of VPP_STABLE_VER files 66/27866/2
Jan Gelety [Thu, 9 Jul 2020 09:28:37 +0000 (11:28 +0200)]
Update of VPP_STABLE_VER files

 - use new vpp ref build - ubuntu 18.04:
   20.09-rc0~194-g52e9aaf0b

 - use new vpp ref build - centos7:
   20.09-rc0~193_gea7e708~b9670
   (build 20.09-rc0~197 not uploaded to packagecloud.io)

Change-Id: I3a6e284daf67e826eda184aa7a9af1dd3b7a6043
Signed-off-by: Jan Gelety <jgelety@cisco.com>
3 years agoReport 2005: Fix titles in test env 69/27869/1
Tibor Frank [Mon, 13 Jul 2020 08:47:54 +0000 (10:47 +0200)]
Report 2005: Fix titles in test env

Change-Id: I55e7a596d5fa23531e68d7ace9364411c3c372fc
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data 67/27867/1
Tibor Frank [Mon, 13 Jul 2020 08:19:28 +0000 (10:19 +0200)]
Report 2005: Add data

- perf-iter-3n-tsh - 1 run
- perf-cov-3n-hsw - 35 runs

Change-Id: I9951cd2aa1883efa944b5c16b3b736761c1ff64f
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Configure rls2005.39 65/27865/1
Tibor Frank [Mon, 13 Jul 2020 07:17:03 +0000 (09:17 +0200)]
Report 2005: Configure rls2005.39

Change-Id: I7ea2e004b79d47420f6a1e0beb88731529b03706
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoreport: fixed misplaced environment versioning section 62/27862/3
Maciek Konstantynowicz [Fri, 10 Jul 2020 16:49:27 +0000 (18:49 +0200)]
report: fixed misplaced environment versioning section

Change-Id: Id0da363b376e12dac583f95624ed6525b24290eb
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
3 years agoDocs: job-specs: dpdk and nfv-density test sets 50/27850/2
Tibor Frank [Fri, 10 Jul 2020 07:24:02 +0000 (09:24 +0200)]
Docs: job-specs: dpdk and nfv-density test sets

+ synchronize iter dpdk test sets with coverage dpdk test sets
+ add iter nfv-density-mrr test sets

Change-Id: I759025b6acf860aca9828f9fc1d82ab3a37ec98f
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoRCA 2005: Update RCA files. 76/27676/9
Vratko Polak [Fri, 10 Jul 2020 10:10:01 +0000 (12:10 +0200)]
RCA 2005: Update RCA files.

+ Current state.
+ Covers skx (also 3n) and clx.
+ Specific Jira sub-tasks, even for VPP progressions.
- Only 2n-skx data for NDR.

Change-Id: I56cf870aedc1319891b7188268ff6cf7bc744ff8
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoreport: added versioning sub-section to VPP Test Environment section 46/27846/3
Maciek Konstantynowicz [Thu, 9 Jul 2020 19:56:19 +0000 (21:56 +0200)]
report: added versioning sub-section to VPP Test Environment section

Change-Id: I59a93c0594ab5e5f0e267cacdf03d1f345e02b29
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
3 years agoReport 2005: Add data 49/27849/2
Tibor Frank [Fri, 10 Jul 2020 06:37:34 +0000 (08:37 +0200)]
Report 2005: Add data

Change-Id: I56b805eec6e5d606a954ba749b16ce72ce6047fa
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoFramework: Allow change of MRR settings 01/27601/5
pmikus [Fri, 19 Jun 2020 12:18:58 +0000 (12:18 +0000)]
Framework: Allow change of MRR settings

Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: Ie88f0df239725a4de62d727e1923cdb3ad040809

3 years agoReport 2005: Add data, review graphs 20/27820/2
Tibor Frank [Thu, 9 Jul 2020 09:16:19 +0000 (11:16 +0200)]
Report 2005: Add data, review graphs

Change-Id: I2913ed579369a26a1226cfbe958432d1b3e11789
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data, hide dnv, soak, reconf, nfv 14/27814/2
Tibor Frank [Wed, 8 Jul 2020 07:59:24 +0000 (09:59 +0200)]
Report 2005: Add data, hide dnv, soak, reconf, nfv

Change-Id: Iff1b1db3e31d2e1f19f8309ba28f1f88e0538f6c
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoPAL: Improve comparison tables 05/27805/15
Tibor Frank [Tue, 7 Jul 2020 07:35:59 +0000 (09:35 +0200)]
PAL: Improve comparison tables

- place RCAs to the columns with diffs

CSIT-1737

Change-Id: I3ccf5ee510796684e3e09dbdc6264b05f391f633
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport: 2005: Add data, configure rls2005.28 04/27804/1
Tibor Frank [Tue, 7 Jul 2020 07:26:02 +0000 (09:26 +0200)]
Report: 2005: Add data, configure rls2005.28

Change-Id: I686102fd7d17ad287ae890f1170ef7c78ff74a0d
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoUpdate TODOs for tox 01/27201/2
Vratko Polak [Thu, 21 May 2020 17:00:51 +0000 (19:00 +0200)]
Update TODOs for tox

Change-Id: I9368557aed7508d9401628c0ddfda475da42bc99
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoReport 2005: Add data 93/27793/1
Tibor Frank [Sun, 5 Jul 2020 15:23:30 +0000 (17:23 +0200)]
Report 2005: Add data

Change-Id: Icc4ea692adcafef542dfa61e0fa594c85db6898e
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data 91/27791/1
Tibor Frank [Fri, 3 Jul 2020 10:00:41 +0000 (12:00 +0200)]
Report 2005: Add data

Change-Id: I8c0be475c73aa5c9c3d04780851faf25f1156aed
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data, change legend in comp tables 89/27789/1
Tibor Frank [Fri, 3 Jul 2020 07:41:57 +0000 (09:41 +0200)]
Report 2005: Add data, change legend in comp tables

Change-Id: I538751d928a547b9c168798f853903dd56887f47
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoTrending and report alignment 55/27755/4
Tibor Frank [Wed, 1 Jul 2020 11:58:40 +0000 (13:58 +0200)]
Trending and report alignment

- use test sets defined in report-iterative also for mrr
  tests and for mrr_daily and mlr_weekly

Change-Id: I47fba2997cddaca934bdebd5fca6936e25332afc
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoInfra: add jlinkes ssh key to ARM nomad servers 57/27757/2
Juraj Linkeš [Wed, 1 Jul 2020 13:24:16 +0000 (15:24 +0200)]
Infra: add jlinkes ssh key to ARM nomad servers

Change-Id: Ie304118e6b1ecb06704e35bbbeccf7cc0f649e8b
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
3 years agoReport 2005: Fix data src for VPP Perf Changes 2n-clx-xxv710 59/27759/1
Tibor Frank [Thu, 2 Jul 2020 08:40:01 +0000 (10:40 +0200)]
Report 2005: Fix data src for VPP Perf Changes 2n-clx-xxv710

Change-Id: I8e8a7e88475a789f04767893866db2502ec5f8ed
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data 58/27758/1
Tibor Frank [Thu, 2 Jul 2020 08:27:56 +0000 (10:27 +0200)]
Report 2005: Add data

Change-Id: I65e1a386a29faa9c890a526b8d264f4eabcbf382
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agodocs: added mrr tests for vpp and dpdk to job_specs/report_iterative 42/27742/2
Maciek Konstantynowicz [Tue, 30 Jun 2020 15:57:39 +0000 (16:57 +0100)]
docs: added mrr tests for vpp and dpdk to job_specs/report_iterative

Change-Id: Ie8fca17baa114fcdf5916e65311716a86e791709
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
3 years agoInfra: Update ansible ARM dev server 54/27754/2
Juraj Linkeš [Wed, 1 Jul 2020 11:21:07 +0000 (13:21 +0200)]
Infra: Update ansible ARM dev server

Remove nomad installation from the ARM dev server and add a generic
vppdev user to it. Add ansible role dev for this.

Change-Id: Icb967ff18cbe939464ddb3d8dbce5101747e739f
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
3 years agoInfra: Update Ansible ARM Nomad 49/27749/1
Juraj Linkeš [Wed, 1 Jul 2020 08:34:41 +0000 (10:34 +0200)]
Infra: Update Ansible ARM Nomad

- Add three new ThunderX servers
- Mark two existing ThunderX servers as out of order, since
  they're inaccessible
- Fix microarchitecture of ARM build servers

Change-Id: I57f5f3f8436a9082a264eb4e0fa4faac77ad53be
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
3 years agoReport 2005: Add data 48/27748/1
Tibor Frank [Wed, 1 Jul 2020 06:49:40 +0000 (08:49 +0200)]
Report 2005: Add data

+ 2001.0-4 vs 2001.0-2 for 3n-hsw

Change-Id: Ice05011ea79681ede74825ed8ae6be849ea06b8f
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agomake graph node variant can be configured from gerrit trigger 96/27396/7
Yulong Pei [Tue, 2 Jun 2020 16:32:40 +0000 (00:32 +0800)]
make graph node variant can be configured from gerrit trigger

for example, below will trigger VPP with default code path test in CSIT,
"csit-2n-clx-perftest mrrAND64bANDnic_intel-xxv710ANDeth-l2bdbasemaclrnNOTdrv_avf"
and below will trigger VPP with avx512 code path test in CSIT,
"csit-2n-clx-perftest icl mrrAND64bANDnic_intel-xxv710ANDeth-l2bdbasemaclrnNOTdrv_avf"

Change-Id: Id3abf932203ce00f34f8643c5bc4d5fc4d3e6efb
Signed-off-by: Yulong Pei <yulong.pei@intel.com>
3 years agoReport 2005: Add 3n-skx mrr env 4 data 31/27731/2
Tibor Frank [Tue, 30 Jun 2020 08:33:10 +0000 (10:33 +0200)]
Report 2005: Add 3n-skx mrr env 4 data

Change-Id: I44f05050c8e0c528c4980ac767232c497d058d9f
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport: Improve containers.rst 64/27564/9
Vratko Polak [Thu, 25 Jun 2020 13:06:00 +0000 (15:06 +0200)]
Report: Improve containers.rst

Change-Id: I2280b3f8c1977b47b222fa5ce3d47f90dc9b7e87
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
3 years agoReport 2005: Add 2n-clx and 3n-hsw mrr env 4 data 30/27730/1
Tibor Frank [Tue, 30 Jun 2020 05:13:33 +0000 (07:13 +0200)]
Report 2005: Add 2n-clx and 3n-hsw mrr env 4 data

Change-Id: I431215d96699381385199b61e9e5f37a581d91f4
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Replace data in thput graphs by env 4 22/27722/1
Tibor Frank [Mon, 29 Jun 2020 11:29:45 +0000 (13:29 +0200)]
Report 2005: Replace data in thput graphs by env 4

Change-Id: I6bd871d85c7ca2ed9748a96b0568f8f3dab82008
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Configure rls2005.27, add data 15/27715/1
Tibor Frank [Mon, 29 Jun 2020 05:58:35 +0000 (07:58 +0200)]
Report 2005: Configure rls2005.27, add data

Change-Id: I6a9911b500d40708e2570be9d83cdd860c677977
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Fix comparison table hdr, add data 06/27706/3
Tibor Frank [Fri, 26 Jun 2020 10:14:27 +0000 (12:14 +0200)]
Report 2005: Fix comparison table hdr, add data

Change-Id: I3ef6dfb1b3da4f91c77841536a840a439816e61f
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data, fix sorting in comp tables 04/27704/1
Tibor Frank [Fri, 26 Jun 2020 06:52:46 +0000 (08:52 +0200)]
Report 2005: Add data, fix sorting in comp tables

Change-Id: Ib0c42f5b9fbd9b9f9ebf0bff437ef9d145e6ddda
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data 03/27703/1
Tibor Frank [Fri, 26 Jun 2020 04:40:24 +0000 (06:40 +0200)]
Report 2005: Add data

Change-Id: I7ffbe353739c3077bbfc03f25f4b6e03b2fdcccd
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReporet 2005: Add data, Fix comparison columns 80/27680/1
Tibor Frank [Thu, 25 Jun 2020 18:36:25 +0000 (20:36 +0200)]
Reporet 2005: Add data, Fix comparison columns

Change-Id: I83bd99c7afd63bd9ef0e432562c76cbe4ac8cb9d
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data 72/27672/2
Tibor Frank [Thu, 25 Jun 2020 09:40:19 +0000 (11:40 +0200)]
Report 2005: Add data

Change-Id: I35803563351aa5419726a8a469f23557b49189f2
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add 3n-skx comp tables 71/27671/1
Tibor Frank [Thu, 25 Jun 2020 08:09:23 +0000 (10:09 +0200)]
Report 2005: Add 3n-skx comp tables

Change-Id: Ic3237d165fdc9b7ca95498f724e0173f1cd29dad
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoReport 2005: Add data 70/27670/1
Tibor Frank [Thu, 25 Jun 2020 07:39:32 +0000 (09:39 +0200)]
Report 2005: Add data

Change-Id: I11c238327de71e75ca1ad099455f8daeb1ce996b
Signed-off-by: Tibor Frank <tifrank@cisco.com>
3 years agoreport: updated rls notes for vpp performance section 85/27585/5
Maciek Konstantynowicz [Wed, 17 Jun 2020 18:25:01 +0000 (19:25 +0100)]
report: updated rls notes for vpp performance section

Change-Id: I114c7d64c24210df4273086757ef33735baffe94
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
3 years agoReport: Improve soak_tests/index.rst 40/27540/5
Vratko Polak [Mon, 15 Jun 2020 15:39:00 +0000 (17:39 +0200)]
Report: Improve soak_tests/index.rst

- The -iterative- jobs should be used in future,
  but for 2005 the results still come from -verify- jobs.

Change-Id: Ief6651a17543f27a3ccfaf2861aa6f511d80127e
Signed-off-by: Vratko Polak <vrpolak@cisco.com>