csit.git
4 years agoReport: Add VPP 2n-clx cx556a 53/24753/3
Tibor Frank [Mon, 3 Feb 2020 13:59:46 +0000 (14:59 +0100)]
Report: Add VPP 2n-clx cx556a

Change-Id: I95e713f1e1d41eb886261b802f711fbb6ad6d27f
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoAutogen: Fix typo affecting DPDK 49/24749/1
Vratko Polak [Mon, 3 Feb 2020 12:15:49 +0000 (13:15 +0100)]
Autogen: Fix typo affecting DPDK

With the typo, only the first type+nic combination is generated.
With fix, all combinations are generated.

Change-Id: Ieb2db31cebed4940b063505b70f4d12a30da489c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoReport: Add DPDK 2n-clx 41/24741/1
Tibor Frank [Mon, 3 Feb 2020 09:21:29 +0000 (10:21 +0100)]
Report: Add DPDK 2n-clx

Change-Id: I4cd0fb5169a8ce571cce23bf8c3b9709afe8dc9b
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoTrending: Add cx556a to urls 39/24739/1
Tibor Frank [Mon, 3 Feb 2020 07:39:05 +0000 (08:39 +0100)]
Trending: Add cx556a to urls

Change-Id: Ib86e9b1ad340c214fd98c7e736449e5ae3206096
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoTrending: Input data 33/24733/1
Tibor Frank [Sun, 2 Feb 2020 19:17:10 +0000 (20:17 +0100)]
Trending: Input data

Change-Id: I27aa8d5a5934e1c60c6e71f8dc95d25e0217a7a0
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoUpdate CRC comments 80/24680/3
Vratko Polak [Fri, 31 Jan 2020 09:19:20 +0000 (10:19 +0100)]
Update CRC comments

+ Link to current stable build.
+ Remove policer comment (as it is already a devicetest).
+ Add nsim comment.
+ Include perftest tag pattern to run a drv-avf test.
+ Switch from 2n-skx to 2n-clx to test also rdma-core driver.
+ Add tag for vfio-pci driver for the rest of tag expressions.

Change-Id: Id2f2ee0525e27437166431045a174b3097dff52b
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoRemove trailing space from CLI command 60/20560/3
Vratko Polak [Tue, 9 Jul 2019 12:52:37 +0000 (14:52 +0200)]
Remove trailing space from CLI command

Change-Id: I49adff478302da036ecfacb675983923dc3f1ce1
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoTox: Sort items and add comments on sort criterion 11/24711/2
Vratko Polak [Fri, 31 Jan 2020 13:34:00 +0000 (14:34 +0100)]
Tox: Sort items and add comments on sort criterion

Change-Id: I8d8619f5a0b7ec7ee45b85ef78ff6bb8e9ea88a1
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoT-Rex: CPU pinning 72/24672/10
Peter Mikus [Thu, 30 Jan 2020 13:45:11 +0000 (13:45 +0000)]
T-Rex: CPU pinning

+ Detect NUMA
+ Pin based on numa location

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ife350f8c70e5437ac7c1413c7753f2a2f62777d9

4 years agoTrending: Reduce input data 03/24703/1
Tibor Frank [Fri, 31 Jan 2020 09:59:32 +0000 (10:59 +0100)]
Trending: Reduce input data

Change-Id: Ic09061050e0e55e3beb5aaa13a8914ea4959efa2
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoTrending: Add mellanox 00/24700/1
Tibor Frank [Fri, 31 Jan 2020 09:23:23 +0000 (10:23 +0100)]
Trending: Add mellanox

Change-Id: Id6f621ee4d604da96cf65c7f1cb889ec055df9b1
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoperf: Add hoststack NSIM+LDPRELOAD+IPERF3 test suite 37/24337/31
Dave Wallace [Wed, 15 Jan 2020 06:29:16 +0000 (06:29 +0000)]
perf: Add hoststack NSIM+LDPRELOAD+IPERF3 test suite

Change-Id: Ia7a876b1aa240676e1f2d23618c1d4e09ead14f0
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
4 years agoPAL: Latency processing 28/24628/2
Tibor Frank [Wed, 29 Jan 2020 15:30:05 +0000 (16:30 +0100)]
PAL: Latency processing

Very ugly solution for now. It will be replaced by using JSON structure
for test results soon.

Change-Id: If5b63de94bf02e7fc2a518c1f6a94f42dcec9e07
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoDisplay target_tr for ndr/pdr results 26/24626/2
Vratko Polak [Wed, 29 Jan 2020 14:28:16 +0000 (15:28 +0100)]
Display target_tr for ndr/pdr results

Small differences in Tx packet counts can cause
the previous way (which used transmit_rate,
computed from packets sent and nominal duration)
to report lower bounds as higher than upper bounds.

Also, reconf tests need the target (not actual) rate.

Change-Id: If5222dabd9762b8d9c2e22d875c12ad355bc6838
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoNdrpdr: Remove unused latency measurements 94/24594/2
Vratko Polak [Tue, 28 Jan 2020 14:26:01 +0000 (15:26 +0100)]
Ndrpdr: Remove unused latency measurements

+ Improve keyword Documentation.
+ Reorder measurements, higher loads first.

Change-Id: I1aeab1c8ada268422e9f421e075aff2ac9eec23f
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoUpdate of VPP_STABLE_VER files 35/24635/1
Jan Gelety [Sun, 19 Jan 2020 21:31:12 +0000 (22:31 +0100)]
Update of VPP_STABLE_VER files

 - use new vpp ref build - ubuntu 18.04:
   20.05-rc0~50-g31b1a6ce1

 - use new vpp ref build - centos7:
   20.05-rc0~50_g31b1a6c~b8758

Change-Id: Ic7e1b5e373070d8b30f0c7811fb85e7e5d88bfd7
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoperf: QUIC transport hoststack test suite 54/24254/36
Dave Wallace [Wed, 8 Jan 2020 21:14:25 +0000 (21:14 +0000)]
perf: QUIC transport hoststack test suite

Change-Id: I73f4be7ea315c7a5dcce46e1bd3034bcb0a97ee2
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
4 years agoReport: Fix: Add new NFV data 21/24621/1
Tibor Frank [Wed, 29 Jan 2020 12:38:13 +0000 (13:38 +0100)]
Report: Fix: Add new NFV data

Change-Id: Iae3557db30c5a95cb26d2e9b5acf24df2f64e429
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoReport: Add new NFV data 17/24617/3
Tibor Frank [Wed, 29 Jan 2020 12:25:48 +0000 (13:25 +0100)]
Report: Add new NFV data

Change-Id: Ic5a20940466545611150cfceaed58cee702636c4
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoIPSEC: Change plugin naming 07/24607/2
Peter Mikus [Wed, 29 Jan 2020 07:31:23 +0000 (07:31 +0000)]
IPSEC: Change plugin naming

https://gerrit.fd.io/r/c/vpp/+/24574

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ic2b7c925ceba1e16de87c64003ecceeea69c681c

4 years agoReport: Add NFV data for existing elements 06/24606/3
Tibor Frank [Wed, 29 Jan 2020 07:32:00 +0000 (08:32 +0100)]
Report: Add NFV data for existing elements

Change-Id: Ib76dbf36f6eb0234f6947fc43bd6c31e12a1d8e9
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoIpsec: Unify first line of Local Template doc 91/24591/3
Vratko Polak [Tue, 28 Jan 2020 12:20:24 +0000 (13:20 +0100)]
Ipsec: Unify first line of Local Template doc

Seeing differences when diff-ing between suites is distracting.

+ Bump copyright year, even for files with no change.

Change-Id: Iaca79647821dd8233bdbe6b0ac8b14fdb04060a8
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoperf: hoststack iperf3 test tuning 55/24455/17
Dave Wallace [Tue, 21 Jan 2020 19:02:14 +0000 (19:02 +0000)]
perf: hoststack iperf3 test tuning

Change-Id: I53425f57fe9ecef9cff2c94642cc7cb24537a961
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
4 years agoFix two auth_alg values 88/24588/3
Vratko Polak [Tue, 28 Jan 2020 11:34:24 +0000 (12:34 +0100)]
Fix two auth_alg values

Change-Id: I0e85fc958779df3d5dbacf1ad1e3898268a832ec
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoUpdate INFO.yaml file 62/24562/2
Vanessa Rene Valderrama [Mon, 27 Jan 2020 19:48:58 +0000 (13:48 -0600)]
Update INFO.yaml file

Updating the INFO.yaml to get the job passing and allow for future
automation

Signed-off-by: Vanessa Rene Valderrama <vvalderrama@linuxfoundation.org>
Change-Id: I62963efce7078872268cd177623660cbce06dd19

4 years agoUpdate overheads for IPsec CBC tests 51/24551/3
Vratko Polak [Tue, 28 Jan 2020 11:18:22 +0000 (12:18 +0100)]
Update overheads for IPsec CBC tests

Updated to the values as seen in packet trace.
Even if VPP creates wrongly sized packets (compared to RFCs),
the overhead should correspond to the actual packet size present,
in order to correctly prevent linerate on DUT-DUT link.

The new overhead values are 62 (256SHA) and 78 (512SHA).
The GCM value is already correct, at 54 bytes, so density tests are ok.

- The lispgpe test is not updated, as it currently fails.
  We will update overhead there in (or after) 24578.

Change-Id: I5cc6920205f37ddc80e76804fabd90b67174addf
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoCheck interval validity in reconf tests 12/24512/4
Vratko Polak [Fri, 24 Jan 2020 15:16:10 +0000 (16:16 +0100)]
Check interval validity in reconf tests

Without this, if the search part returns min_rate,
then reconf part reports few seconds with a small loss count.

But min_rate means there is a bug or misconfiguration,
so the tests should fail, instead of returning misleading results.

Change-Id: I566608b40ae11d6aaa3fe022a32da97901e1999c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoPerform some cleanup discovered by GPL work 29/24429/5
Vratko Polak [Fri, 24 Jan 2020 11:22:14 +0000 (12:22 +0100)]
Perform some cleanup discovered by GPL work

+ Remove one __init__.robot file with no function.
+ Remove executable flags from traffic profiles.
+ Improve indentation in some traffic profiles.

Change-Id: I099e63069003fba788976a9470a909ccff51a88c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoFIX: dot1qip4vxlan tests 84/24584/2
Peter Mikus [Tue, 28 Jan 2020 10:33:36 +0000 (10:33 +0000)]
FIX: dot1qip4vxlan tests

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ieb6e6010108987f99f55730149e3e4b7f1a7fc21

4 years agoReport: Add RC2 data 80/24580/1
Tibor Frank [Tue, 28 Jan 2020 10:16:34 +0000 (11:16 +0100)]
Report: Add RC2 data

Change-Id: I29dc3e34293d8bcc940e2adc3034e32b3806c71c
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoPAL: Add latency by percentile graph 43/24543/18
Tibor Frank [Mon, 27 Jan 2020 14:24:08 +0000 (15:24 +0100)]
PAL: Add latency by percentile graph

Change-Id: I38c03e089fd709afc848ecec55474d5c6925a58e
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoIncrease min_rate for NDRPDR tests 31/24531/2
Vratko Polak [Mon, 27 Jan 2020 13:17:24 +0000 (14:17 +0100)]
Increase min_rate for NDRPDR tests

Latency measurements break for loads less than 9 kpps per direction.
Setting min_rate to 90 kpps prevents 10% latency trial from breaking.
90 kpps should be enough for any type of test.

When some test finds a lower value, this patch will make the fail
with a message saying the loss rate at minimal load is too high
(instead of a cryptic error from 10% measurement without this patch).

Change-Id: I97b8428aa5044da23343b86447857b3a101f6baa
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoFIX: t29 yaml file 41/24541/1
Jan Gelety [Sun, 19 Jan 2020 03:31:10 +0000 (04:31 +0100)]
FIX: t29 yaml file

Change-Id: I20e01457b492ddb9f23cc31f5eab8388f3ea2405
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoReport: Add RC1 data 36/24536/1
Tibor Frank [Mon, 27 Jan 2020 14:31:29 +0000 (15:31 +0100)]
Report: Add RC1 data

Change-Id: I3d6d9c56456a12435096f7c61f744519279a850b
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoReport: Add RC1 data 32/24532/1
Tibor Frank [Mon, 27 Jan 2020 13:33:25 +0000 (14:33 +0100)]
Report: Add RC1 data

Change-Id: I7ebbabd1f56b892b6f88a6a183632cb11fc2bc3c
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoRestore PDR stat run for NDRPDR 78/24478/4
Vratko Polak [Mon, 27 Jan 2020 12:26:27 +0000 (13:26 +0100)]
Restore PDR stat run for NDRPDR

Before [0], the NDRPDR test ended with two measurements
that perform runtime stats collection, each 2 seconds long.
[0] then decreased that duration to one second,
and removed the PDR measurement.

This change re-adds the PDR measurement,
while keeping the duration at 1 second.

[0] https://gerrit.fd.io/r/c/csit/+/24427

Change-Id: I76828f11863a9e278cdd0cf60f2792d5b614ce43
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoAdd t29 Mellanox NIC into topology 28/24528/3
Peter Mikus [Mon, 27 Jan 2020 12:49:23 +0000 (12:49 +0000)]
Add t29 Mellanox NIC into topology

Change-Id: I1361595b2a7bda78f45b1379c29a293f863728a8
Signed-off-by: Peter Mikus <pmikus@cisco.com>
4 years agoReport: Add RC1 data 24/24524/2
Tibor Frank [Mon, 27 Jan 2020 12:41:22 +0000 (13:41 +0100)]
Report: Add RC1 data

Change-Id: I7ab49f6a910b9273b78e89096e06761ff7537ec9
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoUpdate of VPP_STABLE_VER files 22/24522/1
Jan Gelety [Sun, 19 Jan 2020 00:36:03 +0000 (01:36 +0100)]
Update of VPP_STABLE_VER files

 - use new vpp ref build - ubuntu 18.04:
   20.05-rc0~29-ga273ce62d

 - use new vpp ref build - centos7:
   20.05-rc0~29_ga273ce6~b8737

Change-Id: I57f181961796d90ee4064b3f03d7f421d382cd27
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoAttempt to fix journalctl 06/24506/2
Peter Mikus [Fri, 24 Jan 2020 12:08:52 +0000 (12:08 +0000)]
Attempt to fix journalctl

- AWK
- ECHO
- pipe
+ Much simpler

Change-Id: I4876f4eae68c96571dc38fd21e53ba269f5f888e
Signed-off-by: Peter Mikus <pmikus@cisco.com>
4 years agoFIX: VTS tests 92/24492/2
Jan Gelety [Sat, 18 Jan 2020 10:03:52 +0000 (11:03 +0100)]
FIX: VTS tests

Change-Id: I594d248c58dcdcdeceea57af2dd25e2b2e08247f
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoFIX: nfv_density 49/24449/4
Jan Gelety [Fri, 17 Jan 2020 17:53:27 +0000 (18:53 +0100)]
FIX: nfv_density

- use correct osi_layer=L2 (so L2 spoofing check is switched off
  in case of avf driver)
- add pci address information to eth interface in topology file
- nfv_density chain_ipsec tests work only with DPDK
  in current implementation

Change-Id: I233c6e5634a14581960c7459b87f11fcee8365bd
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoFIX: Various issues 74/24474/7
Peter Mikus [Wed, 22 Jan 2020 13:11:14 +0000 (13:11 +0000)]
FIX: Various issues

- TrafficProfile
- IPsec jumbo
- TDLK leftover
- Trex scapy server

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I0601ea75ed6e7318c46124fb02e9c61066089791

4 years agoReport: Add diagrams for testbeds 76/24476/3
Tibor Frank [Wed, 22 Jan 2020 14:27:29 +0000 (15:27 +0100)]
Report: Add diagrams for testbeds

- Diagrams for testbeds:
  - 3n-dnv
  - 3n-tsh
  - 2n-clx
- Replace s/Cascadelake/Cascade Lake

Change-Id: I77e7659a0aba4766a28577f940b7e44e60cbd82d
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoGenerate tests in deterministic order 44/24444/5
Vratko Polak [Tue, 21 Jan 2020 11:47:46 +0000 (12:47 +0100)]
Generate tests in deterministic order

Change-Id: I47e45f395976fd65a8da0e93bc33588cc213c79e
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agofix loadbalancer running error(Caused by code upgrades) 61/24461/2
haiyanx1.zhang [Tue, 21 Jan 2020 10:08:08 +0000 (10:08 +0000)]
fix loadbalancer running error(Caused by code upgrades)

Signed-off-by: haiyanx1.zhang <haiyanx1.zhang@intel.com>
Change-Id: I39bd1d9ad59971bb8cba6e664ba5509abba6a527

4 years agoPAL: Process HDRH Latency 47/24447/2
Tibor Frank [Tue, 21 Jan 2020 13:04:01 +0000 (14:04 +0100)]
PAL: Process HDRH Latency

Change-Id: Ifc48869ccb6c05f4b5af6cd97e05f009c7632dc7
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoNdrpdr: Add latency trials for smaller loads 27/24427/8
Vratko Polak [Tue, 21 Jan 2020 11:02:46 +0000 (12:02 +0100)]
Ndrpdr: Add latency trials for smaller loads

Change-Id: I46bd4b58bdd02a3deed38c158214e1e9977dde05
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoUpdate of VPP_STABLE_VER files 25/24425/2
Jan Gelety [Fri, 17 Jan 2020 00:59:29 +0000 (01:59 +0100)]
Update of VPP_STABLE_VER files

 - use new vpp ref build - ubuntu 18.04:
   20.05-rc0~14-gdb43bb6af

 - use new vpp ref build - centos7:
   20.05-rc0~14_gdb43bb6~b8721

Change-Id: I482f40023f6708908e510fa5eede391f62f68c95
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoreduce test cases for 2n-dnv and 3n-dnv mrr daily test 16/24416/2
Yulong Pei [Sun, 19 Jan 2020 12:58:07 +0000 (20:58 +0800)]
reduce test cases for 2n-dnv and 3n-dnv mrr daily test

since dnv platform is to use by low end market product, no need care
more about too much flows performance test.

Change-Id: Id7204b0353cbf968e641f14ed980f01af75971b2
Signed-off-by: Yulong Pei <yulong.pei@intel.com>
4 years agoFIX: Hugepages should be detected also from overcommit 83/24083/11
Peter Mikus [Thu, 19 Dec 2019 19:20:20 +0000 (19:20 +0000)]
FIX: Hugepages should be detected also from overcommit

Change-Id: I36ec9d61f775791ad0632a579c4921889d1f5acf
Signed-off-by: Peter Mikus <pmikus@cisco.com>
4 years agorevert "Report: Remove 'avf' from TC name in plots" 20/24420/1
Tibor Frank [Mon, 20 Jan 2020 07:17:02 +0000 (08:17 +0100)]
revert "Report: Remove 'avf' from TC name in plots"

Change-Id: I9b10b8f2f84daad543ac9b84671753785dfb23ad
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoperf: add TCP Iperf3+LDPRELOAD test suite 52/24252/20
Dave Wallace [Wed, 8 Jan 2020 20:51:43 +0000 (20:51 +0000)]
perf: add TCP Iperf3+LDPRELOAD test suite

Change-Id: Icff49fb31cce342a2a4ae799e844ec91f9e5e366
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
4 years agoFix unreservation 08/24408/1
Vratko Polak [Fri, 17 Jan 2020 13:20:49 +0000 (14:20 +0100)]
Fix unreservation

It got broken by the previous reservation fix.

Change-Id: I5d9091b7fa4fcc37148c0223fe8eb83f7e5b6770
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoReport: Remove 'avf' from TC name in plots 03/24403/1
Tibor Frank [Fri, 17 Jan 2020 12:35:31 +0000 (13:35 +0100)]
Report: Remove 'avf' from TC name in plots

Change-Id: Ie1d04347de123032a5d424747d216f90022bc813
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoFIX: remove 4drc nfv_density chain ipsec tests from 3n-hsw daily job 00/24400/1
Jan Gelety [Thu, 16 Jan 2020 22:39:05 +0000 (23:39 +0100)]
FIX: remove 4drc nfv_density chain ipsec tests from 3n-hsw daily job

- not enough cpu cores reported

Change-Id: If3ddbbf4f5af741f328c45ac6b8c874e61d348ab
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoFIX: missing write of mac_dst for direction 0 --> 1 in cnf trex profiles 97/24397/2
Jan Gelety [Thu, 16 Jan 2020 22:22:41 +0000 (23:22 +0100)]
FIX: missing write of mac_dst for direction 0 --> 1 in cnf trex profiles

Change-Id: I3acf5220c24f77bc4bfc451919bf9b5345a988ef
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoFix reservation if connect fails 93/24393/1
Vratko Polak [Fri, 17 Jan 2020 11:26:47 +0000 (12:26 +0100)]
Fix reservation if connect fails

Change-Id: I136cc51cbd5a01451e04a39c5d8e5554d6d39305
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoReport: Prepare data sets 86/24386/2
Tibor Frank [Fri, 17 Jan 2020 09:18:57 +0000 (10:18 +0100)]
Report: Prepare data sets

Change-Id: Ice90ffe8563d6283050f2b229fc5e50ab64ac253
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoFIX: L2XC with vhost and VLAN 82/24382/1
Jan Gelety [Thu, 16 Jan 2020 18:49:06 +0000 (19:49 +0100)]
FIX: L2XC with vhost and VLAN

Change-Id: Ia87abadba431099eeacd8b50ce5d116597a1f393
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoPython3: Migration of files under traffic-profiles/trex 27/24327/6
Balaji Venkatraman [Tue, 14 Jan 2020 14:57:10 +0000 (06:57 -0800)]
Python3: Migration of files under traffic-profiles/trex

Change-Id: Iae16225d3d3daa4e1e9b177228fb89efe4aa5405
Signed-off-by: Balaji Venkatraman <balajiv@cisco.com>
4 years agoReport: Latency by Percentile Distribution 70/24370/1
Tibor Frank [Thu, 16 Jan 2020 12:31:08 +0000 (13:31 +0100)]
Report: Latency by Percentile Distribution

- legend

Change-Id: I35477ce5c2cabfdcef68424854a8574bd3655e55
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoReport: Latency by Percentile Distribution 67/24367/1
Tibor Frank [Thu, 16 Jan 2020 09:53:35 +0000 (10:53 +0100)]
Report: Latency by Percentile Distribution

- Set X-axis

Change-Id: Ica300c7b1352c824bf51ac26e7485c9295ac60dc
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoReport: Add placeholders for new TBs 83/24283/3
Tibor Frank [Fri, 10 Jan 2020 12:41:09 +0000 (13:41 +0100)]
Report: Add placeholders for new TBs

Change-Id: I92cb693343c4f96503b45a15211e0dadd6fcde5f
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoRemove leftovers from previous removals 02/23602/13
Vratko Polak [Thu, 5 Dec 2019 14:37:27 +0000 (15:37 +0100)]
Remove leftovers from previous removals

- Except KubernetesUtils and its bash dependencies.
  It is a dead code currently, but we hope new tests will resurrect it.
+ Mention device tests instead of functional in Readme.

Change-Id: I6bdbea13ea9f995a06a0fb7cb5e98ae4fce29581
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoReport: Latency by Percentile Distribution 52/24352/1
Tibor Frank [Wed, 15 Jan 2020 14:48:48 +0000 (15:48 +0100)]
Report: Latency by Percentile Distribution

- Set the position of the Lagend

Change-Id: I1bcea6002bd8beeba3300c9976d1603a234c3c14
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoReport: Add Latency by Percentile Distribution 44/24344/10
Tibor Frank [Wed, 15 Jan 2020 10:15:14 +0000 (11:15 +0100)]
Report: Add Latency by Percentile Distribution

Change-Id: I5291db8117da611497e910aa7cd2fbb2276bfe65
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoDocs: Update report sections 49/24349/2
Peter Mikus [Wed, 15 Jan 2020 13:12:48 +0000 (13:12 +0000)]
Docs: Update report sections

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ib6e9ac003c1d91673984c52baf2f3eec30c9d21d

4 years agoAnsible: Increase TG hugepages 46/24346/2
Peter Mikus [Wed, 15 Jan 2020 12:01:35 +0000 (12:01 +0000)]
Ansible: Increase TG hugepages

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ifa87e372ed45f6dfc883723acbb325e34fb814a8

4 years agoRemove everything related to TLDK 70/23570/16
Vratko Polak [Fri, 22 Nov 2019 13:14:15 +0000 (14:14 +0100)]
Remove everything related to TLDK

- Leftovers from kubernetes found, but not removed here.

Change-Id: If8cb9269d0f3e69f642d7fe02c59122e17925a4d
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoTrending: new daily set 35/24035/17
Jan Gelety [Fri, 13 Dec 2019 15:11:58 +0000 (16:11 +0100)]
Trending: new daily set

New daily sets are prepared based on information in
https://gerrit.fd.io/r/c/csit/+/24073/1/docs/job_specs/perf_tests_job_specs.md
and previous test set definitions in
docs/job_specs/test_select_list_[2n|3n]_[clx|skx|hsw|tsh|dnv].md
files.

- mrr-daily-2n-clx: 510 TCs (incl. nfv_density), expected exec. time 8:50h
- mrr-daily-2n-skx: 525 TCs (incl. nfv_density), expected exec. time 7:55h
- mrr-daily-3n-skx: 393 TCs (incl. nfv_density), expected exec. time 11:00h
- mrr-daily-3n-hsw: 177 TCs (incl. nfv_density), expected exec. time 7:10h
- mrr-daily-3n-tsh: 204 TCs, expected exec. time 21:00h
- mrr-daily-2n-dnv: 84 TCs, expected exec. time 2:25h
- mrr-daily-3n-dnv: 144 TCs, expected exec. time 6:35h

+ add some missing test suites
+ add trex-sl-2n-ethip4udp-1000u15p.py T-Rex traffic profile
+ correction of TS and TC names and tags in directory
  tests/vpp/perf/nfv_density/chain_ipsec

Change-Id: Icfc86e9af97ed8dd8ccd2a34355c99aad69a28c0
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoSupport suite tags in autogen 54/24054/10
Vratko Polak [Thu, 2 Jan 2020 14:30:09 +0000 (15:30 +0100)]
Support suite tags in autogen

+ Include a script to add suite tags to many suites at once.
+ Add suite tags also to device tests (not covered by autogen).

Change-Id: I514ee6178e22999b43460028fe2696738b012f04
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoAutogen: Generate also NIC drivers. 33/23033/34
Vratko Polak [Fri, 3 Jan 2020 15:01:30 +0000 (16:01 +0100)]
Autogen: Generate also NIC drivers.

+ Disallowed -avf- (or -rdma-) as "template" suites.
 + GBP suite switched to DPDK driver in repo.
+ Each NIC has its own list of supported drivers, in Constants.
+ Updated tag expressions for daily jobs:
 + Feature, ipsec, memif, scale, srv6, tunnels, vhost and vts
   are tested only with vfio-pci.
 + Other (base, dot1q, dot1ad) tested with all drivers.
+ Setup actions currently depend on driver, generated.
 - The performance_rdma action is trivial for now.
 - Several tests fail, to be fixed later, e.g. by performance_rdma.
+ Reconf tests are also supported.
+ Added DRV_VFIO_PCI tags missing, mainly in density tests.
 - Vhost suites (density, reconf) are failing, but suites look good.
- TCP suites do not support NIC drivers yet.
- DPDK obviously not supported.
+ Use Python 3 in regenerate scripts.
+ Fix typos binded => bound.
+ File open modes set either u"rt" or u"wt" everywhere.
+ Remove a trailing space in an environment variable name.

Change-Id: I290470675dc5c9e88b2eaa5ab6285ecd9ed7827a
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoReport: Current vs. Previous Release - NFV Tests 21/24321/4
Tibor Frank [Tue, 14 Jan 2020 12:47:18 +0000 (13:47 +0100)]
Report: Current vs. Previous Release - NFV Tests

Change-Id: I00cbde38b8f900efe3e3123a942de2288381ea3b
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoHoststack perf infrastructure refactoring 03/24103/26
Dave Wallace [Fri, 20 Dec 2019 22:46:51 +0000 (22:46 +0000)]
Hoststack perf infrastructure refactoring

- DUT only topology (hoststack test
  apps are co-located with vpp)
- Make vpp app specific keywords generic
  where applicable
- Add IP4 Prefix to topology file
- Support running wrk in linux namespace
- Refactor namespace cleanup
- Remove redundant namespace creation code
- Refactor test/keyword dirs: tcp -> hoststack
- Add hoststack utility keywords
- Refactor wrk suite setup/teardown
- Update tests with recent perf infra changes

Change-Id: Ia1cf07978d579393eef94923819a87c8c1f36f34
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
4 years agoFIX: VXLAN scale 28/24328/2
Peter Mikus [Tue, 14 Jan 2020 15:05:22 +0000 (15:05 +0000)]
FIX: VXLAN scale

+ Another stupid arp -> neighbor rename

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ia873d6c2a485659033f71cdf546782db74787d13

4 years agoFIX: NF_density tests 15/24315/2
Peter Mikus [Tue, 14 Jan 2020 07:51:12 +0000 (07:51 +0000)]
FIX: NF_density tests

+ ARP -> Neighbor

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I74044bd7db6cb4b1bd3648a424e9c8f1473be8d1

4 years agoUpdate comments in crc file. 00/24300/3
Vratko Polak [Mon, 13 Jan 2020 13:55:27 +0000 (14:55 +0100)]
Update comments in crc file.

+ L2patch is already covered.
+ RDMA is not device testable yet.

Change-Id: I1dcccfcb340a8fe61c20d3516340e26db293cfaa
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoReport: Add VNF Service Chains Tunnels 08/24308/2
Tibor Frank [Mon, 13 Jan 2020 14:20:12 +0000 (15:20 +0100)]
Report: Add VNF Service Chains Tunnels

Change-Id: Iafdcb543f6adc583567a05aa5a882dce17b58f03
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoFIX: NF_density tests 02/24302/2
Peter Mikus [Mon, 13 Jan 2020 13:22:31 +0000 (13:22 +0000)]
FIX: NF_density tests

+ Because arp->neigbor was so important to do.

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I552e175b7555ebf5053d7994c0c9173c0c96fc58

4 years agoReport: Move VTS tests from 3n to 2n 95/24295/2
Tibor Frank [Mon, 13 Jan 2020 10:23:27 +0000 (11:23 +0100)]
Report: Move VTS tests from 3n to 2n

Change-Id: Ie2faa6cb611d372c4a20584f7b40f3281660f2b1
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoCSIT-1597 API cleanup: rdma 80/24280/3
Jan Gelety [Thu, 9 Jan 2020 16:22:06 +0000 (17:22 +0100)]
CSIT-1597 API cleanup: rdma

- cover API changes in VPP: https://gerrit.fd.io/r/c/vpp/+/24225
- update vpp stable to version 20.01-rc0~1011

Change-Id: Ibbfdbe867cf89715619efc5c5507859983025ece
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoReport: Add comparison tables 82/24282/1
Tibor Frank [Fri, 10 Jan 2020 12:22:00 +0000 (13:22 +0100)]
Report: Add comparison tables

CSIT-1654

Change-Id: I280195b295d4fc717cb355e78c5f1731bb1eabe5
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoReport: Add 2n-clx 67/24267/4
Tibor Frank [Thu, 9 Jan 2020 14:02:10 +0000 (15:02 +0100)]
Report: Add 2n-clx

Change-Id: I34dc11c28d21f0b9fed87d76942fe8b8cdd86c88
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoFIX: Container test 68/24268/2
Peter Mikus [Thu, 9 Jan 2020 13:45:06 +0000 (13:45 +0000)]
FIX: Container test

+ We should check if container is UP
+ We should adjust privileges.

Change-Id: I3f604b7b9740943b4fd3b0574c18d2729e88b816
Signed-off-by: Peter Mikus <pmikus@cisco.com>
4 years agoReport: Configure Report 2001 61/24261/3
Tibor Frank [Thu, 9 Jan 2020 09:11:55 +0000 (10:11 +0100)]
Report: Configure Report 2001

Change-Id: I5ef3c37cf329b76a1a7f94d60c3563872eef3c5d
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoDMM: Remove 20/23420/11
Tibor Frank [Thu, 14 Nov 2019 08:28:14 +0000 (09:28 +0100)]
DMM: Remove

Change-Id: Ibbfbed79e473c804390802ae1ecd737b50c06aa3
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoAPI: Add collection for 24225/3 46/24246/1
Jan Gelety [Wed, 8 Jan 2020 02:11:12 +0000 (03:11 +0100)]
API: Add collection for 24225/3

Change-Id: I5c4593d05dbcb4d918611dc7520432a446dbf664
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoPAL: Set y-axis for HDRHistogram 36/24236/5
Tibor Frank [Wed, 8 Jan 2020 09:41:50 +0000 (10:41 +0100)]
PAL: Set y-axis for HDRHistogram

Change-Id: Ibdee625d6d9215b18771a6348735dc6a07e9ae58
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoTrack real(er) duration of traffic 56/24156/3
Vratko Polak [Wed, 8 Jan 2020 14:08:14 +0000 (15:08 +0100)]
Track real(er) duration of traffic

+ Print duration.
- Missing: Parse duration.
  - We do not have good formula to compensate, yet.

Change-Id: I1c76ac35ce531b1994480d6aa0638f16992231f5
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoT-Rex: Upgrade to 2.73 with DDP profile fix 00/23900/7
Peter Mikus [Tue, 10 Dec 2019 09:04:59 +0000 (09:04 +0000)]
T-Rex: Upgrade to 2.73 with DDP profile fix

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Icaa99e0a71a19ba23697c21a4b5aa046813a718a

4 years agoVhost: Add GSO option 53/24153/8
Peter Mikus [Thu, 2 Jan 2020 13:25:01 +0000 (13:25 +0000)]
Vhost: Add GSO option

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Iba9af71c293645b480203af72fcf940cbe9ccb3a

4 years agoFix Trex start configuration 81/24181/11
Vratko Polak [Tue, 7 Jan 2020 09:23:23 +0000 (10:23 +0100)]
Fix Trex start configuration

Since https://gerrit.fd.io/r/c/csit/+/24056
csit-vpp perf tests are failing.

Turns out the reason is too large limit_memory value,
but other improvements applied during investigation
are included as well.

+ Cat trex.log if trex fails to start.
+ Improve trex startup command formatting (remove double spaces).
+ Reorder imports.

Change-Id: I2c1106ea6f88a1a275682e73eba212d08c7947c8
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoRemove leftovers related to Xenial 17/24217/2
Vratko Polak [Tue, 7 Jan 2020 11:44:19 +0000 (12:44 +0100)]
Remove leftovers related to Xenial

+ File VPP_STABLE_VER_UBUNTU.
+ Subtree docs/report/vpp_functional_tests/.

Change-Id: Ifd9dd922152199de9f1352da709b1533d6c15aba
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
4 years agoRemove VIRL Func tests from Report 19/24219/1
Tibor Frank [Tue, 7 Jan 2020 13:27:39 +0000 (14:27 +0100)]
Remove VIRL Func tests from Report

Change-Id: Ic6edf1f6f6c12d8e76da76617dbf1c4e8e1e2952
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoUpdate of VPP_STABLE_VER files 12/24212/2
Jan Gelety [Tue, 7 Jan 2020 08:59:10 +0000 (09:59 +0100)]
Update of VPP_STABLE_VER files

 - use new vpp ref build - ubuntu 18.04:
   20.01-rc0~983-g78565f38e

 - use new vpp ref build - centos7:
   20.01-rc0~983_g78565f3~b8651

  + remove EXPECTED_FAILING tag from tc01-64B-ethip4-l2patch-dev test

Change-Id: Iab47a66003926024f87e028b1b1d9136b8fb4ec4
Signed-off-by: Jan Gelety <jgelety@cisco.com>
4 years agoTrending: Move VTS tests from 3n to 2n 13/24213/1
Tibor Frank [Tue, 7 Jan 2020 09:43:08 +0000 (10:43 +0100)]
Trending: Move VTS tests from 3n to 2n

Change-Id: I420cef699d42973602f1290ccd0cc5e378d42492
Signed-off-by: Tibor Frank <tifrank@cisco.com>
4 years agoAdd new NIC limits 52/24152/2
Peter Mikus [Thu, 2 Jan 2020 09:58:34 +0000 (09:58 +0000)]
Add new NIC limits

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I2b1a0d1455ae74a2ad4fdb553286aef5942f2a99

4 years agoRevert "L2Patch: Remove EXPECTED_FAILING" 55/24155/1
Peter Mikus [Thu, 2 Jan 2020 14:31:18 +0000 (14:31 +0000)]
Revert "L2Patch: Remove EXPECTED_FAILING"

This reverts commit 09c5a6b8e1c6efed8826ef34aa64809226e80edb.

Reason for revert: CSIT has not yet the latest VPP version

Change-Id: Ibaa2c00c639bacef1561898daf9485c3a68efec4
Signed-off-by: Peter Mikus <pmikus@cisco.com>
4 years agoL2Patch: Remove EXPECTED_FAILING 51/24151/1
Peter Mikus [Thu, 2 Jan 2020 09:35:02 +0000 (09:35 +0000)]
L2Patch: Remove EXPECTED_FAILING

Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Ie950370c47403872597f8857edd651df2552ccb2