csit.git
5 years agoPatch: Remove show stats keyword from teardown phase 57/1257/3
Peter Mikus [Wed, 25 May 2016 12:32:07 +0000 (14:32 +0200)]
Patch: Remove show stats keyword from teardown phase

- JIRA: CSIT-104
- Remove duplicity of showing stats
- Remove "Show statistics on all DUTs" from "Test Teardown"

Change-Id: Ic8e913396144c1ee50e878d9b0f85dffceee5287
Signed-off-by: Peter Mikus <pmikus@cisco.com>
5 years agoPatch: Set duration of performance long tests to 10s 97/1097/4
Peter Mikus [Fri, 13 May 2016 11:57:00 +0000 (13:57 +0200)]
Patch: Set duration of performance long tests to 10s

- JIRA: CSIT-77
Decrease duration time from 60 second to 10 second for all long
performance tests. This patch is preparation for replacement of linear
search for binary search.

Change-Id: I61fbec1dcaca73ba383690e695c3030d182a90f9
Signed-off-by: Peter Mikus <pmikus@cisco.com>
5 years agoReplace linear search with binary search 89/1189/5
Peter Mikus [Wed, 18 May 2016 16:45:30 +0000 (18:45 +0200)]
Replace linear search with binary search

- JIRA: CSIT-97
- Replace linear search with binary search for long performance testing

Change-Id: I19cbcf3d9f7f5e6ec5128287b58a2ea3aceff00c
Signed-off-by: Peter Mikus <pmikus@cisco.com>
5 years agoAdd test cases with "dpdk no-multi-seg" in vpp startup config 71/1071/19
Peter Mikus [Wed, 11 May 2016 13:07:06 +0000 (15:07 +0200)]
Add test cases with "dpdk no-multi-seg" in vpp startup config

- JIRA: CSIT-73
- Add ability to configure no-multi-seg parameter in VPP startup
  configuration
- Add keywords to selectively configure parameters in startup
  configuration
- Modify testcases

Change-Id: I3c5c0a1cc2b1156d5a87ff36628b50793dd9ac5f
Signed-off-by: Peter Mikus <pmikus@cisco.com>
5 years agoAdd temporary fix for CSIT-90 by flushing QGA's input buffer 30/1230/2
Carsten Koester [Mon, 23 May 2016 17:02:00 +0000 (13:02 -0400)]
Add temporary fix for CSIT-90 by flushing QGA's input buffer

This is not a final fix. The final fix SHOULD send a "guest-sync"
request as per"
http://wiki.qemu.org/Features/QAPI/GuestAgent#QEMU_Guest_Agent_Protocol

However, this temporary fix will eliminate the nested VM startup
failures triggered by the nested VM's guest-agent receiving a partial
guest-ping request.

Change-Id: Ice409b7c966a527b03cb71c042b476e1771676d0
Signed-off-by: Carsten Koester <ckoester@cisco.com>
5 years agoCapture VPP stats for Long and Short tests 95/1195/6
Miroslav Miklus [Thu, 19 May 2016 11:59:30 +0000 (13:59 +0200)]
Capture VPP stats for Long and Short tests

JIRA: CSIT-71
- remove warmup for short runs
- clear and collect stats for short run
- run short test with no failure for long test with NDR/PDR rate

Change-Id: I1b8879c3b891165072822207d0781687d8159336
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoPatch: Start binary search from max range rate 83/1183/5
Peter Mikus [Wed, 18 May 2016 12:53:41 +0000 (14:53 +0200)]
Patch: Start binary search from max range rate

- JIRA: CSIT-96
- Add option to start binary search with max range rate value first and
  then continue the binary search of half interval.

Change-Id: Ia6d2f4aa6fc20ca2ee4d43d0ddb3966422b28144
Signed-off-by: Peter Mikus <pmikus@cisco.com>
5 years agoPerformance l2-vlan-dot1q library and short/long test cases 73/973/17
pmikus [Tue, 3 May 2016 13:45:09 +0000 (15:45 +0200)]
Performance l2-vlan-dot1q library and short/long test cases

- JIRA: CSIT-59
- create topology with VLAN dot1q sub-interfaces inter-connected using
  L2 cross connect in 3-node topology
- create keywords to be used in test cases
- create l2-vlan-dot1q test cases (64B, 1518B, 9000B)

Change-Id: Ic5f31b4451965bb7f5d3d5b7061c6600f63da898
Signed-off-by: pmikus <pmikus@cisco.com>
Signed-off-by: Peter Mikus <pmikus@cisco.com>
5 years agoTB3 topology file with X520-DA2 NICs 02/1202/3
Miroslav Miklus [Thu, 19 May 2016 15:43:24 +0000 (17:43 +0200)]
TB3 topology file with X520-DA2 NICs

JIRA: CSIT-8

Change-Id: I95baf116cfcde85fdbaaaac079f27c898a959f58
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoVPP in host user-mode, part II 87/987/24
Zdeno Olsovsky [Wed, 4 May 2016 11:48:36 +0000 (13:48 +0200)]
VPP in host user-mode, part II

- JIRA: CSIT-61

Change-Id: I688b1013a78a1fa3ecdb44c2072f8bd1b46364fb
Signed-off-by: Zdeno Olsovsky <zolsovsk@cisco.com>
5 years agoAdd Honeycomb TAP interface test 37/1137/7
selias [Fri, 13 May 2016 11:33:45 +0000 (13:33 +0200)]
Add Honeycomb TAP interface test

JIRA: CSIT-75

 - add test suite for TAP interface management through Honeycomb
 - add keywords required for TAP interface tests
 - move Honeycomb suite setup and teardown to new __init__ file
 - cleanup unused imports in Honeycomb test case files
 - modify "...should be empty" keywords to allow more reuse

Change-Id: Ie3dc9069678b60c359f69c5a216e452e856a0107
Signed-off-by: selias <samelias@cisco.com>
5 years agoTests for vhost-user interface configuration 51/1151/7
Tibor Frank [Mon, 16 May 2016 16:20:30 +0000 (18:20 +0200)]
Tests for vhost-user interface configuration

JIRA: CSIT-76

- add tests for vhost-user interface manipulation:
  - create, modify, delete
- add negative tests:
  - wrong interface type
  - wrong configuration

Change-Id: I09239ef099a6f9672895f9540762e5044123f7ec
Signed-off-by: Tibor Frank <tifrank@cisco.com>
5 years agoTB1 topology file with X520-DA2 NICs only 42/1042/7
Miroslav Miklus [Mon, 9 May 2016 16:23:43 +0000 (18:23 +0200)]
TB1 topology file with X520-DA2 NICs only

JIRA: CSIT-8

Change-Id: I9c4d5d5502ed6e387a93e545c30278618a14a7fc
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoParse robot output.xml for performance reporting 68/968/9
pmikus [Tue, 3 May 2016 05:21:15 +0000 (07:21 +0200)]
Parse robot output.xml for performance reporting

- JIRA: CSIT-58
- parse robot framework output.xml file
- find performance related data
- write formatted json to specified file
- copy archive artifact to directory

Change-Id: I47e45bcb68c06044a23192cb1fca46f43782941e
Signed-off-by: pmikus <pmikus@cisco.com>
Signed-off-by: Peter Mikus <pmikus@cisco.com>
5 years agoT-REX stl traffic send improvement for async calls 90/1090/10
Miroslav Miklus [Thu, 12 May 2016 22:35:53 +0000 (00:35 +0200)]
T-REX stl traffic send improvement for async calls

JIRA: CSIT-68
- show runtime statistics with running traffic
- T-REX driver async. driver

Change-Id: Ie5eb7021f610fb58383b033dda5b1b867f7d3d2c
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoPERF bootstrap, apply global PYBOT args to all test types 98/998/5
Miroslav Miklus [Thu, 5 May 2016 12:06:11 +0000 (14:06 +0200)]
PERF bootstrap, apply global PYBOT args to all test types

Change-Id: Ia51a437d03e33df8e27da0afc099f808e255937c
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoExplicitly list the NIC type for perf tests 99/999/9
Miroslav Miklus [Thu, 5 May 2016 12:34:17 +0000 (14:34 +0200)]
Explicitly list the NIC type for perf tests

JIRA: CSIT-74
- FIX test names
- list NICs

Change-Id: Ic5dba9381b7bf7129cb0944deb33a86ccdbf2c7b
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoPatch: Detection of running t-rex instance before initializing 32/1132/4
Peter Mikus [Mon, 16 May 2016 07:55:39 +0000 (09:55 +0200)]
Patch: Detection of running t-rex instance before initializing

- JIRA: CSIT-89
- Add detection of running t-rex traffic generator before new instance
  is initialized.

Change-Id: I77009b0c5e83bb6e162b9800d02b4e5a2abd541f
Signed-off-by: Peter Mikus <pmikus@cisco.com>
5 years agoUpdate vpp stable build used in csit-vpp-verify-weekly jenkins job 44/1144/1
Jan Gelety [Mon, 16 May 2016 13:35:29 +0000 (15:35 +0200)]
Update vpp stable build used in csit-vpp-verify-weekly jenkins job

- vpp build version updated to release 1.0.0-451~g686a5a5_amd64

Change-Id: Ic7d177d72a6d50c083d5b9733c8f63298eebd7c6
Signed-off-by: Jan Gelety <jgelety@cisco.com>
5 years agoAdd IPsec traffic script 98/1098/3
Matus Fabian [Fri, 13 May 2016 12:42:44 +0000 (05:42 -0700)]
Add IPsec traffic script

JIRA: CSIT-28

Change-Id: Iaebea917ade85965b0dbaf35d58ea1ee14d76e3f
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoCIMANAGE-5: update bootstrap for csit-vpp-verify-weekly job 58/1058/2
Jan Gelety [Tue, 10 May 2016 21:34:15 +0000 (23:34 +0200)]
CIMANAGE-5: update bootstrap for csit-vpp-verify-weekly job

- file rename
- usage of vpp-verified build

Change-Id: I61b7ef4121d02149cf5b24514176b375e660b2ae
Signed-off-by: Jan Gelety <jgelety@cisco.com>
5 years agoAdd VIRL host selection mechanism to bootstrap 29/1029/3
Carsten Koester [Sun, 8 May 2016 05:46:13 +0000 (01:46 -0400)]
Add VIRL host selection mechanism to bootstrap

Add a mechanism to the bootstrap script that allows it to select
one out of multiple candidate VIRL hosts. In addition, fix a few
cosmetic issues wrt. connecting to the VIRL host:
* Use consistent set of SSH options
* Consistently use constant instead of hardcoded key filename
* Use relative path names for start-/stop-testcase scripts

Change-Id: I0d2972f170de82dd5d98da88656f5f962fca2415
Signed-off-by: Carsten Koester <ckoester@cisco.com>
5 years agoAdd Honeycomb VxLAN test suite 96/1096/5
selias [Fri, 13 May 2016 09:51:32 +0000 (11:51 +0200)]
Add Honeycomb VxLAN test suite

JIRA: CSIT-47

 - add test suite for VxLAN management through Honeycomb
 - add keywords required for VxLAN management tests
 - add resource file with additional variables for testing
 - increment bridge domain test suite index

Change-Id: Ica0d7ae15f5b7058832a944d5ca0c72026ba299c
Signed-off-by: selias <samelias@cisco.com>
5 years agoCSIT-18: Add GRE tunnel libraries and tests. 17/917/17
Matej Klotton [Thu, 28 Apr 2016 16:15:49 +0000 (18:15 +0200)]
CSIT-18: Add GRE tunnel libraries and tests.

Change-Id: I91ccd65a26ea1782611bccc3badd91e64366a162
Signed-off-by: Matej Klotton <mklotton@cisco.com>
5 years agoUpdate Honeycomb bridge domain tests 72/1072/2
selias [Wed, 11 May 2016 13:44:13 +0000 (15:44 +0200)]
Update Honeycomb bridge domain tests

 - fix interface assignment verification through VAT
 - add verification of split horizon group setting through VAT
 - clean up documentation of high level bridge domain keywords

Change-Id: I7552494d68c6bbc86d3e5ddea4e29c6a7444d50c
Signed-off-by: selias <samelias@cisco.com>
5 years agoAdd keyword to manipulate vhost-user parameters 86/1086/5
Tibor Frank [Thu, 12 May 2016 12:30:39 +0000 (14:30 +0200)]
Add keyword to manipulate vhost-user parameters

JIRA: CSIT-70

- add a keyword to be able to:
  - configure all vhost-user parameters at once
  - configure vhost-user parameters one by one
  - remove a vhost-user parameter
  - remove all vhost-user parameters at once
- add a keyword which adds a new vhost-user interface

Change-Id: I144bc035b959a1e4eb37c6a3d554e6843ddab4a7
Signed-off-by: Tibor Frank <tifrank@cisco.com>
5 years agoCSIT-65: Add new test for LISP API 00/800/22
Andrej Kozemcak [Fri, 8 Apr 2016 08:26:53 +0000 (10:26 +0200)]
CSIT-65: Add new test for LISP API

Test set and unset this API:
lisp_add_del_locator_set
lisp_add_del_locator
lisp_add_del_local_eid
lisp_add_del_map_resolver
lisp_gpe_add_del_iface

Change-Id: I9c701fc421d4106147891a7e75e5d68885b9ba59
Signed-off-by: Andrej Kozemcak <akozemca@cisco.com>
5 years agoAdd keyword to manipulate interface TAP parameters 85/1085/4
Tibor Frank [Thu, 12 May 2016 12:23:29 +0000 (14:23 +0200)]
Add keyword to manipulate interface TAP parameters

JIRA: CSIT-69

- add a keyword to be able to:
  - configure all TAP parameters at once
  - configure TAP parameters one by one
  - remove a TAP parameter
  - remove all TAP parameters at once
- add a keyword which adds a new TAP interface

Change-Id: I5620adc3f777f7a337bbd737e685891b425d4e09
Signed-off-by: Tibor Frank <tifrank@cisco.com>
5 years agoAdd possibility to create a VxLAN interface. 94/1094/3
Tibor Frank [Fri, 13 May 2016 08:25:16 +0000 (10:25 +0200)]
Add possibility to create a VxLAN interface.

JIRA: CSIT-51

- add a keyword which adds a new VxLAN interface
- change InterfaceUtil.vxlan_dump to be able to return info about all VxLAN
  interfaces.

Change-Id: I8ad3dc1d414924e74ea1ecea1f316ca5c648e2d0
Signed-off-by: Tibor Frank <tifrank@cisco.com>
5 years agoAdd possibility to change VxLAN parameters at once 83/1083/2
Tibor Frank [Thu, 12 May 2016 09:49:48 +0000 (11:49 +0200)]
Add possibility to change VxLAN parameters at once

JIRA: CSIT-51

- modify existing keyword "configure_interface_vxlan" to be able to:
  - configure all VxLAN parameters at once
  - configure VxLAN parameters one by one
  - remove a parameter
  - remove all parameters at once

Change-Id: I50551fbc3ae5c6aa147d531fc66b583af36bb541
Signed-off-by: Tibor Frank <tifrank@cisco.com>
5 years agoUpdate of vpp build for perf tests 31/931/6
Miroslav Miklus [Fri, 29 Apr 2016 11:55:40 +0000 (13:55 +0200)]
Update of vpp build for perf tests

build: 1.0.0-437~g8f15e92_amd64

Change-Id: Ie0d0f693bcae0d0e56472c25b6353890d07e85d9
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoUpdate Honeycomb interface IPv4 test 63/1063/7
selias [Wed, 11 May 2016 09:42:54 +0000 (11:42 +0200)]
Update Honeycomb interface IPv4 test

 - add verification of ipv4 subnet prefix support
 - modify keywords to allow setting ipv4 address with network prefix
 - fix variable definitions to evaluate numbers as ints, not strings
 - add exception to ipv4 netmask/prefix conversion method

Change-Id: I9343ceb35856ddb33674d7067f1def6d40e99acc
Signed-off-by: selias <samelias@cisco.com>
5 years agoAdd functions to get interface name or index from JSON interface dump. 16/916/10
Matej Klotton [Thu, 28 Apr 2016 16:10:05 +0000 (18:10 +0200)]
Add functions to get interface name or index from JSON interface dump.

Change-Id: I4778c9fa72e418f6fea427f88fd9aa1ea579ac99
Signed-off-by: Matej Klotton <mklotton@cisco.com>
5 years agoAdd library for set path variables for testing. 15/915/22
Matej Klotton [Thu, 28 Apr 2016 14:39:18 +0000 (16:39 +0200)]
Add library for set path variables for testing.

Change-Id: Icb243bacc9329c4bf3bc28098fced473b4e3c991
Signed-off-by: Matej Klotton <mklotton@cisco.com>
5 years agoPerformance IPv6 library and short/long test cases 24/824/17
pmikus [Wed, 20 Apr 2016 11:24:11 +0000 (13:24 +0200)]
Performance IPv6 library and short/long test cases

- JIRA: CSIT-2
- Create IPv6 test cases for long/short/singlethread/multithread performance testing
- Create robot KWs for IPv6 performance testing
- Add T-Rex IPv6 profile

Change-Id: I306d5c894d9b9287bf91929764aeb1c7083d802c
Signed-off-by: pmikus <pmikus@cisco.com>
Signed-off-by: Peter Mikus <pmikus@cisco.com>
5 years agoExtend ranges of binary search in ipv4 perf testing 18/1018/5
Peter Mikus [Fri, 6 May 2016 13:18:28 +0000 (15:18 +0200)]
Extend ranges of binary search in ipv4 perf testing

- Extend ranges of binary search in Long IPv4 testing to reflect the
  performance variability

Change-Id: I7632bbda361cb278be0776ec31461554297fe852
Signed-off-by: Peter Mikus <pmikus@cisco.com>
5 years agoMove Honeycomb libraries to honeycomb subdirectory. 50/1050/2
Tibor Frank [Tue, 10 May 2016 12:50:41 +0000 (14:50 +0200)]
Move Honeycomb libraries to honeycomb subdirectory.

- move all Honeycomb related libraries written in python to new
  directory resources/libraries/python/honeycomb
- update all related includes in all source files

Change-Id: Ifa96b8f670b456b14421929d6020ddf8e02d9cbc
Signed-off-by: Tibor Frank <tifrank@cisco.com>
5 years agoAdd iACL IPv4/IPv6 tests. 72/972/14
Patrik Hrnciar [Tue, 3 May 2016 12:40:52 +0000 (14:40 +0200)]
Add iACL IPv4/IPv6 tests.

 - IPv4: CSIT-15
 - IPv6: CSIT-16

Change-Id: I6e66aa853dfaebf1388f1191dbb63f5216820325
Signed-off-by: Patrik Hrnciar <phrnciar@cisco.com>
5 years agoAdd Max TX Queues configuration to VppConfigGenerator 84/984/3
pmikus [Wed, 4 May 2016 05:40:18 +0000 (07:40 +0200)]
Add Max TX Queues configuration to VppConfigGenerator

- Add ability to configure max-tx-queus parameter in VPP startup
  configuration

Change-Id: Iaf857ed037c934fbda46eda6bfe1be4ef7faba15
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoCSIT-9: COP Whitelist/blacklist 04/804/15
Zdeno [Mon, 18 Apr 2016 14:17:26 +0000 (16:17 +0200)]
CSIT-9: COP Whitelist/blacklist

- Included are also IPv6 tests
- JIRA: CSIT-17

Change-Id: I89ae6c38cdc6742a597c0dc24ed1c033c1b5d155
Signed-off-by: Zdeno <zolsovsk@cisco.com>
5 years agoCSIT-19: MAC split-horizon group 19/919/6
Jan Gelety [Thu, 28 Apr 2016 20:54:59 +0000 (22:54 +0200)]
CSIT-19: MAC split-horizon group

- test case to test MAC split-horizon group functionality

Change-Id: Iadee707f3670e5fc9c209bd8b072a65ea1af27cd
Signed-off-by: Jan Gelety <jgelety@cisco.com>
5 years agoAdd Honeycomb bridge domain management tests 69/969/7
selias [Fri, 29 Apr 2016 11:35:25 +0000 (13:35 +0200)]
Add Honeycomb bridge domain management tests

JIRA: CSIT-41, CSIT-42, CSIT-43

 - add test cases for Honecomb bridge domain management
 - add keywords used by these tests
 - prefix the names of Honeycomb test suites with numbers, so that they
   execute in a specific order
 - moved suite teardown with "Stop Honeycomb service" to bridge domain
   suite

Note: Due to time constraints, the Honeycomb service cannot be restarted
between executions of individual Honeycomb test suites. Therefore
Honeycomb will be setup in the first test suite and shut down after the
last one.

Change-Id: I36b7f8335e5ba54985da5bd83730cf856d5ab48b
Signed-off-by: selias <samelias@cisco.com>
5 years agoQEMU VM guest-ping fix 34/1034/2
Matus Fabian [Mon, 9 May 2016 07:17:44 +0000 (00:17 -0700)]
QEMU VM guest-ping fix

JIRA: CSIT-62

Change-Id: I0affda16af8f983895c50fbc306f45f30fbb69cd
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoInterface VxLAN test 57/957/8
Tibor Frank [Mon, 2 May 2016 16:19:04 +0000 (18:19 +0200)]
Interface VxLAN test

JIRA: CSIT-45

- add test: Honeycomb modifies interface configuration - VxLAN
- add keywords used in these tests to set and verify configuration
- add method and vat template for dumping an interface's configured VxLAN
- fixed requirements.txt

Change-Id: I6d6c360a4d478a0e40eecf7c0235969e7c06f7a9
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Signed-off-by: C.J. Collier <cjcollier@linuxfoundation.org>
5 years agoLF2 testbed topology file 54/954/5
Miroslav Miklus [Mon, 2 May 2016 15:25:46 +0000 (17:25 +0200)]
LF2 testbed topology file

Change-Id: I7c465a057640782219a8eb3b4c131d2d8edfcadc
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoAdd Honeycomb interface management tests (ipv4, ipv6, ethernet, routing) 30/930/13
selias [Fri, 29 Apr 2016 10:50:33 +0000 (12:50 +0200)]
Add Honeycomb interface management tests (ipv4, ipv6, ethernet, routing)

 - add test cases for managing ipv4, ipv6, ethernet and routing
 configuration on VPP interface
 - add keywords used in these tests to set and verify configuration
 - add method and vat template for dumping an interface's
configured IP addresses
 - add method for converting between ipv4 netmasks and
   network prefixes
 - clean up documentation of older Honeycomb keywords

Note: Test cases are tagged with a placeholder tag to allow manual
test runs locally. It should be replaced with the appropriate
environment tags once Honeycomb service is deployed on virl.

Change-Id: I99ce1e72c8550f520f8c35ef4a93eaebb530db78
Signed-off-by: selias <samelias@cisco.com>
Signed-off-by: Matej Klotton <mklotton@cisco.com>
5 years agoHoneycomb API keywords - interface, bridge domain 13/913/6
Tibor Frank [Thu, 28 Apr 2016 13:44:25 +0000 (15:44 +0200)]
Honeycomb API keywords - interface, bridge domain

- add keywords to manipulate data  using Honeycomb REST API to configure
  interfaces and bridge domains
- remove "add_vpp_to_honeycomb_network_topology" method from
  HoneycombSetup.py
- remove "parse_json_response" from HoneycombUtil.py
- add methods to manipulate data from Honeycomb REST API

Change-Id: I5e6f87097fe9bfccffa3d4aae21f63281353cf29
Signed-off-by: Tibor Frank <tifrank@cisco.com>
5 years agoAdd "show error" output into vpp stats 02/1002/2
Peter Mikus [Thu, 5 May 2016 13:27:08 +0000 (15:27 +0200)]
Add "show error" output into vpp stats

- Add "show error" debug output into VPP show stats as verbose
  subcommand is not parsed correctly in output of vpp_api_test

Change-Id: I924c54bb86375e43770fd449b8d6aaa7a3413bcb
Signed-off-by: Peter Mikus <pmikus@cisco.com>
5 years agoExtend testbed reservation wait time 90/990/2
Miroslav Miklus [Wed, 4 May 2016 15:07:53 +0000 (17:07 +0200)]
Extend testbed reservation wait time

Change-Id: If35705c1934dfb10f117f03ebbba111854f34360
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoVPP build update for lisp tests 04/1004/2
Jan Gelety [Thu, 5 May 2016 16:41:35 +0000 (18:41 +0200)]
VPP build update for lisp tests

- vpp build 437

Change-Id: I75cd51aaa721ada40e7e206a3d4637d238849ae6
Signed-off-by: Jan Gelety <jgelety@cisco.com>
5 years agoUpdate T-rex version to v2.00 62/862/17
pmikus [Mon, 25 Apr 2016 13:58:04 +0000 (15:58 +0200)]
Update T-rex version to v2.00

- JIRA: CSIT-14
- Update T-rex version to v2.00 with latest API
- Compatibility fixes of affected libraries and robot files
- T-rex performance tuning

Change-Id: I3a90553032d6d82e6e03df7e0b52e205d3f6ba4e
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoPatch to reset default VPP configuration in perf. 45/945/5
pmikus [Sun, 1 May 2016 16:28:59 +0000 (18:28 +0200)]
Patch to reset default VPP configuration in perf.

- JIRA: CSIT-57
- fix situation when vpp startup configuration is replaced by default
  with no interface configuration

Change-Id: Iea23dcc0707bc96627b99d4500268a8e01bfda39
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoTopo installation script fix 01/901/10
Miroslav Miklus [Wed, 27 Apr 2016 15:38:41 +0000 (17:38 +0200)]
Topo installation script fix

-more logging
-use single SSH instance
-dpkg --force-all
-update referece vpp version and reference short test threshold

Change-Id: Ie67633755f451dba9d6a3ee1fc431d2c3f3b506f
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoAdd vpp_create_loopback 10/910/3
Matus Fabian [Thu, 28 Apr 2016 12:41:32 +0000 (14:41 +0200)]
Add vpp_create_loopback

Change-Id: If934d7ec28f86b070a0cafa0ee56d0bcf4e2ac6d
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoInstallation of stable vpp version for csit-vpp-perf-hw-all jenkins job 32/832/8
Miroslav Miklus [Thu, 21 Apr 2016 13:09:55 +0000 (15:09 +0200)]
Installation of stable vpp version for csit-vpp-perf-hw-all jenkins job

JIRA: CSIT-5
- vpp installation for csit-vpp-verify-* jobs
- use single bootstrap for all verify-perf jobs

Change-Id: Ifb30a360daecc90446f53d20440e2723b01c6050
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoUpdate vpp version in bootstrap.sh 98/898/1
Jan Gelety [Wed, 27 Apr 2016 11:17:02 +0000 (13:17 +0200)]
Update vpp version in bootstrap.sh

- vpp version changed to 1.0.0-377

Change-Id: I776fc49a1af15824a4c55071eecfc80fb4824295
Signed-off-by: Jan Gelety <jgelety@cisco.com>
5 years agoAdd qinq through xconnect tests. 74/674/15
Patrik Hrnciar [Mon, 4 Apr 2016 11:09:09 +0000 (13:09 +0200)]
Add qinq through xconnect tests.

Change-Id: Ia20f5959dab41290bf60490ead830f0e37ecff0d
Signed-off-by: Patrik Hrnciar <phrnciar@cisco.com>
5 years agoAdd BD vpp-vm-vpp tests 23/823/7
Matus Fabian [Wed, 20 Apr 2016 09:49:36 +0000 (11:49 +0200)]
Add BD vpp-vm-vpp tests

JIRA: CSIT-20

Change-Id: Id22102519cdc8afef63c0e727fb47e2a2f1d769b
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoExtend host topology with NIC type filtering 08/808/12
Miroslav Miklus [Mon, 18 Apr 2016 15:15:40 +0000 (17:15 +0200)]
Extend host topology with NIC type filtering

JIRA: CSIT-1
Changes to allow filtering based on NIC model.
Switched xconnect perf test to use filtered topology.

Change-Id: Id526f47dc28f92bf26d070e54819ad29bccc0440
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoHoneycomb interface state management test 09/809/11
selias [Mon, 18 Apr 2016 16:21:20 +0000 (18:21 +0200)]
Honeycomb interface state management test

 - add simple test for interface management through Honeycomb API
 - add status code 500 to honeycomb startup check whitelist,
   due to Honeycomb changes in https://gerrit.fd.io/r/772
 - increase timeout value for Honeycomb startup check
 - modify Honeycomb setup keywords to allow starting Honeycomb
   on individual nodes
 - remove sanity test suite, tests now sorted by functional areas
   such as interfaces, bridge domains, VXlan,...

Change-Id: I87f9cf69577706f3b00e24a8d2c01df52fc707c6
Signed-off-by: selias <samuel.elias@pantheon.tech>
5 years agoAdd optional args to traffic script arg parser 80/880/2
Matus Fabian [Tue, 26 Apr 2016 08:58:52 +0000 (10:58 +0200)]
Add optional args to traffic script arg parser

Change-Id: I92e890e0c42da34e1e72a442f7a3bbe12fe9aef1
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoAdd IPsec utilities python library 83/783/6
Matus Fabian [Thu, 14 Apr 2016 11:04:55 +0000 (13:04 +0200)]
Add IPsec utilities python library

JIRA: CSIT-28

Change-Id: I9513f14a9920bfbdaf34c5cc5d4619d16a383ca2
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoUpdate of bootstrap-verify-master.sh file. 63/863/1
Jan Gelety [Mon, 25 Apr 2016 15:41:57 +0000 (17:41 +0200)]
Update of bootstrap-verify-master.sh file.

- VIRL image switched to double-ring-nested topology
- removed exclusion of 3_NODE_DOUBLE_LINK_TOPO test cases
- included test cases with tags:
  - VM_ENV and 3_NODE_SINGLE_LINK_TOPO
  - VM_ENV and 3_NODE_DOUBLE_LINK_TOPO

Change-Id: I99084b57c1fdd4aefb028cb144822a645baf9321
Signed-off-by: Jan Gelety <jgelety@cisco.com>
5 years agoHoneycomb API keywords 82/782/5
Tibor Frank [Thu, 14 Apr 2016 10:46:22 +0000 (12:46 +0200)]
Honeycomb API keywords

- add keywords to get, put post and delete configuration data using Honeycomb
  REST API
- add keywords to get operational data using Honeycomb REST API

Change-Id: If75221777e0e83a809c834addf82e95bbd540e21
Signed-off-by: Tibor Frank <tifrank@cisco.com>
5 years agoReformat python libraries. 07/607/11
Matej Klotton [Thu, 24 Mar 2016 15:14:20 +0000 (16:14 +0100)]
Reformat python libraries.

PEP8 reformat
fix typos
docstrings reformat

Change-Id: Ic48ba4e06490630808b8e2ab1ab0b046ec7eeed7
Signed-off-by: Matej Klotton <mklotton@cisco.com>
5 years agoBridge domain tests and user keywords refactor. 11/811/6
Jan Gelety [Mon, 18 Apr 2016 22:07:13 +0000 (00:07 +0200)]
Bridge domain tests and user keywords refactor.

- BD tests changed to behavior-driven-test style
- added documentation to BD tests
- added documentation and examples to BD user KWs

Change-Id: Ifad45df3ef0bb55f5295363a264fe5f4e54f5762
Signed-off-by: Jan Gelety <jgelety@cisco.com>
5 years agoVatExecutor update 42/842/2
Matus Fabian [Fri, 22 Apr 2016 11:58:22 +0000 (13:58 +0200)]
VatExecutor update

Strip non-JSON data from VAT terminal output
Add method for VAT command retval verification

Change-Id: Ibe68020393cb1a700ad048dc78078f71064707b4
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoPerformance test cases 26/826/5
pmikus [Wed, 20 Apr 2016 13:49:44 +0000 (15:49 +0200)]
Performance test cases

- JIRA: CSIT-10
- Add 1518B and 9000B test for multicore tests
- Fine tune threshold for short tests based on previous benchmarks
- Fix SI units

Change-Id: I5c4657dfd881a24b2e40c91ea80a979003754258
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoTopology schema 93/793/5
Tibor Frank [Fri, 15 Apr 2016 12:46:21 +0000 (14:46 +0200)]
Topology schema

- fix Honeycomb part of the topology schema
- add topology file checking using pykwalify before tests are started

Change-Id: I907d41b66f5660a2ab4de8857312e9277052bae6
Signed-off-by: Tibor Frank <tifrank@cisco.com>
5 years agoRemove horizontal whitespace (a line) before [Command_start_exec] tag. 16/816/2
mkonstan [Tue, 19 Apr 2016 16:17:42 +0000 (17:17 +0100)]
Remove horizontal whitespace (a line) before [Command_start_exec] tag.

Change-Id: I51e5616cb45b3c67e82153cc4c0f42ec9c5d75d7
Signed-off-by: mackonstan <mkonstan@cisco.com>
5 years agoInclude tags to CSIT performance boostrap script 91/791/2
pmikus [Fri, 15 Apr 2016 10:08:51 +0000 (12:08 +0200)]
Include tags to CSIT performance boostrap script

- include tags to CSIT performance boostrap script
- allows future expansion of selective test execution

Change-Id: I3870f52d86ff39234eb724500d3871fed0cbdb30
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoMake VXLAN test dot1Q as noncritical for now. 19/819/1
Stefan Kobza [Tue, 19 Apr 2016 20:53:23 +0000 (22:53 +0200)]
Make VXLAN test dot1Q as noncritical for now.

Change-Id: I0d41aad3fc1f96b58b83c6f0aa5a0e7606868ee4
Signed-off-by: Stefan Kobza <skobza@cisco.com>
5 years agoMake dut_setup.sh verbose; add sleep after vpp restart. 14/814/5
Stefan Kobza [Tue, 19 Apr 2016 12:37:20 +0000 (14:37 +0200)]
Make dut_setup.sh verbose; add sleep after vpp restart.

 - there's a race condition between VPP service restart and vpp_api_test
 execution; this patch addresses it with (for now) sleep after service
 vpp restart
 - above is just a workaround until VPP provides us with a notification
 of readiness..

Change-Id: I1f92c1d86e4f90c9432ea57c1747629f619f003e
Signed-off-by: Stefan Kobza <skobza@cisco.com>
5 years agoMulticore VPP setup for performance testing 10/710/12
pmikus [Thu, 7 Apr 2016 14:36:31 +0000 (16:36 +0200)]
Multicore VPP setup for performance testing

- add multithread TAGS documentation
- add methods to VppConfigGenerator for RSS configuration
- create KW for multithread setup
- create sample test case using multithread vpp setup
- add Documentation into TCs

Change-Id: Id40862490d49380dc76d1d3ce39314603f983fd3
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoUpdate Qemu library 77/677/9
Matus Fabian [Mon, 4 Apr 2016 14:11:46 +0000 (16:11 +0200)]
Update Qemu library

Incorporated changes from nested VM on VIRL.

Change-Id: I76ceeb2dde635c2cf0d3a53d29bf24fed72a5437
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoChange the performance result message 84/784/3
pmikus [Thu, 14 Apr 2016 11:30:12 +0000 (13:30 +0200)]
Change the performance result message

- change the result message of performance search to more verbose level
  (rate per stream, rate total, throughput in Gbps)

Change-Id: I7562d6d33c525654131c75ebcc776cba438f68f4
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoUpdate of bootstrap.sh file. 37/737/4
Jan Gelety [Mon, 11 Apr 2016 16:08:13 +0000 (18:08 +0200)]
Update of bootstrap.sh file.

- updated vpp build version
- VIRL image switched to double-ring-nested topology
- removed exclusion of 3_NODE_DOUBLE_LINK_TOPO test cases
- included test cases with tags:
  - VM_ENV and 3_NODE_SINGLE_LINK_TOPO
  - VM_ENV and 3_NODE_DOUBLE_LINK_TOPO
- added missing tags to TC VPP reports interfaces in Bridge
  Domain test suite

Change-Id: Ia6a7a54ef9ad6245835b2762c303935fe39f4079
Signed-off-by: Jan Gelety <jgelety@cisco.com>
5 years agoVXLAN test with dot1q tagging. 21/521/35
Matej Klotton [Fri, 11 Mar 2016 12:56:39 +0000 (13:56 +0100)]
VXLAN test with dot1q tagging.

Change-Id: I3dbd12983736e338d757c580570d91680aedd83f
Signed-off-by: Matej Klotton <mklotton@cisco.com>
5 years agoMove methods from topology.py to more appropriate place. 99/599/18
Matej Klotton [Wed, 23 Mar 2016 14:01:30 +0000 (15:01 +0100)]
Move methods from topology.py to more appropriate place.

Change-Id: I2612a9466095b3d79a4a9d938fcdbf9f93133f69
Signed-off-by: Matej Klotton <mklotton@cisco.com>
5 years agoHoneycomb setup and utils 89/689/9
Tibor [Wed, 6 Apr 2016 11:33:42 +0000 (13:33 +0200)]
Honeycomb setup and utils

- re-implement HTTPCodes as IntEnum rather then dictionary
- add methods to manipulate data using honeycomb - get, set, delete
- change the name of url file from vpp_version.url to oper_vpp_version.url
- improve checking of startup and shutdown state of honeycomb
- PEP8 fixes
- add docstrings in all modules and classes
- move logging to the lowest possible level
- improve logging in exceptions
- add method exec_command_sudo_log to resources.libraries.python.ssh module

Change-Id: I54e0c6b45313e3a3c11bafa475488ae2b1e605c2
Signed-off-by: Tibor Frank <tifrank@cisco.com>
5 years agodocumentation: long_xconnect test suite documentation 51/751/2
Miroslav Miklus [Tue, 12 Apr 2016 12:54:12 +0000 (14:54 +0200)]
documentation: long_xconnect test suite documentation

Generate testdoc by using:
 python -m robot.testdoc long_xconnect.robot perf_long_xconnect.html

Change-Id: I12d83de4abb1af4d94e6fd78d05c46018ee5d7d7
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>
5 years agoAdd keywords for Honeycomb tests 94/694/12
selias [Wed, 6 Apr 2016 14:49:15 +0000 (16:49 +0200)]
Add keywords for Honeycomb tests

 -add interface dump and bridge-domain dump to Vat command templates
 -add low level keywords to implement vat dump commands

The resulting data dumps will be used in Honeycomb testing,
to verify data retrieved through Honeycomb API.

Change-Id: I2a913eaf23aa13d7223576220681821007672812
Signed-off-by: selias <samuel.elias@pantheon.tech>
5 years agoBoostrap verify installation directory removal 50/750/1
pmikus [Tue, 12 Apr 2016 11:38:20 +0000 (13:38 +0200)]
Boostrap verify installation directory removal

- Boostrap verify installation directory removal

Change-Id: I091240f8a3f7e5074cdc2f069837963e4c8bd192
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoBootstrap verify performance fix 49/749/1
pmikus [Tue, 12 Apr 2016 10:34:02 +0000 (12:34 +0200)]
Bootstrap verify performance fix

- Bootstrap verify performance fix

Change-Id: I8f77c26aef805f5223d03e2a0bc4a97f88195ed8
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoBootstrap verify performance fix 48/748/1
pmikus [Tue, 12 Apr 2016 10:21:53 +0000 (12:21 +0200)]
Bootstrap verify performance fix

- fix installation directory create and remove

Change-Id: I0acd47b886e869958a2075889af3bbe166d7808e
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoBootstrap verify perfomance fix 45/745/2
pmikus [Tue, 12 Apr 2016 07:58:59 +0000 (09:58 +0200)]
Bootstrap verify perfomance fix

- fix test tags to lowercase

Change-Id: I6e01a677f15843105106300097b4d0b23e2cba5d
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoUpdate of topo_installation file 42/742/2
pmikus [Tue, 12 Apr 2016 05:27:27 +0000 (07:27 +0200)]
Update of topo_installation file

- deb package path fix

Change-Id: I94065581886c4d5b9f15ced18857ddfcde442153
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoFix cosmetic issues in VIRL topologies 23/723/3
Carsten Koester [Sun, 10 Apr 2016 15:51:18 +0000 (11:51 -0400)]
Fix cosmetic issues in VIRL topologies

Fix cosmetic issues in VIRL topologies:
- Replace double whitespace by single whitespace in XML
- Remove reserved but unused IP address allocations from VIRL config
- Modify "start-testcase" script to handle links without IP address

Change-Id: Ie447bd91172df61b30ebb8650b56bdfca0f25fd4
Signed-off-by: Carsten Koester <ckoester@cisco.com>
5 years agoAdd VIRL double-ring (load-balancing) topology with nested VM support 24/724/2
Carsten Koester [Sun, 10 Apr 2016 15:54:47 +0000 (11:54 -0400)]
Add VIRL double-ring (load-balancing) topology with nested VM support

Change-Id: Idcf173b1711076a5d7e4af8fe46e302b9eae4ff6
Signed-off-by: Carsten Koester <ckoester@cisco.com>
5 years agoCombined DropRateSearch 71/671/7
pmikus [Mon, 4 Apr 2016 09:10:12 +0000 (11:10 +0200)]
Combined DropRateSearch

- combined search algorithm using linear and binary search
- add KWs into performance library for binary and combined search
- remove short_bridge_domain_binary.robot as binary and combined search
  KWs are in library for optional use
- PEP8 fixes

Change-Id: I4a6c3c07c82db4f855fb93214b0532ee0c783e0a
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoAdd Vagrantfile for local testing. 23/523/11
Stefan Kobza [Sat, 5 Mar 2016 09:19:16 +0000 (10:19 +0100)]
Add Vagrantfile for local testing.

Vagrantfile contains 3 VMs as of now, 2 DUTs 1 TG, with these notes:
 - login is csit/csit
 - by default provision script installs all deb packages from the dir
    where Vagrantfile is
 - developed for, and only tested on vbox (someone can pick up vmware)
 - All nodes have 1 shared mgmt network: 192.168.255.0/24
 - hosts have these IP addresses in host-only network
    TG : 192.168.255.100
    DUT1 : 192.168.255.101
    DUT2 : 192.168.255.102
 - script created to download MAC address information
 - PCI addresses are always the same for vbox (not sure about vmware)

HOWTO (will create a wiki page once one is created for CSIT project):
 - copy Vagrantfile to separate dir on host
 - vagrant up --parallel
    sit-back-and-relax
 - from VM that has access to the same host-only network (192.168.255.0 above)
    - copy your ssh-key to csit@192.168.255.{101,102,250} using
        ssh-copy-id
    - cd ${csit_dir}
    - virtualenv & pip as in README
    - export PYTHONPATH=${csit_dir}
    - resources/tools/topology/update_topology.py -v -f
        -o topologies/available/vagrant_pci.yaml \
        topologies/available/vagrant.yaml
    - pybot -L TRACE \
        -v TOPOLOGY_PATH:topologies/available/vagrant_pci.yaml -s \
        "ipv4" tests
    - see tests results

Change-Id: Ic27626605a9c820bca977b38f4e8ca37d1504ff5
Signed-off-by: Stefan Kobza <skobza@cisco.com>
5 years agoAdd tag support to boostrap-verify-perf 76/676/5
pmikus [Mon, 4 Apr 2016 13:26:41 +0000 (15:26 +0200)]
Add tag support to boostrap-verify-perf

- support tags in vpp-verify-perfomance-* jenkins jobs

Change-Id: I7e3abfe9e552a3f2fa9ebf95df9b48bcbae3561a
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoAdd "Stop and Clear QEMU" KW 73/673/3
Matus Fabian [Mon, 4 Apr 2016 10:07:51 +0000 (12:07 +0200)]
Add "Stop and Clear QEMU" KW

Change-Id: Ib1b3907cd30c20c89283bd20e84ed8b829540ca4
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoSetup and check honeycomb on all DUTs 75/575/11
selias [Mon, 21 Mar 2016 12:06:49 +0000 (13:06 +0100)]
Setup and check honeycomb on all DUTs

- methods implementing HTTP requests (PUT,GET,POST,DELETE)
- methods for parsing HTTP responses
- methods for honeycomb setup on DUT
- updated constants.py
- keywords for honeycomb setup and communication
- simple honeycomb sanity test (not enabled for jenkins job runs)

Change-Id: I589f0ca56cc01072b92fe9363aed16a4098aee40
Signed-off-by: selias <samuel.elias@pantheon.tech>
5 years agoAdd new tag for VPP-VM testing enviroment 69/669/3
Matus Fabian [Mon, 4 Apr 2016 07:40:52 +0000 (09:40 +0200)]
Add new tag for VPP-VM testing enviroment

Change-Id: I2bdca4d8451bec8215b2d8f637809f8781e2e9d2
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoAdd VIRL topology files and template for nested virtualisation 65/665/2
Carsten Koester [Fri, 1 Apr 2016 23:14:37 +0000 (19:14 -0400)]
Add VIRL topology files and template for nested virtualisation

Change-Id: I57dbe4f179b3cfd3f51c9dec503ca45a5c0521fd
Signed-off-by: Carsten Koester <ckoester@cisco.com>
5 years agoChange send_ip_icmp recv timeout to 2sec 47/647/2
Matus Fabian [Wed, 30 Mar 2016 11:39:36 +0000 (13:39 +0200)]
Change send_ip_icmp recv timeout to 2sec

Traffic script send_ip_icmp recv timeout 10 sec is too long and execution
rather end with ssh timeout.

Change-Id: Id4df679799191fd03e2987f068f3ea9e7bf12fdf
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoAdd get_vhost_user_if_name_by_sock 52/652/2
Matus Fabian [Thu, 31 Mar 2016 07:46:39 +0000 (09:46 +0200)]
Add get_vhost_user_if_name_by_sock

Change-Id: I1d7f2cdfcf29763ce94417c98d363021f1fb27a1
Signed-off-by: Matus Fabian <matfabia@cisco.com>
5 years agoLong perf tests 74/574/7
pmikus [Mon, 21 Mar 2016 09:29:17 +0000 (10:29 +0100)]
Long perf tests

- added new long tests for ipv4 and xconnect
- modified long bridge domain test
- added 64B, 1518B and 9000B frame sizes

Change-Id: I88f75c375345f2d708df3fd24dd50d77bc63c495
Signed-off-by: pmikus <pmikus@cisco.com>
5 years agoBootstrap to test perf of proposed patch vpp-csit-hw-perf 31/631/4
pmikus [Tue, 29 Mar 2016 10:56:12 +0000 (12:56 +0200)]
Bootstrap to test perf of proposed patch vpp-csit-hw-perf

- Based on #570 Lab reservation tool
- Get the latest build of VPP
- Make reservation on testbed by using existing topology files
- Copy and install VPP image on all DUTs
- Run performance tests

Change-Id: I8034fd9c284cbe9a68e0c9116d0826965464b5f9
Signed-off-by: pmikus <pmikus@cisco.com>