Merge "Get CSIT branch from build-root/scripts/csit-test-branch"

Merge "Add simple plot into csit/perf job"

Add common setup for vpp plugin verifies

Change-Id: Iac8b1d5408a9187fd39276881e21501c247c2231
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Fix issue in cleanup script

Change-Id: I06c1b51a423fac735767abf6c5f5ffdc602b0126
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Provide common cleanup script to be used across projects

Change-Id: Iebafbd25bee5cbf9c09fe0183b5d7e4c1eef8bb8
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Fix for nsh_sfc jjb

Change-Id: I871c8723dd443e4472e851b2ecda76f53cdd530d
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Initial nsh_sfc jjb

Change-Id: If77fdd4778eb54b3cba683a94b3cb84d1203fd00
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Get CSIT branch from build-root/scripts/csit-test-branch

If build-root/scripts/csit-test-branch exists, run
it to get the CSIT branch from which to run CSIT
tests.  Otherwise, default to csit-verified.

Related to patches vpp patches:
https://gerrit.fd.io/r/#/c/1343/ (master branch)
https://gerrit.fd.io/r/#/c/1345/ (stable/1606 branch)

Change-Id: I1146c322c1948076ead4639263f9b25e965009f2
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Revert "Turn on debugging"

This reverts commit cfaf0bf146c2c9e7598faf6c9efb044b8b7dba33.

https://gerrit.fd.io/r/#/c/1399/
As it bore no resemblance to the original gerrit submitted and currently has broken all of tldk verify.

Change-Id: If9914f8ade100b56c3a979adda1107f21b9c3c3f
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Add simple plot into csit/perf job

- Use plot plugin for 64B framesize, single core with NDR/PDR

Change-Id: Iea0697f31db3e4e0242c638bc1ebf0603081c8fd
Signed-off-by: pmikus <pmikus@cisco.com>

Turn on debugging

* sourcing a debug script if DEBUG is set
* echo command before executing (set -x)
* reduce duplicate code
* give more information about uptime

Change-Id: Ic8409ea3a819f2070eac01ac84e2b3a6fe975592
Signed-off-by: Ed Warnicke <eaw@cisco.com>
Signed-off-by: C.J. Collier <cjcollier@linuxfoundation.org>

Adding verify and merge jobs for tldk

Change-Id: I527e3b58038b20380fef03dcc4c8db7cced71658
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Use branching instead of tagging in csit-vpp-verify-weekly job

- branch name in format 'csit-verified-YYMMDD'

Change-Id: I87ee7d3af417f0c2d9022ed914ec978b96c973e2
Signed-off-by: Jan Gelety <jgelety@cisco.com>

Be less agressive with the set

Change-Id: I221374b6515e145f4fc5a31e3c30192732e6aeab
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Fix propogation of build errors.

Change-Id: I3e600ab74f5290008489cdbaa1624559067ad8b2
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Revert "Add timestamps to console logs for vpp verify jobs"

This reverts commit 233f45d3fc425cde64c2615c2f36025f61374451.
https://gerrit.fd.io/r/#/c/1282/

Change-Id: I7b65a3d60c17c832d88caa47a46a788984c358f0
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Merge "Sonar analysis added to Honeycomb jobs"

Sonar analysis added to Honeycomb jobs

The configuration is based on information from:
http://docs.openstack.org/infra/jenkins-job-builder/builders.html
and
http://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+Jenkins#AnalyzingwithSonarQubeScannerforJenkins-Job-levelConfiguration

This setting should be tested on the actual infrastructure.

Change-Id: I7416c84881d8cf36330f81318e178140dbce755b
Signed-off-by: Tibor Frank <tifrank@cisco.com>

Update of artifact directory in csit-vpp-master-verify-semiweekly job.

- artifact directory set to csit/archive
- artifacts set to *.* to keep all files we need
  in the archive direcotry

Change-Id: I3d862f06e3bb6deb6a208ba0e410299d99e8d4f6
Signed-off-by: Jan Gelety <jgelety@cisco.com>

Merge "Replace obsolete bootstrap file"

Replace obsolete bootstrap file

- JIRA: CIMANAGE-8
- Replace bootstrap-hw-tb2.sh with bootstrap-verify-perf.sh

Change-Id: I33ed6ff596faf46ec45e2483b0f60a6707d30056
Signed-off-by: Peter Mikus <pmikus@cisco.com>

Add timestamps to console logs for vpp verify jobs

Change-Id: I811483179ac6c80c8d90690ce672d3a49b4cdeae
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Merge "Switch to using {stream} for honeycomb"

Merge "Fix honeycomb integration to trigger from current vpp merge job"

Document the process of re-spinning an image

* Document the process of respinning jcloud images in README.txt
* Create /scripts/respin-jcloud-images.sh
* Create /vagrant/lib/respin-functions.sh
* Automate the process of fetching latest base images
* Automate the process of creating new "LF upload" snapshots
* Automate process of creating jcloud image
- Boot fresh VM
- run bootstrap.sh
- run system_reseal.sh
- snapshot re-sealed system
* Unable to automate updating jcloud image regex for staging nodes
* Correct shasum to sha1sum
* Add cloud-initramfs-* to PACKAGES list
* Install cloud-init during system reseal

Change-Id: Ie70d5f1fdee0e9f06810da6f37bfc710e3e06b8e
Signed-off-by: C.J. Collier <cjcollier@linuxfoundation.org>

Update ci merge job to use 4 threads

JJB 1.5.0 supports the --workers flag which allows us to tell it to use
multiple thread to update jobs.

Change-Id: I4626777990ca243d93a250e05d7791319eb20ac2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Simplify cfp jjb configuration

JJB 1.5.0 now allows CFP to use freeform IDs so lets use it to simplify
the yaml configuration.

Change-Id: Id64e772cb9fc1ed2f0deb11b6ba405f4ab43fe1e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Switch to using {stream} for honeycomb

Change-Id: Id91f825463bad7d8f48bb2efe1c5ac300e06a45e
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Fix honeycomb integration to trigger from current vpp merge job

Change-Id: I911e6f366b4831dc87523b6b0529353f233f0791
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Merge "CIMANAGE-7: csit-vpp-master-verify-semiweekly scheduled job"

csit-vpp-pylint: Adjust Violations plugin values

sunny weather: 0 - 10
stormy weather: 10 - 250
mark as unstable: 250+

Change-Id: Iac89bdcb74f2b8710597180340551386ef5cf3d8
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>

CIMANAGE-7: csit-vpp-master-verify-semiweekly scheduled job

- test vpp latest release from nexus with csit-verified test cases
- in case of success the vpp build will be used as vpp-master-verified
  build for csit-* jobs
- sheduled to run every Tuesday and Saturday at 04:00 UTC
- manual trigger possible too

- contains aslo correction of path of artifact in csit-vpp-verify-weekly job

Change-Id: Id5da3edb1b4579e6a3bd008694b9e5c754fd8729
Signed-off-by: Jan Gelety <jgelety@cisco.com>

Use correct server-name Primary

Perhaps 'Default' was the name at one point and then got renamed to
'Primary' and we forgot to update the configuration. Default is not
currently a valid server-name so resulted in Gerrit Trigger not watching
any valid Gerrit instance for the event-stream.

The current name of the Gerrit server-name is 'Primary' as defined in
Jenkins > Manage Jenkins > Gerrit Trigger.

Change-Id: Ic9481b53424d6fc8358131d384157779a7753fb0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Correct init script to use sysv on 14.04

condition wrapped around code

Change-Id: I27a426504fe5786e9a45d7145d9c3332b4542860
Signed-off-by: C.J. Collier <cjcollier@linuxfoundation.org>

Merge "Provide clarity as to when merge job fails are build or push"

Merge "Correct deprecated include syntax"

Provide clarity as to when merge job fails are build or push

Merge jobs can fail because a build has failed, or because
a push of artifacts has failed.  This should make
it clear which occurrred in the build logs.

Change-Id: I77e2ed0c756104492ee7a16378a5445162406f40
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Instrument Jenkins images/builds (take2)

Change-Id: I167cb259fce64f96d385294268a78bf6d161c8de
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Correct deprecated include syntax

* perl -i -pe 's/include-raw-escape /include-raw-escape: /' vpp.yaml

Change-Id: I62d6813e71b6f915524c395b3adc328698f755ce
Signed-off-by: C.J. Collier <cjcollier@linuxfoundation.org>

Instrument Jenkins images/builds

This patch instruments so we can be certain *what* bootstrap.sh
was used to build the running image and what
include-raw-vpp-build.sh was run for the Job

Change-Id: I69782c3bca71f24d24133205130be059a14a114c
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Add build dependencies for documentation

Add doxygen and graphviz to the dependency list.

Change-Id: I546b41a7613d8492f4054b3674905f05c504fad3
Signed-off-by: Chris Luke <chrisy@flirble.org>
Signed-off-by: C.J. Collier <cjcollier@linuxfoundation.org>

Disable things which may lock /var/lib/dpkg/lock

The ubuntu1604 verify job has been failing due
to locking of /var/lib/dpkg/lock:

https://lists.fd.io/pipermail/vpp-dev/2016-May/000992.html

This appears to only happen with the first job run against
a particular Jenkins slave, and upon using:

https://gerrit.fd.io/r/#/c/1121/

to probe it, two candidates emerge:

apt.systemd.daily

and

unattended-upgrades

So this patch removes unattended-upgrades

and

stops/disables apt.systemd.daily via systemd

Change-Id: Ic2bc9e4ec54ef1f2d0ab35813c4623a67a43f00a
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Add capacity for honeycomb to verify images.

Change-Id: Ie5fe2507b760b3f681be4291f9c52bf609dbfaca
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Fix for package pushx

Change-Id: Iebfcd2973bb319be4db64eecc4e35b218fb25b3b
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Fix reponames

Change-Id: I44c56b3c78deefdd91b1d9febfc015d9c2e927b2
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Merge "CIMANAGE-5: csit-vpp-verify-weekly scheduled job"

Enable jobs for throttle branches

Change-Id: I70cd178f52480bcab2bd84b7bdb967f1799acdbd
Signed-off-by: Ed Warnicke <eaw@cisco.com>

CIMANAGE-5: csit-vpp-verify-weekly scheduled job

- test csit master branch against verified vpp build
- in case of success apply csit-verified tag to csit test cases
- sheduled to run every Sunday at 04:00 UTC
- manual trigger possible too

Change-Id: I1db550f0770681991fa1be0fa72533637ba41257
Signed-off-by: Jan Gelety <jgelety@cisco.com>

Enable Serial Console

This patch will allow access to the bootloader and kernel boot
messages for debugging and maintenance

* add function deb_enable_serial_console to bootstrap-functions.sh
* call deb_enable_serial_console from bootstrap.sh

Change-Id: Ia79881a91ab4d10b1f5ed6deb3a43a7a4d037fa5
Signed-off-by: C.J. Collier <cjcollier@linuxfoundation.org>

Revert "Changes to accomodate vpp throttle branch."

This reverts commit 5d1d00330488903ae810de20bfd035fde9cdc25a.

This reverts gerrit: https://gerrit.fd.io/r/#/c/1074/

Change-Id: I5d937a4486ae7426debefdfaf85019508253c3f0
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Changes to accomodate vpp throttle branch.

Change-Id: If3ea89cbd6970a53bad8c13900c2eb8f15a7708c
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Check to see if a VPP patch needs rebasing

Change-Id: I9eca0b820a82dac4dbe58759a47f3294c6fcb14b
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Fix for javahome update

Change-Id: Ifbe5962e0161ccf223baa4bb4fc03f8c86d859b1
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Merge "Fix for java home for honeycomb"

Fix for java home for honeycomb

Change-Id: Ib4381c10763251105ca1fc418e70c0e06905ba9a
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Rename vpp-csit-verify-perf-{type} to vpp-csit-verify-hw-perf-{type}

The job name should not contain string from comment trigger.
If the job name contains the same string as the trigger, the verification
ends up in an endless loop because jenkins post comment with the job
name and that triggers the job again.

Change-Id: If25b8530984e047b96f4b571495ecedf586aa84f
Signed-off-by: Miroslav Miklus <mmiklus@cisco.com>

Merge changes from topics 'bootstrap-refactor', '994'

* changes:
  Re-factor vagrant bootstrap
  Sync bootstrap.sh with build.sh
  Cleanup vagrant

Unlink CSIT from zuul

Change-Id: If6ef9bb82f5d3fc3b78e699b2503006d5c2457e5
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Re-factor vagrant bootstrap

* created new file to hold functions
* moved sections of bootstrap in to functions
* called functions from case statement block
* integrate changes from Andrew Grimberg <agrimberg@linuxfoundation.org> (change 994)
* integrate changes from Ed Warnicke <eaw@cisco.com> (change 996)
* update debian package list after installing PPA
* synchronized redhat package list with VPP Makefile

Change-Id: I3d005c0fe01211c78c8b120b266964d873ee25b7
Signed-off-by: C.J. Adams-Collier <cjac@colliertech.org>

Remove csit-vpp-functional-nodepool job

Change-Id: I19356d04090a9db6fda294bc763142b739b42d46
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Sync bootstrap.sh with build.sh

Change-Id: I82fd236337ef939fd28b0495b8fa0e1516a754ac
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Cleanup vagrant

* Cleanup the Vagrantfile of unneeded cruft
* Cleanup the boostrap to be a bit more quiet and also give context as
  to what it's doing
* Make sure that the reseal has a sync / sleep

Change-Id: Ia1aa4e30b7a6342689c6fce30a18db1f52d27182
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Add jjb job to validate images

Change-Id: Ib1c29c39e845d3a30c4b82ef0b590f48df466a7b
Signed-off-by: Ed Warnicke <eaw@cisco.com>

re-index system certificates

* corrected truncation of PACKAGES file
* using headless variant of JDK
* removed -qq from apt-get commands to provide better accounting
* run update-ca-certificates -f to ensure that nexus.fd.io is trusted

Change-Id: I57f7267d6c70e40d63c86234e4e1723983170c3a
Signed-off-by: C.J. Collier <cjcollier@linuxfoundation.org>

Clean up apt package list

* Combine all packages in to a single call to apt-get install
* Modify /etc/apt/apt.conf
* corrected a comment
* not doing redundant edit of /etc/sysctl.conf
* adding emacs files to .gitignore
* Exercised this code while producing a new base image for each targetted platform
* install java8 from official openjdk PPA if platform is 14.04
* detect correct python package to install in CentOS
* open (and later close) stdout and stderr logs for later review

Change-Id: I669c0c3043f0c90df7af5033b8e180b0e6dc696a
Signed-off-by: C.J. Collier <cjcollier@linuxfoundation.org>

Merge "Use build.sh for vpp build"

VPP-8: use jdk8 for VPP builds

Change-Id: Ieec25be815128e53328493700d9a3d228bceba2b
Signed-off-by: Marek Gradzki <mgradzki@cisco.com>

Use build.sh for vpp build

https://gerrit.fd.io/r/#/c/765/
is refactoring the vagrant to have a separate
build.sh suitable for use outside of vagrant.
It autodetects supported distros, installs dependencies,
etc.

This patch switches vpp jenkins jobs over to using
build.sh.  This should give us a unified path where:

1) CI Builds
2) Vagrant
3) Users starting out on their own Centos/Ubuntu boxes
for the first time

are all using the same build.sh.

Change-Id: Iae7bca4e2a9e183e878d7638701e66735b388cbc
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Change performance job name

- JIRA: CIMANAGE-4
- Change vpp-verify-performance job name to naming convention

Change-Id: If873f29b3bb82de2bdf2e84614d132867c8d06c8
Signed-off-by: pmikus <pmikus@cisco.com>

fix for centos merge maven push

Change-Id: I16fd1580afa8fe229fc9ff15283f402333478fe2
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Fix push_deb

Change-Id: Ie33def177bca89a25f4d7a007603b3822ba7f6c5
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Enable ubuntu1604 merge and push debs to repo

Change-Id: I1d2ec5b4ac01f000fb180f1f82b6de2a6126f9e7
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Merge "Added python3-dev dependency to support Python API bindings."

Support for ubuntu1604 in CI verify

This patch should enable ubuntu1604 verify in CI management.

Change-Id: I17b6e8addcfddf6cbbc3774205beecc00c4d3fcf
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Added python3-dev dependency to support Python API bindings.

Change-Id: Ie7eb3d22e81fe718d688fbd1f5ffebb282e06e17
Signed-off-by: Ole Troan <ot@cisco.com>

Merge "Rename job to performance"

Merge "Correction of csit-vpp-master-verify-weekly jenkins job"

Correction of csit-vpp-master-verify-weekly jenkins job

- added archive artifact (RF log, report and output) files
  to publishers
- added manual trigger

Change-Id: I21b6798a357cb8b14371cee5f35ae259b63b384a
Signed-off-by: Jan Gelety <jgelety@cisco.com>

Add dependency on dh-systemd

Change-Id: I9a10dd107150ba31f13a00f7faabd0cc9fe2503f
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Rename job to performance

- rename csit-vpp-functional-hw-tb2 job to csit-vpp-perf-hw (as testing performance) with unified trigger for performance testing
- parameterized job for future extension

Change-Id: I72d1562fad8ad188930f39374f3ebfe8ebb984aa
Signed-off-by: pmikus <pmikus@cisco.com>

Query RPM files directly using rpm for upload info

Instead of trying to figure out all the maven fields based upon the
filename with possible failures that this can cause, instead, use the
rpm command itself to query the built artifacts for the needed
information.

Also, .src.rpm should be the canonical name for source RPM files. Make
sure that .srpm and .src.rpm are handled properly and they normalize to
.src.rpm in nexus.

Change-Id: I6b35c15e8e6b189a2144fe3cfb991e467933ad46
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Add 'mock' to allow cleaner rpmbuild

Change-Id: I800ae2b6be86ac54c259f952da9fb9b423a338f3
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Make rpm push handle SRPMs

* Fix the rpm push to search out and push SRPMs

* Fix the rpm push to actually push a file as it was referencing a bad
  variable

* Fix up some minor linting issues

Change-Id: I4d2e66ad52e593b71b969ab1b5faad254cc35e2d
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Fix virl scripts broken by {os}

Change-Id: I9f16666cf18753ce47d74fa21c125b1375407ecf
Signed-off-by: Ed Warnicke <eaw@cisco.com>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Have merge job push rpms to the yum repo

Change-Id: I0d16b75d8e8770894577659cd9b6ae0fc2140db9
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Turn on multi-os merge jobs

Change-Id: Iff99c3e8b8dadc6334b33fe5aff688102273fcb5
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Enabled centos7 build

Change-Id: I7fafacbe2fa58313e320de2e3a8a498898e89601
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Machinery for multi-os build.

This patch should provide machinery for multi os build
It also provides machinery for multi-branch build

Change-Id: I29db20f159f18af2cab512ab6df6b03d9f81ff15
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Split vpp-verify-perfomance job

- split job to include manual trigger keyword: verify-perf-long and
  verify-perf-short

Change-Id: Ie2048704313634cc9ea62ce67e80fc921bef2e18
Signed-off-by: pmikus <pmikus@cisco.com>

Merge "Make vpp-csit-verify-virl patch-triggered."

Make vpp-csit-verify-virl patch-triggered.

Change-Id: I90855a0b7a9a0e346a83beb42ada824bebd6eff6
Signed-off-by: Stefan Kobza <skobza@cisco.com>

VPP performance check job

- add job-template into jenkins layout view

Change-Id: I41a9e865713100b63913b9f5e4a3ea6d97bb3231
Signed-off-by: pmikus <pmikus@cisco.com>

VPP performance check job

- vpp-csit-hw-perf job to check performance of proposed patch
- build, copy, install, start perf tests

Change-Id: I2ae4cee916532c4d57715e2bd739d3109ab7ccf6
Signed-off-by: pmikus <pmikus@cisco.com>

Merge "Use tagging instead of branching for csit-verified"

Use tagging instead of branching for csit-verified

- use tags instead of branches in case of
  job "csit-vpp-master-verify-weekly"

Change-Id: Icbdcc2a57e1ab17d634ce3b8deedbc481f8f313c
Signed-off-by: Jan Gelety <jgelety@cisco.com>

Make robot framework output-path parametrizable.

csit-vpp-verify-virl job is blocked now, because output-path is not a
variable in robot-framework macro. This patch changes that.

Change-Id: I1c31342af6dfae66ce3e21e751470e476cc5be39
Signed-off-by: Stefan Kobza <skobza@cisco.com>

Fix rh_systems shell function

The rh_systems shell function was not properly defined. This corrects it
so that the bootstrapping can work

Change-Id: Ic5f03feac7deb5c8da54f3db4b0fb6a599ff0d85
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Add basebuild for centos

Change-Id: I8a8ba3fe1fd3fb12ed3f3004bbc863cb7b2c9872
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Refactor basebuild in prep to add Centos image

Change-Id: I9583a1e3a64c40460d5196ce28261230234ff80b
Signed-off-by: Ed Warnicke <eaw@cisco.com>

Update of the job 'vpp-csit-verify-virl'

Job is updated to be run manually and to use
csit-verified branch to test vpp.

Change-Id: I7a56594cb840c374bc849c095241655290e6b0bd
Signed-off-by: Jan Gelety <jgelety@cisco.com>