Merge "Update hst CPU/MemoryMB cloud config to limit jobs to 1 per server"

feat(CSIT): Add soak job

Signed-off-by: Peter Mikus <peter.mikus@protonmail.ch>
Change-Id: I47bb5849cc13065416a3c4521317f9a75e54004b

Update hst CPU/MemoryMB cloud config to limit jobs to 1 per server

Change-Id: Ic173bf2e6d053d87e703dddbefe850c7d4ee027a
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

Add HostStack Test nomad jenkins-config cloud configuration.

Change-Id: I5d95c1d02f0976a2ce875a904cb92ac5aa930371
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

Feat!: Upgrade Jenkins-job-builder to 6x

Jenkins Job Builder 6.0.0 is released and breaks backward
compatibility with 5.1.0.

The breaking changes with previous 5.1.0 release:

- Jobs are now expanded the same way as job templates.
- Macros without parameters are now expanded the same way as macros
  with parameters.
- Tags !include-raw: and !include-raw-escape: should now be used the
  same way in jobs and macros without parameters as they are used in
  job templates and macros with parameters.

Note: This breaks backward compatibility with older version of JJB
therefore care must be taken when upgrading the ci-man repositories.

Ref:
https://storyboard.openstack.org/#!/story/2010963
https://storyboard.openstack.org/#!/story/2010588
https://jira.linuxfoundation.org/browse/RELENG-5182

Change-Id: Iea1f78ae0806d75566c93b4ba4cb37cfb260ffdf
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Merge "Add stable/2402 branch and remove stable/2306 branch to docker executor image scripts"

Add stable/2402 branch and remove stable/2306 branch to docker executor image scripts

- Refactor csit_pip_cache function to use csit bash
  functions that activate the python virtualenv in
  production jobs.
- Remove stable/2306 branch due to pip issue
  https://github.com/pypa/pip/issues/11527
  causing extraneous downloading of pip wheels in
  CSIT jobs.
- Remove installation of packages for old CSIT docs
  generation.

Change-Id: Ib63aa654a35ad01fbfb5b7f593d3f8ffa6499b07
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

disable csit-verify-device voting until failure is resolved

Change-Id: Id8161bd0cb1d1e0bf3bcdd0758433ddbddb872cb
Signed-off-by: Naveen Joy <najoy@cisco.com>

Revert "Feat!: Upgrade Jenkins-job-builder to 6x"

This reverts commit cbe4988f0e9e9668919c0dd5cdd03d1e925ce93f.

Reason for revert: the change in jjb/global_macros.yaml replacing !include-raw with !include-raw-escape mangles the python code in the here_doc that gets written to /w/workspace/publish_library.py, thus all jobs fail during post job upload of log files to AWS s3 storage.

Change-Id: I8346ed9db9810f595614c28c114cc08170fdeb1a
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

Feat!: Upgrade Jenkins-job-builder to 6x

Jenkins Job Builder 6.0.0 is released and breaks backward
compatibility with 5.1.0.

The breaking changes with previous 5.1.0 release:

- Jobs are now expanded the same way as job templates.
- Macros without parameters are now expanded the same way as macros
  with parameters.
- Tags !include-raw: and !include-raw-escape: should now be used the
  same way in jobs and macros without parameters as they are used in
  job templates and macros with parameters.

Note: This breaks backward compatibility with older version of JJB
therefore care must be taken when upgrading the ci-man repositories.

Ref:
https://storyboard.openstack.org/#!/story/2010963
https://storyboard.openstack.org/#!/story/2010588

Issue: LF-JIRA RELENG-5182
Change-Id: Ib9acd0b9e7c264ac3531af858431e738397833b5
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Chore: Update common-packer to the latest v0.14.1

Symlink the required hcl templates to common-packer

Change-Id: I46db2277546a6e0bdedc8f9af0eb9bbbbdcb568c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

CSIT: Set the cadence of selected daily jobs to normal

- 3nb-spr

Change-Id: Iac397abdff6ae3d57e32382932ea270a016a6fe7
Signed-off-by: Tibor Frank <tifrank@cisco.com>

fix jjb/scripts/vpp/build.sh

Change-Id: I35843842a995931c1985971a606f1d085fd5aa4f
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

Merge "feat(device): unify voting for device jobs"

fix: add building static vppctl as part of the CI for VPP

Change-Id: I8b225b4677deb525ed1702d0c377ef58808a0881
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>

feat(device): unify voting for device jobs

The new 1n-alt is reliable, it should be voting both on CSIT and VPP.
The older 1n-spr is unreliable, should be running but not voting.
The oldest 1n-tx2 is no longer available, delete the remains.

Change-Id: I89e4cd574158711ad0a0c4ed5de9c42e38ce425c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

Merge "Chore: Upgrade global-jjb to v0.89.9"

CSIT: Set the cadence of selected daily jobs to normal

- all except 3nb-spr

Change-Id: I38f495166f73ecae69df17ecacfa60a7aa078765
Signed-off-by: Tibor Frank <tifrank@cisco.com>

Fix: Remove gerrit-verify workflow

The gerrit-verify workflow should not have been addd to this repo

Change-Id: I227a1fca7d01b40f9c57ad3ab5df57c53fac12bd
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Chore: Upgrade global-jjb to v0.89.9

Upgrading global-jjb to v0.89.9
Tested locally using xmllint and got no errors

Change-Id: I69ca884b6f2d402e50aca264b12e174deb76a89d
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

CSIT: Set the cadence of selected daily jobs to normal

- 2n-clx
- 2n-icx
- 2n-tx2
- 3n-icxd

Change-Id: I68cb17dfa8af7438bfc6a10965055c818ad46f8b
Signed-off-by: Tibor Frank <tifrank@cisco.com>

CSIT: Move aws jobs to Monday, remove 2n-spr mrr weekly

- 2n-spr mrr weekly never run (no job spec), no one is interested in
  it.

Change-Id: Ic29e596190307f7e79f48e34e5433780814c33ee
Signed-off-by: Tibor Frank <tifrank@cisco.com>

CSIT: Reduce daily jobs cadence for 2402 report data collection

Change-Id: I6d93994b398768b54f3ac6280fe63bfb8ccbb92f
Signed-off-by: Tibor Frank <tifrank@cisco.com>

CSIT: Enable 1n-alt

Change-Id: I7d4490116a5f513cfe2f0e043bbe12df7d295b62
Signed-off-by: Tibor Frank <tifrank@cisco.com>

Add VPP 24.02 branch

Change-Id: Idb04b5878f9089a70ff075499e1c6685e61f8533
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>

feat(csit): Disable 1n-tx2

Signed-off-by: Peter Mikus <peter.mikus@protonmail.ch>
Change-Id: I134d101a7a8a7e87e310b3121e3abdc732ea2d67

Merge "CSIT: Replace 2310 jobs by 2402 jobs"

feat(vpp): add a new job for bisecting

Change-Id: I8856e5cde4b85edd01a2c10af88823890c3afcb1
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

CI: Point git sources to gitolite mirror

Pointing back to the gitolite mirror to resolve fetch failures which
have increased since pointing to GitHub

Change-Id: I70312e266c32504c5c96e705a0934228cb882458
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

CI: Repoint git sources to GitHub mirror

The local gitolite mirror has been having semi-regular issues with
multiple jobs accessing it at the same time. As of November 28, 2023 all
repos are properly mirroring to GitHub so we'll use them as a mirror
instead.

Change-Id: I7cffe69bd819bb1b42ea33003d711caffeb96001
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Fix: Configure labels based on environment

Configuring the node labels based on sandbox or production to resolve
JCasC cron failures

Change-Id: I94bb55dc9f47b93bb0698e78b4e0266497eda4f2
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Revert "CI: Repoint git sources to GitHub mirror"

This reverts commit 85279e944f464abd5594a629f2975a636fa337ef.

Change-Id: I3c528f41a637631f5cc24a5e55b2df3f736d1113
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

CI: Repoint git sources to GitHub mirror

The local gitolite mirror has been having semi-regular issues with
multiple jobs accessing it at the same time. As of November 28, 2023 all
repos are properly mirroring to GitHub so we'll use them as a mirror
instead.

Change-Id: I03f6fc3e66b7ecab92c656b7d684d974c67593b3
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

feat(csit): 2n-c7g

Change-Id: Iefe63afc72b01e3ac4a110d683596d11de925d8d
Signed-off-by: Peter Mikus <peter.mikus@protonmail.ch>
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

CSIT: Replace 2310 jobs by 2402 jobs

Change-Id: Ib387592f17002888da091c7603d229ed85e83ef1
Signed-off-by: Tibor Frank <tifrank@cisco.com>

CSIT: Change the names of [semi]weekly jobs to keep the rules

- csit-vpp-device-semiweekly-{stream}-{os}-{executor-arch}-{node-arch}
- csit-vpp-device-weekly-{stream}-{os}-{executor-arch}-{node-arch}

Change-Id: Ic93c3077962c47ffe86c43448c53805986b6424b
Signed-off-by: Tibor Frank <tifrank@cisco.com>

chore(post-release-cleanup): clean up VPP CI jobs post VPP 23.10 release

- Remove references to unsupported VPP 23.02 release

Change-Id: I540a2f0d43ff2cdd29c8d62c729463282b76b8fb
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

Merge "CSIT: Set the cadence of selected daily jobs to normal"

CSIT: Set the cadence of selected daily jobs to normal

- 3nb-spr

Change-Id: I759a53033f5787c5e7244456655d4cbfb31a5359
Signed-off-by: Tibor Frank <tifrank@cisco.com>

feat(csit): 3n-icxd

Signed-off-by: Peter Mikus <peter.mikus@protonmail.ch>
Change-Id: Iba3ad49b245e42d1e6e76c1ee758484ab053ce87

CSIT: Set the cadence of selected daily jobs to normal

- 2n-zn2
- 3n-icx
- 3n-alt
- 3na-spr

Change-Id: I7f3b4de90037251fdc5143169b6373ffc038b659
Signed-off-by: Tibor Frank <tifrank@cisco.com>

CSIT: Set the cadence of selected daily jobs to normal

- 2n-spr
- 2n-tx2
- 3n-tsh

Change-Id: Ibaebcd43ff08059783da443e5abcccbbbf25f611
Signed-off-by: Tibor Frank <tifrank@cisco.com>

CSIT: Set the cadence of selected daily jobs to normal

- 2n-clx
- 2n-icx
- 3n-snr

Change-Id: I8d85a84cab527b0b4b8373507db03651c6564de0
Signed-off-by: Tibor Frank <tifrank@cisco.com>

Merge "fix(vpp): change vpp-cov-verify trigger to not match job name"

Chore: Upgrade to latest version of verify

Upgrade to the latest tagged version of ci-management GHA verify script

Change-Id: Iffaeab46c28d21cef0b0902f0def25b8db46284b
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

fix(vpp): change vpp-cov-verify trigger to not match job name

Change-Id: I1555473a3a76f96d0b648b268789f3a94b22e915
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

fix(core): revert docs upload back to aws_s3_bucket_object

- aws_s3_object fails due to missing MD5 Header checksum in PUT request

Change-Id: I73bda43c9b228fe6402cfc09bf31b5810af07344
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

fix(core): Pin terraform provider version

Signed-off-by: Peter Mikus <peter.mikus@protonmail.ch>
Change-Id: I6f243a8cf05c0d5242177f5ae8a4c99bbd83ebf3

fix(core): Terraform code adaptation for docs upload

Signed-off-by: Peter Mikus <peter.mikus@protonmail.ch>
Change-Id: Id1bdd18500877b79634a6c4df2574aa4d3ed1dee

Add stable/2310 branch to docker executor image scripts

- also clean up stale TODO's

Change-Id: I7106bcf8566560c3f85767afac1405d4a7c8d25a
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

CSIT: Reduce daily jobs cadence for 2310 report data collection

Change-Id: Iae8327f5ffbc92c385b0a83202adf1d10a7b2cc1
Signed-off-by: Tibor Frank <tifrank@cisco.com>

Add VPP 23.10 branch

Change-Id: Ic7b413c9eb2adf3d9581f9be0ec814bad91963b4
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>

Merge "feat(csit): Use only 1n-spr"

Fix: Add sandbox labels to nomad prod templates

Change-Id: Ieca16d264ef5479c8b5f952a3e3435b6f838755a
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

feat(csit): Use only 1n-spr

Signed-off-by: Peter Mikus <peter.mikus@protonmail.ch>
Change-Id: I96b885b5c5bff9ac53723f5e70b00c4fa3077bad

Fix: add prod labels to sandbox templates

Change-Id: Ib1769d612bc147fe602c881915f0e2059ff2a410
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

feat(csit): Add 1n-spr non-vote

Signed-off-by: Peter Mikus <peter.mikus@protonmail.ch>
Change-Id: Ic806aaeac91425742babb9903927cd477c90bc54

feat(vpp-csit-perf): add 3na-spr and 3nb-apr

Change-Id: I8ed7f1b14aa4d27016ad6e9c99ff2b6ac0a11778
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

vpp: disable voting on 1n-skx vpp-device job

Change-Id: I5b19af50e7b91b4d777a7eb23b5d6bd54c3a8c9a
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

Fix: Remove Jenkins URL

Removing the Jenkins URL, it's an invalid configuration
element

Change-Id: I8cc5ebb4d01bde1abc3f58d886421bbe185756c8
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Fix: typo in FDIONOMAD job template files

Change-Id: I95b03936c4af3383ae104476b6222a894e3961be
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

Merge "Fix: Adding Jenkins URL"

Fix: Adding Jenkins URL

Adding the Jenkins URL back to the environment configs

Change-Id: Ib7bf2f35009ba4b844995880ee99c826ae326b50
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Fix: Correct Jenkins URLs for sandbox nodes

Change-Id: If538ecdf4dc9c8ae22f684903eb9d468cf88525d
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Fix: Removing unneeded Artifact

RelativeDest is an optional configuration and not needed.

Change-Id: If08fe09a2228bc45a5dc585ad800d2b98f87ef97
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Fix: Modify the RelativeDest path

The RelativeDest is a default setting. The documentation points to
local/. This change is to align with the documentation.

Change-Id: I4ca1d7f476f13997ddb763c38a891662b0dc3727
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Fix: Correct prod Docker Registry

Change-Id: I9fb6802aaa8ba605886b19107dcb48541e26e847
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Fix: Correct node defaults and sandbox nodes

Change-Id: I342ec75bd3c7e8de2dc6a33435ade63ac2c92351
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Test: Minor nomad changes to check scripts updates

Change-Id: Ifd01d73286bc155cfbba0ed6c321f005f1ce9f92
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Fix: Remove old (and now bad) password

Password got into the repo accidentally. Getting it rotated and updated
in the correct encrypted location for automatic fill in.

Change-Id: Id19fed0697dc70cb72814e9768e686dfb89fcdb7
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Chore: Modify Nomad configs

Modifying Nomad configs with updates from the latest Nomad plugin

Change-Id: I06b34f3202d0ad511274a38b472239dd7a1ea537
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Merge "vpp: disable ccache for gcc builds"

vpp: disable ccache for gcc builds

- avoid build breakage for incompatibility between
  gcc & clang object files

Change-Id: I521863c5a651a87f06f23b6308a0f8ce91347feb
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

vpp 23.06 post release clean up.

- Remove vpp stable/2210 branch support

Change-Id: Icef62c0e073a787645fe1663d7ca7e0da0ba440b
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

Merge "vpp checkstyle: verify dpdk/rdma-core matrix"

Chore: Update global-jjb to the latest v0.88.5

- Pull in fix for sandbox cleanup job failure.

Change-Id: I6af59e3b722723883bf958267bfdeccb27ce252d
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

CSIT: Add rls2310 jobs.

- remove 2306 jobs

Change-Id: Ic0ce7017fcc8caa626056ce3404bfce8fb61aa0f
Signed-off-by: Tibor Frank <tifrank@cisco.com>

vpp checkstyle: verify dpdk/rdma-core matrix

- If mlx_rdma_dpdk_matrix.txt file has been updated
  in the current changeset, verify the current
  rdma-core_version and dpdk_version exist in
  the matrix file.

Change-Id: Ib7bf2969c48badb6795de440a858796c83ad1083
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

CI: Deprecate VSAP builds

This repo is archived as per IT-25893

Change-Id: Iee367fe742b2b996fd6efe10077a936025fdc22d
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>

feat(csit): Add new AWS jobs

Signed-off-by: Peter Mikus <peter.mikus@protonmail.ch>
Change-Id: I73bce1e5d84a5ca034ed5282c86886286a4d64e8

CI: Add cloud-env in hcl format for packer jobs

Change-Id: I44fc1ea43f77b19b9520cbaea04e893126b57648
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Chore: update global-jjb to the latest v0.86.9

Change-Id: Ifef15990a9e48748e31a31574c196941f27333a7
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Restore vpp-gcc-verify* job

- otherwise gcc build breakage will kill vpp-cov daily job.

Change-Id: I72effa7725d2e0cd3fadcfc6a359f24f372ba80c
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

CSIT: Set the cadence of selected daily jobs to normal

- 2n-spr

Change-Id: Ic71e858c500faa365be4f6cc4efb7fdef93ca560
Signed-off-by: Tibor Frank <tifrank@cisco.com>

Move vpp-cov-* CI job from per-patch to daily execution

Change-Id: I2798fafcc3182a735f6fd380ef558c585fd28b64
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

Chore: Update packer version to 1.8.2

v1.4.0 is outdated and the min required version is v1.8.2
for a while with common-packer.

Change-Id: Ie5b021b7977b7e2c2f51b132369761bbb41e84d0
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

CSIT: Set the cadence of selected daily jobs to normal

- 3n-snr
- 3nb-spr

Change-Id: Ic8dc98cd301bf50fecc9c8507089218828827a67
Signed-off-by: Tibor Frank <tifrank@cisco.com>

CSIT: Set the cadence of selected daily jobs to normal

- 3n-alt
- 3n-icx
- 3na-spr

Change-Id: I14139491c667389c7eacd7b0ccad0732d53bfe0f
Signed-off-by: Tibor Frank <tifrank@cisco.com>

Chore: Add packagecloud-account managed file

Change-Id: I217c3d2bcb45c1cc8a3fbe380a4e5108c9633f1d
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Merge "docker: add support for vpp stable/2306 branch"

Feat: Configure self-service-release

Configuring self-service-release features for PackageCloud

Change-Id: I030679257694e0d4ed49b8a327cecd3ffd98cd16
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Chore: Configure Sigul for self-service releases

Change-Id: I3a430eff4f202b415713f97df725e0f9386daf28
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Merge "Replace vpp-gcc job with vpp-cov job per-patch"

CSIT: Set the cadence of selected daily jobs to normal

- 2n-clx
- 2n-icx
- 2n-tx2
- 2n-zn2
- 3n-tsh

Change-Id: Ie3f8640c1443cdd245cafca6937df8b16e6ab667
Signed-off-by: Tibor Frank <tifrank@cisco.com>

Replace vpp-gcc job with vpp-cov job per-patch

- the original gcc job was intended to ensure that gcc
  builds worked to generate test coverage reports.  The
  vpp-cov-verify job produces test coverage reports
  per patch and stores them in the S3 7-day docs
  retention bucket.

Change-Id: I28c7c323ea8d673cdee66b85189e20029e63336d
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>

Chore: Add global vars for self-service-releases

Change-Id: I1acd3bdb1bfb66fc9b784e0d5ff697736e8f13bb
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

Chore: Update Ubuntu builders

Change-Id: I089e641405901a0dc24052ab3a117d4d6d03327f
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

CI: Update verify to use reusable workflow

Change-Id: I84b9cd69919c200dc2a4787ef0cdf9a39caebd40
Signed-off-by: Vanessa Valderrama <vvalderrama@linuxfoundation.org>

CI: Add pre-commit bits

Change-Id: Ie207583b16bda0f1851dcf31e6dfae386a59945c
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

CSIT: Reduce daily jobs cadence for 2306 report data collection

Change-Id: Ibe7171ceca7926e53198098ca51910d1a9404f2a
Signed-off-by: Tibor Frank <tifrank@cisco.com>