changes which would decrease performance without a good reason.
Existing jobs
-`````````````
+~~~~~~~~~~~~~
VPP is the only project currently using such jobs.
They are not started automatically, must be triggered on demand.
based on various processors.
Their Gerrit triggers words are of the form "perftest-{node_arch}"
where the node_arch combinations currently supported are:
-2n-clx, 2n-dnv, 2n-skx, 2n-tx2, 2n-zn2, 3n-dnv, 3n-skx, 3n-tsh.
+2n-clx, 2n-tx2, 2n-zn2, 3n-tsh.
Test selection
---------------
+~~~~~~~~~~~~~~
..
TODO: Majority of this section is also useful for CSIT verify jobs. Move it somewhere.
to help users to select the minimal set of tests cases.
Verify cycles
-_____________
+`````````````
When Gerrit schedules multiple jobs to run for the same patch set,
it waits until all runs are complete.
After Gerrit is done waiting, it becames possible to trigger
the same job again.
-Example. User triggers one set of tests on 2n-skx and immediately
-also triggers other set of tests on 3n-skx. Then the user notices
-2n-skx run end early because of a typo in tag expression.
-When the user tries to re-trigger 2n-skx (with fixed tag expression),
+Example. User triggers one set of tests on 2n-icx and immediately
+also triggers other set of tests on 3n-icx. Then the user notices
+2n-icx run end early because of a typo in tag expression.
+When the user tries to re-trigger 2n-icx (with fixed tag expression),
that comment gets ignored by Jenkins.
-Only when 3n-skx job finishes, the user can trigger 2n-skx.
+Only when 3n-icx job finishes, the user can trigger 2n-icx.
One comment many jobs
-_____________________
+`````````````````````
In the past, the CSIT code which parses for perftest trigger comments
was buggy, which lead to bad behavior (as in selection all performance test,
to use just one trigger word per Gerrit comment, just to be safe.
Multiple test cases in run
-__________________________
+``````````````````````````
While Robot supports OR operator, it does not support parentheses,
so the OR operator is not very useful. It is recommended
to use space instead of OR operator.
Example template:
-perftest-2n-skx {tag_expression_1} {tag_expression_2}
+perftest-2n-icx {tag_expression_1} {tag_expression_2}
See below for more concrete examples.
Suite tags
-__________
+``````````
Traditionally, CSIT maintains broad Robot tags that can be used to select tests,
for details on existing tags, see
to a single test case within a suite.
Fully specified tag expressions
-_______________________________
+```````````````````````````````
Here is one template to select a single test case:
{test_type}AND{nic_model}AND{nic_driver}AND{cores}AND{frame_size}AND{suite_tag}
As there are more test cases than CSIT can periodically test,
it is possible to encounter an old test case that currently fails.
To avoid that, you can look at "job spec" files we use for periodic testing,
-for example `this one <https://github.com/FDio/csit/blob/master/docs/job_specs/report_iterative/2n-skx/vpp-mrr-00.md>`_.
+for example `this one <https://github.com/FDio/csit/blob/master/resources/job_specs/report_iterative/2n-icx/vpp-mrr-00.md>`_.
-..
- TODO: Explain why "periodic" job spec link lands at report_iterative.
Shortening triggers
-___________________
+```````````````````
Advanced users may use the following tricks to avoid writing long trigger comments.
which does not support RDMA core driver.
Complete example
-________________
+````````````````
A user wants to test a VPP change which may affect load balance whith bonding.
Searching tag documentation for "bonding" finds LBOND tag and its variants.
As not all NICs and testbeds offer enogh ports for 2 parallel DUT-DUT links,
the user looks at `testbed specifications <https://github.com/FDio/csit/tree/master/topologies/available>`_
-and finds that only x710 NIC on 3n-skx testbed matches the requirements.
+and finds that only xxv710 NIC on 3n-icx testbed matches the requirements.
Quick look into the suites confirm the smallest frame size is 64 bytes
(despite DOT1Q robot tag, as the encapsulation does not happen on TG-DUT links).
-It is ok to use just 1 physical core, as 3n-skx has hyperthreading enabled,
+It is ok to use just 1 physical core, as 3n-icx has hyperthreading enabled,
so VPP vswitch will use 2 worker threads.
The user decides the vswitch forwarding mode is not important
and both 1 and 2 parallel links.
After shortening, this is the trigger comment fianlly used:
-perftest-3n-skx mrrANDnic_intel-x710AND1cAND64bAND?lbvpplacp-dot1q-l2xcbase-eth-2vhostvr1024-1vm*NOTdrv_af_xdp
+perftest-3n-icx mrrANDnic_intel-x710AND1cAND64bAND?lbvpplacp-dot1q-l2xcbase-eth-2vhostvr1024-1vm*NOTdrv_af_xdp
Basic operation
-```````````````
+~~~~~~~~~~~~~~~
The job builds VPP .deb packages for both the patch under test
(called "current") and its parent patch (called "parent").
or if any test was declared a regression.
Temporary specifics
-```````````````````
+~~~~~~~~~~~~~~~~~~~
The Minimal Description Length analysis is performed by
CSIT code equivalent to jumpavg-0.1.3 library available on PyPI.
probability of false positives.
Console output
-``````````````
+~~~~~~~~~~~~~~
The following information as visible towards the end of Jenkins console output,
repeated for each analyzed test.
Code Review / csit.git / blobdiff