+Motivation
+~~~~~~~~~~
+
+Aside of per-release activities (release report), CSIT also provides testing
+that requires somewhat tight coupling to the latest (merged but not released)
+VPP code. Currently, HEAD of one project is run against somewhat older codebase
+of the other project. Definition of what is the older codebase to use
+is maintained by CSIT project. For older CSIT codebase, there are so-called
+"oper" branches. For older VPP codebase, CSIT master HEAD contains identifiers
+for "stable" VPP builds. Such older codebases are also used for verify jobs,
+where HEAD of the other project is replaced by the commit under review.
+
+One particular type of jobs useful for VPP development is trending jobs.
+They test latests VPP build with latest oper branch of CSIT,
+and analytics is applied to detect regressions in preformance.
+For this to work properly, VPP project needs a warning against breaking
+the assumptions the current oper branch makes about VPP behavior.
+In the past, the most frequent type of such breakage was API change.
+
+Earlier attempts to create a process to minimize breakage have focused
+on creating a new verify job for VPP (called api-crc job) that
+votes -1 on a change that affects CRC values for API messages CSIT uses.
+The list of messages and CRC values (multiple "collections" are allowed)
+is maintained in CSIT repository (in oper branch).
+The process was less explicit on how should CSIT project maintain such list.
+As CSIT was not willing to support two incpompatible API messages
+by the same codebase (commit), there were unavoidable windows
+where either trenging jobs, or CSIT verify jobs were failing.
+
+Practice showed that human (or infra) errors can create two kinds of breakages.
+Either the unavoidable short window gets long, affecting a trending job run
+or two, or the api-crc job starts giving -1 to innocent changes
+because oper branch went out of sync with VPP HEAD codebase.
+This second type of failure prevents any merges to VPP for a long time
+(12 hours is the typical time, give time zone differences).
+
+The current version of this document introduces two new requirements.
+Firstly, the api-crc job should not give false -1, under any
+(reasonable) circumstances. That means, if a VPP change
+(nor any of its unmerged ancestor commits) does not affect any CRC values
+for messages used by CSIT, -1 should only mean "rebase is needed",
+and rebasing to HEAD should result in +1 from the api-crc job.
+Secondly, no more than one VPP change is allowed to be processed
+(at the same time).
+
+Naming
+~~~~~~
+
+It is easier to define the process after chosing shorter names
+for notions that need long definition.
+
+Note: Everytime a single job is mentioned,
+in practice it can be a set of jobs covering parts of functionality.
+A "run" of the set of jobs passes only if each job within the set
+has been run (again) and passed.
+
+Jobs
+----
+
++ A *vpp verify* job: Any job run automatically, and voting on open VPP changes.
+ Some verify jobs compile and package VPP for target operating system
+ and processor architecture, the packages are NOT archived (currently).
+ They should be cached somewhere in future to speed up in downstream jobs,
+ but currently each such downstream job can clone and build.
+
++ The *api-crc* job: Quick verify job for VPP changes, that accesses
+ CSIT repository (checkout latest oper branch HEAD) to figure out
+ whether merging the change is safe from CSIT point of view.
+ Here, -1 means CSIT is not ready. +1 means CSIT looks to be ready
+ for the new CRC values, but there still may be failures on real tests.
+
++ A *trending* job: Any job that is started by timer and performs testing.
+ It checkouts CSIT latest oper branch HEAD, downloads the most recent
+ completely uploaded VPP package, and unconditionally runs the tests.
+ CRC checks are optional, ideally only written to console log
+ without otherwise affecting the test cases.
+
++ A *vpp-csit* job: A slower verify job for VPP changes, that accesses CSIT
+ repository and runs tests from the correct CSIT commit (chosen as in trending)
+ against the VPP (built from the VPP patch under review).
+ Vote -1 means there were test failures. +1 means no test failures, meaning
+ there either was no API change, or it was backward compatible.
+
++ A *csit-vpp* job: Verify job for open CSIT changes. Downloads the
+ (completely uploaded) VPP package marked as "stable", and runs a selection
+ of tests (from the CSIT patch under review).
+ Vote +1 means all tests have passed, so it is safe to merge
+ the patch under review.
+
++ A *patch-on-patch* job: Manually triggered non-voting job
+ for open CSIT changes. Compiles and packages from VPP source
+ (usually of an unmerged change). Then runs the same tests as csit-vpp job.
+ This job is used to prove the CSIT patch under review is supporting
+ the specified VPP code.
+ In practice, this can be a vpp-csit job started with CSIT_REF set.
+
++ A *manual verification* is done by a CSIT committer, locally executing steps
+ equivalent to the patch-on-patch job. This can to save time and resources.
+
+CRC Collections
+---------------
+
+Any commit in/for the CSIT repository contains a file (supported_crcs.yaml),
+which contains either one or two collections. A collection is a mapping
+that maps API message name to its CRC value.
+
+A collection name specifies which VPP build is this collection for.
+An API message name is present in a collection if and only if
+it is used by a test implementation (can be in different CSIT commit)
+targeted at the VPP build (pointed out by the collection name).
+
++ The *stable collection*: Usually required, listed first, has comments and name
+ pointing to the VPP build this CSIT commit marks as stable.
+ The stable collection is only missing in deactivating changes (see below)
+ when not mergeable yet.
+
++ The *active collection*: Optional, listed second, has comments and name
+ pointing to the VPP Gerrit (including patch set number)
+ the currently active API process is processing.
+ The patch set number part can be behind the actual Gerrit state.
+ This is safe, because api-crc job on the active API change will fail
+ if the older patch is no longer API-equivalent to the newer patch.
+
+Changes
+-------
+
++ An *API change*: The name for any Gerrit Change for VPP repository
+ that does not pass api-crc job right away, and needs this whole process.
+ This usually means .api files are edited, but a patch that affects
+ the way CRC values are computed is also an API change.
+
+ Full name could be VPP API Change, but as no CSIT change is named "API change"
+ (and this document does not talk about other FD.io or external projects),
+ "API change" is shorter.
+
+ TODO: Is there a magic incantation for Gerrit WebUI to search for API changes?
+ Open, -1 from api-crc job, +1 from other (non-csit) jobs.
+
++ A *blocked change*: The name for open Gerrit Change for VPP repository
+ that got -1 from some of voting verify jobs.
+
++ A *VPP-blocked change": A blocked change which got -1 from some "pure VPP"
+ verify job, meaning no CSIT code has been involved in the vote.
+ Example: "make test" fails.
+
+ VPP contributor is expected to fix the change, or VPP developers
+ are expected to found a cause in an earlier VPP change, and fix it.
+ No interaction with CSIT developers is necessary.
+
++ A *CSIT-blocked change*: A blocked change which is not VPP-blocked,
+ but does not pass some vpp-csit job.
+ To fix a CSIT-blocked change, an interaction with a CSIT committer
+ is usually necessary. Even if a VPP developer is experienced enough
+ to identify the cause of the failure, a merge to CSIT is usually needed
+ for a full fix.
+
+ This process does not specify what to do with CSIT-blocked changes
+ that are not also API changes.
+
++ A *candidate API change*: An API change that meets all requirements
+ to become active (see below). Currently, the requirements are:
+
+ + No -1 nor -2 from from any human reviewer.
+
+ + All verify jobs (except vpp-csit ones) pass.
+
+ + +1 from a VPP committer.
+
+ The reason is to avoid situations where an API change becomes active,
+ but the VPP committers are unwilling to merge it for some reason.
+
++ The *active API change*: The candidate API change currently being processed
+ by the API Flag Day Algorithm.
+ While many API changes can be candidates at the same time,
+ only one is allowed be active at a time.
+
++ The *activating change*: The name for a Gerrit Change for CSIT repository
+ that does not change the test code, but adds the active CRC collection.
+ Merge of the opening change (to latest CSIT oper branch) defines
+ which API change has become active.
+
++ The *deactivating change*: The name for Gerrit Change for CSIT repository
+ that only supports tests and CRC values for VPP with the active API change.
+ That implies the previously stable CRC collection is deleted,
+ and any edits to the test implementation are done here.
+
++ The *mergeable deactivating change*: The deactivating change with additional
+ requirements. Details on the requirements are listed in the next section.
+ Merging this change finishes the process for the active API change.
+
+It is possible for a single CSIT change to act both as a mergeable
+deactivating change for one API change, and as an activating change
+for another API change. As English lacks a good adjective for such a thing,
+this document does not name this change.
+When this documents says a change is activating or deactivating,
+it allows the possibility for the change to fullfill also other purposes
+(e.g. acting as deactivating / activating change for another API change).
+
+Algorithm Steps
+~~~~~~~~~~~~~~~
+
+The following steps describe the application of the API "Flag Day" algorithm:
+
+#. A VPP patch for an API change is submitted to
+ gerrit for review.
+#. The api-crc job detects the API CRC values have changed
+ for some messages used by CSIT.
+#. The api-crc job runs in parallel with any other vpp-csit verify job,
+ so those other jobs can hint at the impact on CSIT.
+ Currently, any such vpp-csit job is non-voting,
+ as the current process does not guarantee such jobs passes
+ when the API change is merged.
+#. If the api-crc job fails, an email with the appropriate reason
+ is sent to the VPP patch submitter and vpp-api-dev@lists.fd.io
+ including the VPP patch information and .api files that are edited.
+#. The VPP patch developer works with a VPP committer
+ to ensure the patch meets requirements to become a candidate (see above).