4 The VPP binary API is a message passing API. The VPP API language is
5 used to define a RPC interface between VPP and its control plane. The
6 API messages supports shared memory transport and Unix domain sockets
9 The wire format is essentially that of a network formatted (big-endian)
12 The VPP API compiler is located in *src/tools/vppapigen* and can
13 currently compile to JSON or C (used by the VPP binary itself).
21 There are 3 types of message exchanges:
23 - Request/Reply The client sends a request message and the server
24 replies with a single reply message. The convention is that the reply
25 message is named as method_name + \_reply.
27 - Dump/Detail The client sends a “bulk” request message to the server,
28 and the server replies with a set of detail messages. These messages
29 may be of different type. A dump/detail call must be enclosed in a
30 control ping block (Otherwise the client will not know the end of the
31 bulk transmission). The method name must end with method + “\_dump”,
32 the reply message should be named method + “\_details”. The exception
33 here is for the methods that return multiple message types
34 (e.g. sw_interface_dump). The Dump/Detail methods are typically used
35 for acquiring bulk information, like the complete FIB table.
37 - Events The client can register for getting asynchronous notifications
38 from the server. This is useful for getting interface state changes,
39 and so on. The method name for requesting notifications is
40 conventionally prefixed with “want\_”. E.g. “want_interface_events”.
41 Which notification types results from an event registration is
42 defined in the service definition.
44 A message from a client must include the ‘client_index’, an opaque
45 cookie identifying the sender, and a ‘context’ field to let the client
46 match request with reply.
48 An example of a message definition. The client sends the show_version
49 request, the server replies with the show_version_reply.
51 The *client_index* and *context* fields are required in all requests.
52 The *context* is returned by the server and is used by the client to
53 match up request and reply messages.
62 define show_version_reply
68 string build_date [32];
69 /* The final field can be a variable length argument */
70 string build_directory [];
73 The flags are not used by the clients, but have special meaning for some
74 of the tracing and debugging of the API. The *autoreply* flag is a
75 shorthand for a reply message with just a *retval* field.
79 define : DEFINE ID '{' block_statements_opt '}' ';'
80 define : flist DEFINE ID '{' block_statements_opt '}' ';'
88 block_statements_opt : block_statements
89 block_statements : block_statement
90 | block_statements block_statement
91 block_statement : declaration
93 declaration : type_specifier ID ';'
94 | type_specifier ID '[' ID '=' assignee ']' ';'
95 declaration : type_specifier ID '[' NUM ']' ';'
96 | type_specifier ID '[' ID ']' ';'
113 The *option* word is used to specify meta information. The only current
114 use is to specify a semantic version of the .api file itself.
120 option version = "1.0.0";
125 option : OPTION ID '=' assignee ';'
134 New user defined types are defined just like messages. A typedef has two
135 forms. It can either define an alias for a different type (or array).
141 typedef u8 ip4_address[4];
142 typedef u8 ip6_address[16];
144 Where the above defines two new types *vl_api_ip4_address_t* and
145 *vl_api_ip6_address_t*. These are aliases for the underlying u8 array.
147 In the other form, it is used to specify an abstract data type.
151 enum address_family {
156 union address_union {
157 vl_api_ip4_address_t ip4;
158 vl_api_ip6_address_t ip6;
162 vl_api_address_family_t af;
163 vl_api_address_union_t un;
166 Where the new type *vl_api_address_t*
170 typedef : TYPEDEF ID '{' block_statements_opt '}' ';'
171 typedef : TYPEDEF declaration
173 Importing Definitions
174 ~~~~~~~~~~~~~~~~~~~~~
176 You can use definitions from other .api files by importing them. To
177 import another .api’s definitions, you add an import statement to the
180 import “vnet/ip/ip_types.api”;
182 By default you can only use definitions from directly imported .api
185 The API compiler searches for imported files in a set of directories
186 specified on the API compiler command line using the –includedir flag.
190 import : IMPORT STRING_LITERAL ';'
195 The API language uses C style comments.
205 Enums are similar to enums in C.
207 Every enum definition must contain a constant that maps to zero as its
208 first element. This is because:
210 There must be a zero value, so that we can use 0 as a numeric default
211 value. The zero value needs to be the first element.
213 As in C, enums can be used as flags or just as numbers. The on-wire, and
214 in memory representation size of an enum can be specified. Not all
215 language bindings will support that. The default size is 4 (u32).
221 enum ip_neighbor_flags
223 IP_API_NEIGHBOR_FLAG_NONE = 0,
224 IP_API_NEIGHBOR_FLAG_STATIC = 0x1,
225 IP_API_NEIGHBOR_FLAG_NO_FIB_ENTRY = 0x2,
228 Which generates the vl_api_ip_neighbor_flags_t in the C binding. In
229 Python that is represented as an IntFlag object
230 VppEnum.vl_api_ip_neighbor_flags_t.
234 enum : ENUM ID '{' enum_statements '}' ';'
235 enum : ENUM ID ':' enum_size '{' enum_statements '}' ';'
239 enum_statements : enum_statement
240 | enum_statements enum_statement
241 enum_statement : ID '=' NUM ','
247 The service statement defines the relationship between messages. For
248 request/response and dump/details messages it ties the request with the
249 reply. For events, it specifies which events that can be received for a
250 given ``want_*`` call.
257 rpc want_interface_events returns want_interface_events_reply
258 events sw_interface_event;
261 Which states that the request want_interface_events returns a
262 want_interface_events_reply and if enabled the client will receive
263 sw_interface_event messages whenever interface states changes.
267 service : SERVICE '{' service_statements '}' ';'
268 service_statements : service_statement
269 | service_statements service_statement
270 service_statement : RPC ID RETURNS NULL ';'
271 | RPC ID RETURNS ID ';'
272 | RPC ID RETURNS STREAM ID ';'
273 | RPC ID RETURNS ID EVENTS event_list ';'
285 ========= ======== =============== ===========
286 .api type size C type Python type
287 ========= ======== =============== ===========
298 string variable vl_api_string_t str
299 ========= ======== =============== ===========
307 +--------------------+--------+-------------+-------------------------+
308 | .api type | size | C type | Python type |
309 +====================+========+=============+=========================+
310 | vl_api_address_t | 20 | vl_ap | ` |
311 | | | i_address_t | `<class 'ipaddress.IPv4 |
312 | | | | Address'> or <class 'ip |
313 | | | | address.IPv6Address'>`` |
314 +--------------------+--------+-------------+-------------------------+
315 | vl | 4 | vl_api_ip | ``<class 'ip |
316 | _api_ip4_address_t | | 4_address_t | address.IPv4Address'>`` |
317 +--------------------+--------+-------------+-------------------------+
318 | vl | 16 | vl_api_ip | ``<class 'ip |
319 | _api_ip6_address_t | | 6_address_t | address.IPv6Address'>`` |
320 +--------------------+--------+-------------+-------------------------+
321 | vl_api_prefix_t | 21 | vl_a | ` |
322 | | | pi_prefix_t | `<class 'ipaddress.IPv4 |
323 | | | | Network'> or <class 'ip |
324 | | | | address.IPv6Network'>`` |
325 +--------------------+--------+-------------+-------------------------+
326 | v | 5 | vl_api_i | ``<class 'ip |
327 | l_api_ip4_prefix_t | | p4_prefix_t | address.IPv4Network'>`` |
328 +--------------------+--------+-------------+-------------------------+
329 | v | 17 | vl_api_i | ``<class 'ip |
330 | l_api_ip6_prefix_t | | p6_prefix_t | address.IPv6Network'>`` |
331 +--------------------+--------+-------------+-------------------------+
332 | vl_api_ip4_add | 5 | vl_api_ip4 | ``<class 'ipad |
333 | ress_with_prefix_t | | _address_wi | dress.IPv4Interface'>`` |
334 | | | th_prefix_t | |
335 +--------------------+--------+-------------+-------------------------+
336 | vl_api_ip6_add | 17 | vl_api_ip6 | ``<class 'ipad |
337 | ress_with_prefix_t | | _address_wi | dress.IPv6Interface'>`` |
338 | | | th_prefix_t | |
339 +--------------------+--------+-------------+-------------------------+
341 vnet/ethernet/ethernet_types.api
342 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
344 +---------------------+------+---------------------+-------------------+
345 | .api type | size | C type | Python type |
346 +=====================+======+=====================+===================+
347 | ``vl_ | 6 | ``vl_ | ``class 'vpp_pa |
348 | api_mac_address_t`` | | api_mac_address_t`` | pi.MACAddress'>`` |
349 +---------------------+------+---------------------+-------------------+
351 vnet/interface_types.api
352 ^^^^^^^^^^^^^^^^^^^^^^^^
354 ======================== ==== ======================== ===========
355 .api type size C type Python type
356 ======================== ==== ======================== ===========
357 vl_api_interface_index_t 4 vl_api_interface_index_t int
358 ======================== ==== ======================== ===========
366 A byte string with a maximum length of 64:
372 Before the “string” type was added, text string were defined like this.
373 The implications of that was the user would have to know if the field
374 represented a \\0 ended C-string or a fixed length byte string. The wire
375 format of the ‘string’ type is a u32 length
377 An IPv4 or IPv6 address was previously defined like:
384 Which made it hard for language bindings to represent the address as
385 anything but a byte string. The new explicit address types are shown
391 The VPP API compiler currently has two output modules. One generating
392 JSON and one generating C header files that are directly used by the VPP
393 infrastructure and plugins.
395 The C/C++, Python, Go Lua, and Java language bindings are generated
396 based on the JSON files.
398 Future considerations
399 ~~~~~~~~~~~~~~~~~~~~~
401 - Generate C/C++ (vapi) client code directly from vppapigen
402 - Embed JSON definitions into the API server, so dynamic languages
403 can download them directly without going via the filesystem and JSON
412 To minimize the disruptions to the consumers of the VPP API, while permitting
413 the innovation for the VPP itself.
415 Historically, API changes in VPP master branch were allowed at any point in time
416 outside of a small window between the API freeze milestone and RC1 milestone.
417 The API changes on the throttle branches were not permitted at all. This model
418 proved workable, however all the production use cases ended up on throttle
419 branches, with a lot of forklift activity when it is the time to upgrade to the
422 This formally structured API change process harmonizes the behavior across all
423 the VPP branches, and allows more flexibility for the consumer, while permitting
424 the innovation in the VPP itself.
429 "If a user is running a VPP version N and does not use any deprecated APIs, they
430 should be able to simply upgrade the VPP to version N+1 and there should be no
433 In-Progress, Production and Deprecated APIs
434 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
436 This proposal adds a classification of stability of an API call:
438 - "In-Progress": APIs in the process of the development, experimentation, and
441 - "Production": tested as part of the "make test", considered stable for general
444 - "Deprecated": used as a flag on Production APIs which are slated to be
445 deprecated in the future release.
447 The "In-Progress" APIs or the APIs with the semantic version of 0.x.y are not
448 subject to any stability checks, thus the developers are free to introduce them,
449 modify their signatures, and as well remove them completely at will. The users
450 should not use the in-progress APIs without the interactions with its
451 maintainers, nor base the production code on those APIs. The goal of
452 "in-progress" APIs to allow rapid iteration and modifications to ensure the API
453 signature and function is stabilized. These API calls may be used for testing or
454 experimentation and prototyping.
456 When the maintainer is satisfied with the quality of the APIs, and ensures that
457 they are tested as part of the "Make test" runs, they can transition their
458 status to "Production".
460 The "Production" APIs can *NOT* be changed in any way that modifies their
461 representation on the wire and the signature (thus CRC). The only change that
462 they may incur is to be marked as "Deprecated". These are the APIs that the
463 downstream users can use for production purposes. They exist to fulfill a core
464 promise of this process: The "Deprecated" APIs are the "Production" APIs that
465 are about to be deleted. To ensure the above core promise is maintained, if the
466 API call was marked as deprecated at any point between RC1 of release N and RC1
467 of release N+1, it MUST NOT be deleted until the RC1 milestone of the
468 release N+2. The deprecated API SHOULD specify a replacement API - which MUST
469 be a Production API, so as not to decrease the level of stability.
472 The time interval between a commit that marks an API as deprecated and a commit
473 that deletes that API MUST be at least equal the time between the two subsequent
474 releases (currently 4 months).
477 Doing so allows a for a good heads-up to those who are using the
478 "one free upgrade" property to proactively catch and test the transition from
479 the deprecated APIs using the master.
482 Marking an API as deprecated just 1 day before RC1 branch pull and then deleting
483 that API one day after does *technically* satisfy "one free upgrade" promise,
484 but is rather hostile to the users that are proactively tracking it.
486 Semantic API Versioning
487 ~~~~~~~~~~~~~~~~~~~~~~~
489 VPP APIs use semantic versioning according to semver.org, with the compatibility
490 logic being applied at the moment the messages are marked as deprecated.
492 To discuss: i.e. if message_2 is being introduced which deprecates the
493 message_1, then that same commit should increase the major version of the API.
495 The 0.x.x API versions, by virtue of being in-progress, are exempt from this
501 See https://gerrit.fd.io/r/c/vpp/+/26881:
503 crcchecker.py is a tool to enforce the policy, with a few other bonus uses:
505 extras/scripts/crcchecker.py --check-patchset # returns -1 if backwards incompatible extras/scripts/crcchecker.py --dump-manifest extras/scripts/crcchecker.py --git-revision v20.01 <files> extras/scripts/crcchecker.py -- diff <oldfile> <newfile>
507 Notice that you can use this tool to get the list of API changes since a given past release.
515 1. Production APIs should never change.
516 The definition of a "production API" is if the major version in
517 the API file is > 0 that is not marked as "in-progress".
518 2. APIs that are experimental / not released are not checked.
519 An API message can be individually marked as in progress,
520 by adding the following in the API definition:
522 3. An API can be deprecated in three-to-six steps (the steps
523 with letters can be combined or split, depending on situation):
524 Step 1a: A new "in-progress" API new_api_2 is added that
525 is deemed to be a replacement.
526 Step 1b: The existing API is marked as "replaced_by" this new API:
527 option replaced_by="new_api_2";
528 Step 2a: The new_api_2 is marked as production by deleting its in-progress status,
529 provided that this API does have sufficient test coverage to deem it well tested.
530 Step 2b: the existing API is marked as "deprecated":
531 option deprecated="optional short message to humans reading it";
532 Step 3: the deprecated API is deleted.
534 There is a time constraint that the minimum interval between the steps 2 and 3
535 must be at least 4 months. The proposal is to have step 2 around a couple of
536 weeks before the F0 milestone for a release, as triggered by the release manager
537 (and in the future by an automated means).
542 Adding A New Field To A Production API
543 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
545 The simplest way to add a new field to a Production API message *foo_message* is
546 to create a new In-Progress message *foo_message_v2*, and add the field to that
547 one. Typically it will be an extension - so the API message handlers are
548 trivially chained. If there are changes/adjustments that are needed, this new
549 message can be freely altered without bothering the users of the Production API.
551 When the maintainer is happy with the quality of the implementation, and the
552 foo_message_v2 is tested in "make test" to the same extent as the foo_message,
553 they can make two commits: one, removing the in-progress status for
554 foo_message_v2, and the second one - deprecating foo_message and pointing the
555 foo_message_v2 as the replacement. Technically after the next throttle pull,
556 they can delete the foo_message - the deprecation and the replacement will be
557 already in the corresponding branch.
559 Rapid Experimentation For A New Feature
560 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
562 Add a message that is in-progress, and keep iterating with this message. This
563 message is not subject to the change control process.
565 An In-progress API Accidentally Marked As "production"
566 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
568 This is expected to mainly apply during the initial period of 20.05->20.09, the
569 proposal is to have it active for 4 weeks from Jun 17 till July 15th, with the
572 If a developer finds that a given API or a set of APIs is not ready for
573 production due to lack of tests and/or the general API stability, then they:
575 - Create a new gerrit change with *just* the marking of the API as
576 in_progress, subject being: "api: <feature> api message downgrade" and
577 a comment identifying which APIs are being downgraded and why.
579 - Add ayourtch@gmail.com or the current Release Manager as a reviewer --
580 for help in guiding the process and to ensure that the gerrit change is not
583 - Send an email to vpp-dev mailing list with the subject being the same as the
584 one-liner commit message, reference to the gerrit change, and the reasoning.
586 - Wait for the timeout period of two weeks for the feedback.
588 - If no feedback received, assume the community agreement and commit the
589 change to master branch.
591 This needs to be highlighted that this process is an *exception* - normally the
592 transition is always in_progress => production => deprecated.
597 https://gerrit.fd.io/r/q/+is:merged+message:%2522%255Eapi:.*%2524%2522