Typically, the user header is not present. User headers allow for other
data structures to be built atop vppinfra vectors. Users may specify the
-alignment for data elements via the [vec]()\*\_aligned macros.
+alignment for first data element of a vector via the \[vec\]()\*\_aligned
+macros.
-Vectors elements can be any C type e.g. (int, double, struct bar). This
+Vector elements can be any C type e.g. (int, double, struct bar). This
is also true for data types built atop vectors (e.g. heap, pool, etc.).
-Many macros have \_a variants supporting alignment of vector data and
-\_h variants supporting non-zero-length vector headers. The \_ha
-variants support both.
+Many macros have \_a variants supporting alignment of vector elements
+and \_h variants supporting non-zero-length vector headers. The \_ha
+variants support both. Additionally cacheline alignment within a
+vector element structure can be specified using the
+\[CLIB_CACHE_LINE_ALIGN_MARK\]() macro.
Inconsistent usage of header and/or alignment related macro variants
will cause delayed, confusing failures.
vector element as the second argument to hash\_set\_mem. It is perfectly
fine to memorize constant string addresses in the text segment.
+Timekeeping
+-----------
+
+Vppinfra includes high-precision, low-cost timing services. The
+datatype clib_time_t and associated functions reside in
+./src/vppinfra/time.\[ch\]. Call clib_time_init (clib_time_t \*cp) to
+initialize the clib_time_t object.
+
+Clib_time_init(...) can use a variety of different ways to establish
+the hardware clock frequency. At the end of the day, vppinfra
+timekeeping takes the attitude that the operating system's clock is
+the closest thing to a gold standard it has handy.
+
+When properly configured, NTP maintains kernel clock synchronization
+with a highly accurate off-premises reference clock. Notwithstanding
+network propagation delays, a synchronized NTP client will keep the
+kernel clock accurate to within 50ms or so.
+
+Why should one care? Simply put, oscillators used to generate CPU
+ticks aren't super accurate. They work pretty well, but a 0.1% error
+wouldn't be out of the question. That's a minute and a half's worth of
+error in 1 day. The error changes constantly, due to temperature
+variation, and a host of other physical factors.
+
+It's far too expensive to use system calls for timing, so we're left
+with the problem of continously adjusting our view of the CPU tick
+register's clocks_per_second parameter.
+
+The clock rate adjustment algorithm measures the number of cpu ticks
+and the "gold standard" reference time across an interval of
+approximately 16 seconds. We calculate clocks_per_second for the
+interval: use rdtsc (on x86_64) and a system call to get the latest
+cpu tick count and the kernel's latest nanosecond timestamp. We
+subtract the previous interval end values, and use exponential
+smoothing to merge the new clock rate sample into the clocks_per_second
+parameter.
+
+As of this writing, we maintain the clock rate by way of the following
+first-order differential equation:
+
+
+```
+ clocks_per_second(t) = clocks_per_second(t-1) * K + sample_cps(t)*(1-K)
+ where K = e**(-1.0/3.75);
+```
+
+This yields a per observation "half-life" of 1 minute. Empirically,
+the clock rate converges within 5 minutes, and appears to maintain
+near-perfect agreement with the kernel clock in the face of ongoing
+NTP time adjustments.
+
+See ./src/vppinfra/time.c:clib_time_verify_frequency(...) to look at
+the rate adjustment algorithm. The code rejects frequency samples
+corresponding to the sort of adjustment which might occur if someone
+changes the gold standard kernel clock by several seconds.
+
+
Format
------
```c
vec_add1 (result, 0)
- or
- result = format (result, "<whatever>%c", 0);
+ or
+ result = format (result, "<whatever>%c", 0);
```
Remember to vec\_free() the result if appropriate. Be careful not to
format\_junk() can invoke other user-format functions if desired. The
programmer shoulders responsibility for argument type-checking. It is
-typical for user format functions to blow up if the va\_arg(va,
-type) macros don't match the caller's idea of reality.
+typical for user format functions to blow up spectacularly if the
+va\_arg(va, type) macros don't match the caller's idea of reality.
Unformat
--------
Then loop parsing individual elements:
```c
- while (unformat_check_input (&input) != UNFORMAT_END_OF_INPUT)
+ while (unformat_check_input (&input) != UNFORMAT_END_OF_INPUT)
{
if (unformat (&input, "value1 %d", &value1))
;/* unformat sets value1 */
else if (unformat (&input, "value2 %d", &value2)
;/* unformat sets value2 */
else
- return clib_error_return (0, "unknown input '%U'",
+ return clib_error_return (0, "unknown input '%U'",
format_unformat_error, input);
}
```
As with format, unformat implements a user-unformat function capability
-via a "%U" user unformat function scheme.
+via a "%U" user unformat function scheme. Generally, one can trivially
+transform "format (s, "foo %d", foo) -> "unformat (input, "foo %d", &foo)".
-Vppinfra errors and warnings
-----------------------------
+Unformat implements a couple of handy non-scanf-like format specifiers:
-Many functions within the vpp dataplane have return-values of type
-clib\_error\_t \*. Clib\_error\_t's are arbitrary strings with a bit of
-metadata \[fatal, warning\] and are easy to announce. Returning a NULL
-clib\_error\_t \* indicates "A-OK, no error."
+```c
+ unformat (input, "enable %=", &enable, 1 /* defaults to 1 */);
+ unformat (input, "bitzero %|", &mask, (1<<0));
+ unformat (input, "bitone %|", &mask, (1<<1));
+ <etc>
+```
-Clib\_warning(format-args) is a handy way to add debugging
-output; clib warnings prepend function:line info to unambiguously locate
-the message source. Clib\_unix\_warning() adds perror()-style Linux
-system-call information. In production images, clib\_warnings result in
-syslog entries.
+The phrase "enable %=" means "set the supplied variable to the default
+value" if unformat parses the "enable" keyword all by itself. If
+unformat parses "enable 123" set the supplied variable to 123.
-Serialization
--------------
+We could clean up a number of hand-rolled "verbose" + "verbose %d"
+argument parsing codes using "%=".
-Vppinfra serialization support allows the programmer to easily serialize
-and unserialize complex data structures.
+The phrase "bitzero %|" means "set the specified bit in the supplied
+bitmask" if unformat parses "bitzero". Although it looks like it could
+be fairly handy, it's very lightly used in the code base.
-The underlying primitive serialize/unserialize functions use network
-byte-order, so there are no structural issues serializing on a
-little-endian host and unserializing on a big-endian host.
-
-Event-logger, graphical event log viewer
-----------------------------------------
-
-The vppinfra event logger provides very lightweight (sub-100ns)
-precisely time-stamped event-logging services. See
-./src/vppinfra/{elog.c, elog.h}
+### How to parse a single input line
-Serialization support makes it easy to save and ultimately to combine a
-set of event logs. In a distributed system running NTP over a local LAN,
-we find that event logs collected from multiple system elements can be
-combined with a temporal uncertainty no worse than 50us.
+Debug CLI command functions MUST NOT accidentally consume input
+belonging to other debug CLI commands. Otherwise, it's impossible to
+script a set of debug CLI commands which "work fine" when issued one
+at a time.
-A typical event definition and logging call looks like this:
+This bit of code is NOT correct:
```c
- ELOG_TYPE_DECLARE (e) =
+ /* Eats script input NOT beloging to it, and chokes! */
+ while (unformat_check_input (input) != UNFORMAT_END_OF_INPUT)
{
- .format = "tx-msg: stream %d local seq %d attempt %d",
- .format_args = "i4i4i4",
- };
- struct { u32 stream_id, local_sequence, retry_count; } * ed;
- ed = ELOG_DATA (m->elog_main, e);
- ed->stream_id = stream_id;
- ed->local_sequence = local_sequence;
- ed->retry_count = retry_count;
+ if (unformat (input, ...))
+ ;
+ else if (unformat (input, ...))
+ ;
+ else
+ return clib_error_return (0, "parse error: '%U'",
+ format_unformat_error, input);
+ }
+ }
```
-The ELOG\_DATA macro returns a pointer to 20 bytes worth of arbitrary
-event data, to be formatted (offline, not at runtime) as described by
-format\_args. Aside from obvious integer formats, the CLIB event logger
-provides a couple of interesting additions. The "t4" format
-pretty-prints enumerated values:
+When executed as part of a script, such a function will return "parse
+error: '<next-command-text>'" every time, unless it happens to be the
+last command in the script.
+
+Instead, use "unformat_line_input" to consume the rest of a line's
+worth of input - everything past the path specified in the
+VLIB_CLI_COMMAND declaration.
+
+For example, unformat_line_input with "my_command" set up as shown
+below and user input "my path is clear" will produce an
+unformat_input_t that contains "is clear".
```c
- ELOG_TYPE_DECLARE (e) =
- {
- .format = "get_or_create: %s",
- .format_args = "t4",
- .n_enum_strings = 2,
- .enum_strings = { "old", "new", },
+ VLIB_CLI_COMMAND (...) = {
+ .path = "my path",
};
```
-The "t" format specifier indicates that the corresponding datum is an
-index in the event's set of enumerated strings, as shown in the previous
-event type definition.
-
-The “T” format specifier indicates that the corresponding datum is an
-index in the event log’s string heap. This allows the programmer to emit
-arbitrary formatted strings. One often combines this facility with a
-hash table to keep the event-log string heap from growing arbitrarily
-large.
-
-Noting the 20-octet limit per-log-entry data field, the event log
-formatter supports arbitrary combinations of these data types. As in:
-the ".format" field may contain one or more instances of the following:
-
-- i1 - 8-bit unsigned integer
-- i2 - 16-bit unsigned integer
-- i4 - 32-bit unsigned integer
-- i8 - 64-bit unsigned integer
-- f4 - float
-- f8 - double
-- s - NULL-terminated string - be careful
-- sN - N-byte character array
-- t1,2,4 - per-event enumeration ID
-- T4 - Event-log string table offset
-
-The vpp engine event log is thread-safe, and is shared by all threads.
-Take care not to serialize the computation. Although the event-logger is
-about as fast as practicable, it's not appropriate for per-packet use in
-hard-core data plane code. It's most appropriate for capturing rare
-events - link up-down events, specific control-plane events and so
-forth.
-
-The vpp engine has several debug CLI commands for manipulating its event
-log:
+Here's a bit of code which shows the required mechanics, in full:
-```
- vpp# event-logger clear
- vpp# event-logger save <filename> # for security, writes into /tmp/<filename>.
- # <filename> must not contain '.' or '/' characters
- vpp# show event-logger [all] [<nnn>] # display the event log
- # by default, the last 250 entries
-```
+```c
+ static clib_error_t *
+ my_command_fn (vlib_main_t * vm,
+ unformat_input_t * input,
+ vlib_cli_command_t * cmd)
+ {
+ unformat_input_t _line_input, *line_input = &_line_input;
+ u32 this, that;
+ clib_error_t *error = 0;
+
+ if (!unformat_user (input, unformat_line_input, line_input))
+ return 0;
+
+ /*
+ * Here, UNFORMAT_END_OF_INPUT is at the end of the line we consumed,
+ * not at the end of the script...
+ */
+ while (unformat_check_input (line_input) != UNFORMAT_END_OF_INPUT)
+ {
+ if (unformat (line_input, "this %u", &this))
+ ;
+ else if (unformat (line_input, "that %u", &that))
+ ;
+ else
+ {
+ error = clib_error_return (0, "parse error: '%U'",
+ format_unformat_error, line_input);
+ goto done;
+ }
+ }
+
+ <do something based on "this" and "that", etc>
+
+ done:
+ unformat_free (line_input);
+ return error;
+ }
+ /* *INDENT-OFF* */
+ VLIB_CLI_COMMAND (my_command, static) = {
+ .path = "my path",
+ .function = my_command_fn",
+ };
+ /* *INDENT-ON* */
-The event log defaults to 128K entries. The command-line argument "...
-vlib { elog-events nnn } ..." configures the size of the event log.
+```
-As described above, the vpp engine event log is thread-safe and shared.
-To avoid confusing non-appearance of events logged by worker threads,
-make sure to code vlib\_global\_main.elog\_main - instead of
-vm->elog\_main. The latter form is correct in the main thread, but
-will almost certainly produce bad results in worker threads.
-G2 graphical event viewer
--------------------------
+Vppinfra errors and warnings
+----------------------------
-The g2 graphical event viewer can display serialized vppinfra event logs
-directly, or via the c2cpel tool.
+Many functions within the vpp dataplane have return-values of type
+clib\_error\_t \*. Clib\_error\_t's are arbitrary strings with a bit of
+metadata \[fatal, warning\] and are easy to announce. Returning a NULL
+clib\_error\_t \* indicates "A-OK, no error."
-<div class="admonition note">
+Clib\_warning(format-args) is a handy way to add debugging
+output; clib warnings prepend function:line info to unambiguously locate
+the message source. Clib\_unix\_warning() adds perror()-style Linux
+system-call information. In production images, clib\_warnings result in
+syslog entries.
-Todo: please convert wiki page and figures
+Serialization
+-------------
-</div>
+Vppinfra serialization support allows the programmer to easily serialize
+and unserialize complex data structures.
+The underlying primitive serialize/unserialize functions use network
+byte-order, so there are no structural issues serializing on a
+little-endian host and unserializing on a big-endian host.
Code Review / vpp.git / blobdiff