Typically, the user header is not present. User headers allow for other
data structures to be built atop vppinfra vectors. Users may specify the
-alignment for data elements via the [vec]()\*\_aligned macros.
+alignment for first data element of a vector via the \[vec\]()\*\_aligned
+macros.
-Vectors elements can be any C type e.g. (int, double, struct bar). This
+Vector elements can be any C type e.g. (int, double, struct bar). This
is also true for data types built atop vectors (e.g. heap, pool, etc.).
-Many macros have \_a variants supporting alignment of vector data and
-\_h variants supporting non-zero-length vector headers. The \_ha
-variants support both.
+Many macros have \_a variants supporting alignment of vector elements
+and \_h variants supporting non-zero-length vector headers. The \_ha
+variants support both. Additionally cacheline alignment within a
+vector element structure can be specified using the
+\[CLIB_CACHE_LINE_ALIGN_MARK\]() macro.
Inconsistent usage of header and/or alignment related macro variants
will cause delayed, confusing failures.
vector element as the second argument to hash\_set\_mem. It is perfectly
fine to memorize constant string addresses in the text segment.
+Timekeeping
+-----------
+
+Vppinfra includes high-precision, low-cost timing services. The
+datatype clib_time_t and associated functions reside in
+./src/vppinfra/time.\[ch\]. Call clib_time_init (clib_time_t \*cp) to
+initialize the clib_time_t object.
+
+Clib_time_init(...) can use a variety of different ways to establish
+the hardware clock frequency. At the end of the day, vppinfra
+timekeeping takes the attitude that the operating system's clock is
+the closest thing to a gold standard it has handy.
+
+When properly configured, NTP maintains kernel clock synchronization
+with a highly accurate off-premises reference clock. Notwithstanding
+network propagation delays, a synchronized NTP client will keep the
+kernel clock accurate to within 50ms or so.
+
+Why should one care? Simply put, oscillators used to generate CPU
+ticks aren't super accurate. They work pretty well, but a 0.1% error
+wouldn't be out of the question. That's a minute and a half's worth of
+error in 1 day. The error changes constantly, due to temperature
+variation, and a host of other physical factors.
+
+It's far too expensive to use system calls for timing, so we're left
+with the problem of continously adjusting our view of the CPU tick
+register's clocks_per_second parameter.
+
+The clock rate adjustment algorithm measures the number of cpu ticks
+and the "gold standard" reference time across an interval of
+approximately 16 seconds. We calculate clocks_per_second for the
+interval: use rdtsc (on x86_64) and a system call to get the latest
+cpu tick count and the kernel's latest nanosecond timestamp. We
+subtract the previous interval end values, and use exponential
+smoothing to merge the new clock rate sample into the clocks_per_second
+parameter.
+
+As of this writing, we maintain the clock rate by way of the following
+first-order differential equation:
+
+
+```
+ clocks_per_second(t) = clocks_per_second(t-1) * K + sample_cps(t)*(1-K)
+ where K = e**(-1.0/3.75);
+```
+
+This yields a per observation "half-life" of 1 minute. Empirically,
+the clock rate converges within 5 minutes, and appears to maintain
+near-perfect agreement with the kernel clock in the face of ongoing
+NTP time adjustments.
+
+See ./src/vppinfra/time.c:clib_time_verify_frequency(...) to look at
+the rate adjustment algorithm. The code rejects frequency samples
+corresponding to the sort of adjustment which might occur if someone
+changes the gold standard kernel clock by several seconds.
+
+### Monotonic timebase support
+
+Particularly during system initialization, the "gold standard" system
+reference clock can change by a large amount, in an instant. It's not
+a best practice to yank the reference clock - in either direction - by
+hours or days. In fact, some poorly-constructed use-cases do so.
+
+To deal with this reality, clib_time_now(...) returns the number of
+seconds since vpp started, *guaranteed to be monotonically
+increasing, no matter what happens to the system reference clock*.
+
+This is first-order important, to avoid breaking every active timer in
+the system. The vpp host stack alone may account for tens of millions
+of active timers. It's utterly impractical to track down and fix
+timers, so we must deal with the issue at the timebase level.
+
+Here's how it works. Prior to adjusting the clock rate, we collect the
+kernel reference clock and the cpu clock:
+
+```
+ /* Ask the kernel and the CPU what time it is... */
+ now_reference = unix_time_now ();
+ now_clock = clib_cpu_time_now ();
+```
+
+Compute changes for both clocks since the last rate adjustment,
+roughly 15 seconds ago:
+
+```
+ /* Compute change in the reference clock */
+ delta_reference = now_reference - c->last_verify_reference_time;
+
+ /* And change in the CPU clock */
+ delta_clock_in_seconds = (f64) (now_clock - c->last_verify_cpu_time) *
+ c->seconds_per_clock;
+```
+
+Delta_reference is key. Almost 100% of the time, delta_reference and
+delta_clock_in_seconds are identical modulo one system-call
+time. However, NTP or a privileged user can yank the system reference
+time - in either direction - by an hour, a day, or a decade.
+
+As described above, clib_time_now(...) must return monotonically
+increasing answers to the question "how long has it been since vpp
+started, in seconds." To do that, the clock rate adjustment algorithm
+begins by recomputing the initial reference time:
+
+```
+ c->init_reference_time += (delta_reference - delta_clock_in_seconds);
+```
+
+It's easy to convince yourself that if the reference clock changes by
+15.000000 seconds and the cpu clock tick time changes by 15.000000
+seconds, the initial reference time won't change.
+
+If, on the other hand, delta_reference is -86400.0 and delta clock is
+15.0 - reference time jumped backwards by exactly one day in a
+15-second rate update interval - we add -86415.0 to the initial
+reference time.
+
+Given the corrected initial reference time, we recompute the total
+number of cpu ticks which have occurred since the corrected initial
+reference time, at the current clock tick rate:
+
+```
+ c->total_cpu_time = (now_reference - c->init_reference_time)
+ * c->clocks_per_second;
+```
+
Format
------
```c
vec_add1 (result, 0)
- or
- result = format (result, "<whatever>%c", 0);
+ or
+ result = format (result, "<whatever>%c", 0);
```
Remember to vec\_free() the result if appropriate. Be careful not to
format\_junk() can invoke other user-format functions if desired. The
programmer shoulders responsibility for argument type-checking. It is
-typical for user format functions to blow up if the va\_arg(va,
-type) macros don't match the caller's idea of reality.
+typical for user format functions to blow up spectacularly if the
+va\_arg(va, type) macros don't match the caller's idea of reality.
Unformat
--------
Then loop parsing individual elements:
```c
- while (unformat_check_input (&input) != UNFORMAT_END_OF_INPUT)
+ while (unformat_check_input (&input) != UNFORMAT_END_OF_INPUT)
{
if (unformat (&input, "value1 %d", &value1))
;/* unformat sets value1 */
else if (unformat (&input, "value2 %d", &value2)
;/* unformat sets value2 */
else
- return clib_error_return (0, "unknown input '%U'",
+ return clib_error_return (0, "unknown input '%U'",
format_unformat_error, input);
}
```
As with format, unformat implements a user-unformat function capability
-via a "%U" user unformat function scheme.
+via a "%U" user unformat function scheme. Generally, one can trivially
+transform "format (s, "foo %d", foo) -> "unformat (input, "foo %d", &foo)".
+
+Unformat implements a couple of handy non-scanf-like format specifiers:
+
+```c
+ unformat (input, "enable %=", &enable, 1 /* defaults to 1 */);
+ unformat (input, "bitzero %|", &mask, (1<<0));
+ unformat (input, "bitone %|", &mask, (1<<1));
+ <etc>
+```
+
+The phrase "enable %=" means "set the supplied variable to the default
+value" if unformat parses the "enable" keyword all by itself. If
+unformat parses "enable 123" set the supplied variable to 123.
+
+We could clean up a number of hand-rolled "verbose" + "verbose %d"
+argument parsing codes using "%=".
+
+The phrase "bitzero %|" means "set the specified bit in the supplied
+bitmask" if unformat parses "bitzero". Although it looks like it could
+be fairly handy, it's very lightly used in the code base.
+
+`%_` toggles whether or not to skip input white space.
+
+For transition from skip to no-skip in middle of format string, skip input white space. For example, the following:
+
+```c
+fmt = "%_%d.%d%_->%_%d.%d%_"
+unformat (input, fmt, &one, &two, &three, &four);
+```
+matches input "1.2 -> 3.4".
+Without this, the space after -> does not get skipped.
+
+
+```
+
+### How to parse a single input line
+
+Debug CLI command functions MUST NOT accidentally consume input
+belonging to other debug CLI commands. Otherwise, it's impossible to
+script a set of debug CLI commands which "work fine" when issued one
+at a time.
+
+This bit of code is NOT correct:
+
+```c
+ /* Eats script input NOT beloging to it, and chokes! */
+ while (unformat_check_input (input) != UNFORMAT_END_OF_INPUT)
+ {
+ if (unformat (input, ...))
+ ;
+ else if (unformat (input, ...))
+ ;
+ else
+ return clib_error_return (0, "parse error: '%U'",
+ format_unformat_error, input);
+ }
+ }
+```
+
+When executed as part of a script, such a function will return "parse
+error: '<next-command-text>'" every time, unless it happens to be the
+last command in the script.
+
+Instead, use "unformat_line_input" to consume the rest of a line's
+worth of input - everything past the path specified in the
+VLIB_CLI_COMMAND declaration.
+
+For example, unformat_line_input with "my_command" set up as shown
+below and user input "my path is clear" will produce an
+unformat_input_t that contains "is clear".
+
+```c
+ VLIB_CLI_COMMAND (...) = {
+ .path = "my path",
+ };
+```
+
+Here's a bit of code which shows the required mechanics, in full:
+
+```c
+ static clib_error_t *
+ my_command_fn (vlib_main_t * vm,
+ unformat_input_t * input,
+ vlib_cli_command_t * cmd)
+ {
+ unformat_input_t _line_input, *line_input = &_line_input;
+ u32 this, that;
+ clib_error_t *error = 0;
+
+ if (!unformat_user (input, unformat_line_input, line_input))
+ return 0;
+
+ /*
+ * Here, UNFORMAT_END_OF_INPUT is at the end of the line we consumed,
+ * not at the end of the script...
+ */
+ while (unformat_check_input (line_input) != UNFORMAT_END_OF_INPUT)
+ {
+ if (unformat (line_input, "this %u", &this))
+ ;
+ else if (unformat (line_input, "that %u", &that))
+ ;
+ else
+ {
+ error = clib_error_return (0, "parse error: '%U'",
+ format_unformat_error, line_input);
+ goto done;
+ }
+ }
+
+ <do something based on "this" and "that", etc>
+
+ done:
+ unformat_free (line_input);
+ return error;
+ }
+ /* *INDENT-OFF* */
+ VLIB_CLI_COMMAND (my_command, static) = {
+ .path = "my path",
+ .function = my_command_fn",
+ };
+ /* *INDENT-ON* */
+
+```
+
Vppinfra errors and warnings
----------------------------
Code Review / vpp.git / blobdiff