6 In VPP most things are measured and counted. There are counters for
7 interface statistics, like RX, TX counters, packet drops, and so on.
8 Every node has a set of per-node counters, one set of error counters,
9 like TTL exceeded, or packet to big or out-of-buffers. And a set of
10 performance counters, like number of clocks, vectors, calls and
13 There is also a set of system counters and performance counters,
14 e.g. memory utilization per heap, buffer utilisation and so on.
16 VPP Counter Architecture
17 ------------------------
19 Counters are exposed directly via shared memory. These are the actual
20 counters in VPP, no sampling or aggregation is done by the statistics
21 infrastructure. With the exception of per node performance data under
22 /sys/node and a few system counters.
24 Clients mount the shared memory segment read-only, using a optimistic
25 concurrency algorithm.
27 Directory structure as an index.
32 The memory segment consists of a shared header, containing atomics for
33 the optimistic concurrency mechanism, and offsets into memory for the
34 directory vectors. The only data structure used is the VPP vectors. All
35 pointers are converted to offsets so that client applications can map
36 the shared memory wherever it pleases.
41 Optimistic concurrency
42 ~~~~~~~~~~~~~~~~~~~~~~
47 * Shared header first in the shared memory segment.
50 atomic_int_fast64_t epoch;
51 atomic_int_fast64_t in_progress;
52 atomic_int_fast64_t directory_offset;
53 atomic_int_fast64_t error_offset;
54 atomic_int_fast64_t stats_offset;
55 } stat_segment_shared_header_t;
60 On the VPP side there is a single writer (controlled by a spinlock).
61 When the writer starts it sets in_progress=1, continues with the update
62 of the data-structures, and when done, bumps epoch++ and sets
68 If in_progress=1, there is no point continuing, so reader sits spinning
69 on the in_progress flag until it is 0. Then it sets start_epoch = epoch
70 and continues copying out the counter data it is interested in, while
71 doing strict boundary checks on all offsets / pointers. When the reader
72 is done, it checks if in_progress=1 or if epoch != start_epoch. If
73 either of those are true is discards the data read.
75 How are counters exposed out of VPP?
76 ------------------------------------
81 All counters under /err and /if are the directly exposed VPP counters.
86 - Simple counters, counter_t array of threads of an array of interfaces
87 - Combined counters, vlib_counter_t array of threads of an array of
93 Writing a new client library
94 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
96 A new client library can either wrap the C library (libvppapiclient.so)
97 or it can integrate directly with the shared memory. That involves
98 exchanging a file descriptor over the VPP stats Unix domain socket, and
99 opening the memory mapped segment.
106 #!/usr/bin/env python3
107 from vpp_papi.vpp_stats import VPPStats
108 stats = VPPStats('/run/vpp/stats.sock')
109 dir = stats.ls(['^/if', '/err/ip4-input', '/sys/node/ip4-input'])
110 counters = stats.dump(dir)
112 # Print the RX counters for the first interface on the first worker core
113 print ('RX interface core 0, sw_if_index 0', counters['/if/rx'][0][0])
120 #include <vpp-api/client/stat_client.h>
121 #include <vppinfra/vec.h>
123 int main (int argc, char **argv) {
124 uint8_t *patterns = 0;
126 vec_add1(patterns, "^/if");
127 vec_add1(patterns, "ip4-input");
129 int rv = stat_segment_connect(STAT_SEGMENT_SOCKET_FILE);
130 uint32_t *dir = stat_segment_ls(patterns);
131 stat_segment_data_t *res = stat_segment_dump(dir);
133 for (int i = 0; i < vec_len(res); i++) {
134 switch (res[i].type) {
135 case STAT_DIR_TYPE_COUNTER_VECTOR_SIMPLE:
136 for (k = 0; k < vec_len (res[i].simple_counter_vec) - 1; k++)
137 for (j = 0; j < vec_len (res[i].simple_counter_vec[k]); j++)
138 fformat (stdout, "[%d @ %d]: %llu packets %s\n",
139 j, k, res[i].simple_counter_vec[k][j],
143 case STAT_DIR_TYPE_COUNTER_VECTOR_COMBINED:
144 for (k = 0; k < vec_len (res[i].combined_counter_vec); k++)
145 for (j = 0; j < vec_len (res[i].combined_counter_vec[k]); j++)
146 fformat (stdout, "[%d @ %d]: %llu packets, %llu bytes %s\n",
147 j, k, res[i].combined_counter_vec[k][j].packets,
148 res[i].combined_counter_vec[k][j].bytes,
152 case STAT_DIR_TYPE_ERROR_INDEX:
153 for (j = 0; j < vec_len (res[i].error_vector); j++)
154 fformat (stdout, "[@%d] %llu %s\n", j, res[i].error_vector[j], res[i].name);
157 case STAT_DIR_TYPE_SCALAR_INDEX:
158 fformat (stdout, "%.2f %s\n", res[i].scalar_value, res[i].name);
165 stat_segment_data_free (res);
171 - CLI command. vpp_get_stats [ls \| dump \| poll]
177 - Deprecate the stats over binary API calls that are based on