4 In-band OAM (iOAM) is an implementation study to record operational
5 information in the packet while the packet traverses a path between two
8 Overview of iOAM can be found in
9 `iOAM-Devnet <https://github.com/ciscodevnet/iOAM>`__ page. The
10 following IETF drafts detail the motivation and mechanism for recording
11 operational information: -
12 `iOAM-ietf-requirements <https://tools.ietf.org/html/draft-brockners-inband-oam-requirements-01>`__
13 - Describes motivation and usecases for iOAM -
14 `iOAM-ietf-data <https://tools.ietf.org/html/draft-brockners-inband-oam-data-01>`__
15 - Describes data records that can be collected using iOAM -
16 `iOAM-ietf-transport <https://tools.ietf.org/html/draft-brockners-inband-oam-transport-01>`__
17 - Lists out the transport protocols and mechanism to carry iOAM data
19 `iOAM-ietf-proof-of-transit <https://tools.ietf.org/html/draft-brockners-proof-of-transit-01>`__
20 - Describes the idea of Proof of Transit (POT) and mechanisms to
21 operationalize the idea
26 In-band OAM is expected to be deployed in a specific domain rather than
27 on the overall Internet. The part of the network which employs in-band
28 OAM is referred to as **“in-band OAM-domain”**.
30 In-band OAM data is added to a packet on entering the in-band OAM-domain
31 and is removed from the packet when exiting the domain. Within the
32 in-band OAM-domain, network nodes that the packet traverses may update
33 the in-band OAM data records.
35 - The node which adds in-band OAM data to the packet is called the
36 **“in-band OAM encapsulating node”**.
38 - The node which removes the in-band OAM data is referred to as the
39 **“in-band OAM decapsulating node”**.
41 - Nodes within the domain which are aware of in-band OAM data and read
42 and/or write or process the in-band OAM data are called **“in-band
45 Features supported in the current release
46 -----------------------------------------
48 VPP can function as in-band OAM encapsulating, transit and decapsulating
49 node. In this version of VPP in-band OAM data is transported as options
50 in an IPv6 hop-by-hop extension header. Hence in-band OAM can be enabled
53 The following iOAM features are supported:
55 - **In-band OAM Tracing** : In-band OAM supports multiple data records
56 to be recorded in the packet as the packet traverses the network.
57 These data records offer insights into the operational behavior of
58 the network. The following information can be collected in the
59 tracing data from the nodes a packet traverses:
62 - Ingress interface ID
65 - Pre-configured application data
67 - **In-band OAM Proof of Transit (POT)**: Proof of transit iOAM data is
68 added to every packet for verifying that a packet traverses a
69 specific set of nodes. In-band OAM data is updated at every node that
70 is enabled with iOAM proof of transit and is used to verify whether a
71 packet traversed all the specified nodes. When the verifier receives
72 each packet, it can validate whether the packet traversed the
78 Configuring iOAM involves: - Selecting the packets for which iOAM data
79 must be inserted, updated or removed - Selection of packets for iOAM
80 data insertion on iOAM encapsulating node. Selection of packets is done
81 by 5-tuple based classification - Selection of packets for updating iOAM
82 data is implicitly done on the presence of iOAM options in the packet -
83 Selection of packets for removing the iOAM data is done on 5-tuple based
84 classification - The kind of data to be collected - Tracing data - Proof
85 of transit - Additional details for processing iOAM data to be collected
86 - For trace data - trace type, number of nodes to be recorded in the
87 trace, time stamp precision, etc. - For POT data - configuration of POT
88 profile required to process the POT data
90 The CLI for configuring iOAM is explained here followed by detailed
91 steps and examples to deploy iOAM on VPP as an encapsulating, transit or
92 decapsulating iOAM node in the subsequent sub-sections.
94 VPP iOAM configuration for enabling trace and POT is as follows:
98 set ioam rewrite trace-type <0x1f|0x7|0x9|0x11|0x19>
99 trace-elts <number of trace elements> trace-tsp <0|1|2|3>
100 node-id <node ID in hex> app-data <application data in hex> [pot]
102 A description of each of the options of the CLI follows: - trace-type :
103 An entry in the “Node data List” array of the trace option can have
104 different formats, following the needs of the a deployment. For example:
105 Some deployments might only be interested in recording the node
106 identifiers, whereas others might be interested in recording node
107 identifier and timestamp. The following types are currently supported: -
108 0x1f : Node data to include hop limit (8 bits), node ID (24 bits),
109 ingress and egress interface IDs (16 bits each), timestamp (32 bits),
110 application data (32 bits) - 0x7 : Node data to include hop limit (8
111 bits), node ID (24 bits), ingress and egress interface IDs (16 bits
112 each) - 0x9 : Node data to include hop limit (8 bits), node ID (24
113 bits), timestamp (32 bits) - 0x11: Node data to include hop limit (8
114 bits), node ID (24 bits), application data (32 bits) - 0x19: Node data
115 to include hop limit (8 bits), node ID (24 bits), timestamp (32 bits),
116 application data (32 bits) - trace-elts : Defines the length of the node
117 data array in the trace option. - trace-tsp : Defines the timestamp
118 precision to use with the enumerated value for precision as follows: - 0
119 : 32bits timestamp in seconds - 1 : 32bits timestamp in milliseconds - 2
120 : 32bits timestamp in microseconds - 3 : 32bits timestamp in nanoseconds
121 - node-id : Unique identifier for the node, included in the node ID
122 field of the node data in trace option. - app-data : The value
123 configured here is included as is in application data field of node data
124 in trace option. - pot : Enables POT option to be included in the iOAM
130 On in-band OAM encapsulating node
131 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
133 - **Configure classifier and apply ACL** to select packets for iOAM
136 - Example to enable iOAM data insertion for all the packets towards
137 IPv6 address db06::06:
139 vpp# classify table miss-next node ip6-lookup mask l3 ip6 dst
141 vpp# classify session acl-hit-next node ip6-add-hop-by-hop
142 table-index 0 match l3 ip6 dst db06::06
144 vpp# set int input acl intfc GigabitEthernet0/0/0 ip6-table 0
146 - **Enable tracing** : Specify node ID, maximum number of nodes for
147 which trace data should be recorded, type of data to be included for
148 recording, optionally application data to be included
150 - Example to enable tracing with a maximum of 4 nodes recorded and
151 the data to be recorded to include - hop limit, node id, ingress
152 and egress interface IDs, timestamp (millisecond precision),
153 application data (0x1234):
155 vpp# set ioam rewrite trace-type 0x1f trace-elts 4 trace-tsp 1
156 node-id 0x1 app-data 0x1234
158 On in-band OAM transit node
159 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
161 - The transit node requires trace type, timestamp precision, node ID
162 and optionally application data to be configured, to update its node
163 data in the trace option.
169 vpp# set ioam rewrite trace-type 0x1f trace-elts 4 trace-tsp 1
170 node-id 0x2 app-data 0x1234
172 On the In-band OAM decapsulating node
173 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
175 - The decapsulating node similar to encapsulating node requires
176 **classification** of the packets to remove iOAM data from.
178 - Example to decapsulate iOAM data for packets towards db06::06,
179 configure classifier and enable it as an ACL as follows:
181 vpp# classify table miss-next node ip6-lookup mask l3 ip6 dst
183 vpp# classify session acl-hit-next node ip6-lookup table-index 0
184 match l3 ip6 dst db06::06 opaque-index 100
186 vpp# set int input acl intfc GigabitEthernet0/0/0 ip6-table 0
188 - Decapsulating node requires trace type, timestamp precision, node ID
189 and optionally application data to be configured, to update its node
190 data in the trace option before it is decapsulated.
196 vpp# set ioam rewrite trace-type 0x1f trace-elts 4
197 trace-tsp 1 node-id 0x3 app-data 0x1234
199 Proof of Transit configuration
200 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
202 For details on proof-of-transit, see the IETF draft
203 `iOAM-ietf-proof-of-transit <https://tools.ietf.org/html/draft-brockners-proof-of-transit-01>`__.
204 To enable Proof of Transit all the nodes that participate and hence are
205 verified for transit need a proof of transit profile. A script to
206 generate a proof of transit profile as per the mechanism described in
207 `iOAM-ietf-proof-of-transit <https://tools.ietf.org/html/draft-brockners-proof-of-transit-01>`__
209 `iOAM-Devnet <https://github.com/ciscodevnet/iOAM>`__.
211 The Proof of transit mechanism implemented here is based on Shamir’s
212 Secret Sharing algorithm. The overall algorithm uses two polynomials
213 POLY-1 and POLY-2. The degree of polynomials depends on number of nodes
214 to be verified for transit. POLY-1 is secret and constant. Each node
215 gets a point on POLY-1 at setup-time and keeps it secret. POLY-2 is
216 public, random and per packet. Each node is assigned a point on POLY-1
217 and POLY-2 with the same x index. Each node derives its point on POLY-2
218 each time a packet arrives at it. A node then contributes its points on
219 POLY-1 and POLY-2 to construct POLY-3 (POLY-3 = POLY-1 + POLY-2) using
220 lagrange extrapolation and forwards it towards the verifier by updating
221 POT data in the packet. The verifier constructs POLY-3 from the
222 accumulated value from all the nodes and its own points on POLY-1 and
223 POLY-2 and verifies whether POLY-3 = POLY-1 + POLY-2. Only the verifier
224 knows POLY-1. The solution leverages finite field arithmetic in a field
225 of size “prime number” for reasons explained in description of Shamir’s
226 secret sharing algorithm.
228 | Here is an explanation of POT profile list and profile configuration
229 CLI to realize the above mechanism. It is best to use the script
230 provided at `iOAM-Devnet <https://github.com/ciscodevnet/iOAM>`__ to
231 generate this configuration. - **Create POT profile** : set pot
232 profile name id [0-1]
233 | [validator-key 0xu64] prime-number 0xu64 secret_share 0xu64
234 | lpc 0xu64 polynomial2 0xu64 bits-in-random [0-64]
235 | - name : Profile list name. - id : Profile id, it can be 0 or 1. A
236 maximum of two profiles can be configured per profile list. -
237 validator-key : Secret key configured only on the
238 verifier/decapsulating node used to compare and verify proof of
239 transit. - prime-number : Prime number for finite field arithmetic as
240 required by the proof of transit mechanism. - secret_share : Unique
241 point for each node on the secret polynomial POLY-1. - lpc : Lagrange
242 Polynomial Constant(LPC) calculated per node based on its point (x
243 value used for evaluating the points on the polynomial) on the
244 polynomial used in lagrange extrapolation for reconstructing
245 polynomial (POLY-3). - polynomial2 : Is the pre-evaluated value of the
246 point on 2nd polynomial(POLY-2). This is unique for each node. It is
247 pre-evaluated for all the coefficients of POLY-2 except for the
248 constant part of the polynomial that changes per packet and is
249 received as part of the POT data in the packet. - bits-in-random : To
250 control the size of the random number to be generated. This number has
251 to match the other numbers generated and used in the profile as per
254 - **Set a configured profile as active/in-use** :
255 set pot profile-active name ID [0-1]
257 - name : Name of the profile list to be used for computing POT data
259 - ID : Identifier of the profile within the list to be used.
261 .. _on-in-band-oam-encapsulating-node-1:
263 On In-band OAM encapsulating node
264 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
266 - Configure the classifier and apply ACL to select packets for iOAM
269 - Example to enable iOAM data insertion for all the packet towards
270 IPv6 address db06::06 -
272 vpp# classify table miss-next node ip6-lookup mask l3 ip6 dst
274 vpp# classify session acl-hit-next node ip6-add-hop-by-hop
275 table-index 0 match l3 ip6 dst db06::06
277 vpp# set int input acl intfc GigabitEthernet0/0/0 ip6-table 0
279 - Configure the proof of transit profile list with profiles. Each
280 profile list referred to by a name can contain 2 profiles, only one
281 is in use for updating proof of transit data at any time.
283 - Example profile list example with a profile generated from the
284 script to verify transit through 3 nodes is:
286 vpp# set pot profile name example id 0 prime-number
287 0x7fff0000fa884685 secret_share 0x6c22eff0f45ec56d lpc
288 0x7fff0000fa884682 polynomial2 0xffb543d4a9c bits-in-random 63
290 - Enable one of the profiles from the configured profile list as active
291 so that is will be used for calculating proof of transit
293 Example enable profile ID 0 from profile list example configured above:
297 vpp# set pot profile-active name example ID 0
299 - Enable POT option to be inserted
301 vpp# set ioam rewrite pot
303 .. _on-in-band-oam-transit-node-1:
305 On in-band OAM transit node
306 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
308 - Configure the proof of transit profile list with profiles for transit
311 vpp# set pot profile name example id 0 prime-number
312 0x7fff0000fa884685 secret_share 0x564cdbdec4eb625d lpc 0x1
313 polynomial2 0x23f3a227186a bits-in-random 63
315 On in-band OAM decapsulating node / verifier
316 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
318 - The decapsulating node, similar to the encapsulating node requires
319 classification of the packets to remove iOAM data from.
321 - Example to decapsulate iOAM data for packets towards db06::06
322 configure classifier and enable it as an ACL as follows:
324 vpp# classify table miss-next node ip6-lookup mask l3 ip6 dst
326 vpp# classify session acl-hit-next node ip6-lookup table-index 0
327 match l3 ip6 dst db06::06 opaque-index 100
329 vpp# set int input acl intfc GigabitEthernet0/0/0 ip6-table 0
331 - To update and verify the proof of transit, POT profile list should be
334 - Example POT profile list configured as follows:
336 vpp# set pot profile name example id 0 validate-key
337 0x7fff0000fa88465d prime-number 0x7fff0000fa884685 secret_share
338 0x7a08fbfc5b93116d lpc 0x3 polynomial2 0x3ff738597ce bits-in-random
344 Following CLIs are available to check iOAM operation: - To check iOAM
345 configuration that are effective use “show ioam summary”
351 vpp# show ioam summary
352 REWRITE FLOW CONFIGS - Not configured
353 HOP BY HOP OPTIONS - TRACE CONFIG -
354 Trace Type : 0x1f (31)
355 Trace timestamp precision : 1 (Milliseconds)
356 Num of trace nodes : 4
358 App Data : 0x1234 (4660)
359 POT OPTION - 1 (Enabled)
360 Try 'show ioam pot and show pot profile' for more information
362 - To find statistics about packets for which iOAM options were added
363 (encapsulating node) and removed (decapsulating node) execute *show
366 Example on encapsulating node:
372 1208804706 ip6-inacl input ACL hits
373 1208804706 ip6-add-hop-by-hop Pkts w/ added ip6 hop-by-hop options
375 Example on decapsulating node:
381 69508569 ip6-inacl input ACL hits
382 69508569 ip6-pop-hop-by-hop Pkts w/ removed ip6 hop-by-hop options
384 - To check the POT profiles use “show pot profile”
390 vpp# show pot profile
391 Profile list in use : example
392 POT Profile at index: 0
394 Validator : False (0)
395 Secret share : 0x564cdbdec4eb625d (6218586935324795485)
396 Prime number : 0x7fff0000fa884685 (9223090566081300101)
397 2nd polynomial(eval) : 0x23f3a227186a (39529304496234)
399 Bit mask : 0x7fffffffffffffff (9223372036854775807)
400 Profile index in use: 0
401 Pkts passed : 0x36 (54)
403 - To get statistics of POT for packets use “show ioam pot”
405 Example at encapsulating or transit node:
410 Pkts with ip6 hop-by-hop POT options - 54
411 Pkts with ip6 hop-by-hop POT options but no profile set - 0
412 Pkts with POT in Policy - 0
413 Pkts with POT out of Policy - 0
415 Example at decapsulating/verification node:
420 Pkts with ip6 hop-by-hop POT options - 54
421 Pkts with ip6 hop-by-hop POT options but no profile set - 0
422 Pkts with POT in Policy - 54
423 Pkts with POT out of Policy - 0
425 - Tracing - enable trace of IPv6 packets to view the data inserted and
428 Example when the nodes are receiving data over a DPDK interface: Enable
429 tracing using “trace add dpdk-input 20” and execute “show trace” to view
430 the iOAM data collected:
434 vpp# trace add dpdk-input 20
438 ------------------- Start of thread 0 vpp_main -------------------
442 00:00:19:294697: dpdk-input
443 GigabitEthernetb/0/0 rx queue 0
444 buffer 0x10e6b: current data 0, length 214, free-list 0, totlen-nifb 0, trace 0x0
445 PKT MBUF: port 0, nb_segs 1, pkt_len 214
446 buf_len 2176, data_len 214, ol_flags 0x0, data_off 128, phys_addr 0xe9a35a00
448 IP6: 00:50:56:9c:df:72 -> 00:50:56:9c:be:55
449 IP6_HOP_BY_HOP_OPTIONS: db05::2 -> db06::6
450 tos 0x00, flow label 0x0, hop limit 63, payload length 160
451 00:00:19:294737: ethernet-input
452 IP6: 00:50:56:9c:df:72 -> 00:50:56:9c:be:55
453 00:00:19:294753: ip6-input
454 IP6_HOP_BY_HOP_OPTIONS: db05::2 -> db06::6
455 tos 0x00, flow label 0x0, hop limit 63, payload length 160
456 00:00:19:294757: ip6-lookup
457 fib 0 adj-idx 15 : indirect via db05::2 flow hash: 0x00000000
458 IP6_HOP_BY_HOP_OPTIONS: db05::2 -> db06::6
459 tos 0x00, flow label 0x0, hop limit 63, payload length 160
460 00:00:19:294802: ip6-hop-by-hop
461 IP6_HOP_BY_HOP: next index 5 len 96 traced 96 Trace Type 0x1f , 1 elts left
462 [0] ttl 0x0 node ID 0x0 ingress 0x0 egress 0x0 ts 0x0
464 [1] ttl 0x3e node ID 0x3 ingress 0x1 egress 0x2 ts 0xb68c2213
466 [2] ttl 0x3f node ID 0x2 ingress 0x1 egress 0x2 ts 0xb68c2204
468 [3] ttl 0x40 node ID 0x1 ingress 0x5 egress 0x6 ts 0xb68c2200
471 random = 0x577a916946071950, Cumulative = 0x10b46e78a35a392d, Index = 0x0
472 00:00:19:294810: ip6-rewrite
473 tx_sw_if_index 1 adj-idx 14 : GigabitEthernetb/0/0
474 IP6: 00:50:56:9c:be:55 -> 00:50:56:9c:df:72 flow hash: 0x00000000
475 IP6: 00:50:56:9c:be:55 -> 00:50:56:9c:df:72
476 IP6_HOP_BY_HOP_OPTIONS: db05::2 -> db06::6
477 tos 0x00, flow label 0x0, hop limit 62, payload length 160
478 00:00:19:294814: GigabitEthernetb/0/0-output
480 IP6: 00:50:56:9c:be:55 -> 00:50:56:9c:df:72
481 IP6_HOP_BY_HOP_OPTIONS: db05::2 -> db06::6
482 tos 0x00, flow label 0x0, hop limit 62, payload length 160
483 00:00:19:294820: GigabitEthernetb/0/0-tx
484 GigabitEthernetb/0/0 tx queue 0
485 buffer 0x10e6b: current data 0, length 214, free-list 0, totlen-nifb 0, trace 0x0
486 IP6: 00:50:56:9c:be:55 -> 00:50:56:9c:df:72
488 IP6_HOP_BY_HOP_OPTIONS: db05::2 -> db06::6
490 tos 0x00, flow label 0x0, hop limit 62, payload length 160