3 Access Control Lists (ACLs) with FD.io VPP
4 ==========================================
6 This section is overview of the options available to implement ACLs in
7 FD.io VPP. As there are a number of way's to address ACL-like functionality,
8 it is worth a separate survey of these options with some commentary on
9 features and performance
11 All performance numbers and examples from this document are reused from
12 the `FD.io CSIT v19.04 performance report <https://docs.fd.io/csit/rls1904/report/>`__
13 All information and performance is accurate for
14 `FD.io VPP 19.04 <https://git.fd.io/vpp/tag/?h=v19.04>`__ release. The
15 sections *performance* & *operational data* below correlate directly with
16 those sections from the FD.io CSIT performance report.
21 +---------------------+-----------+-----------------------------------+
22 | Option | Relative | Features & Notes |
25 +=====================+===========+===================================+
26 | :ref:`aclplugin` | Lowest | Match on restricted L2-L4 fields, |
27 | | | stateful & stateless |
28 +---------------------+-----------+-----------------------------------+
29 | :ref:`vppcop` | Highest | Match on Layer 3 IPs, stateless |
32 +---------------------+-----------+-----------------------------------+
33 | :ref:`vppflow` | Highest | Match on restricted L2-L4 fields, |
34 | | (accelera | stateless, limited number of |
36 +---------------------+-----------+-----------------------------------+
37 | :ref:`classifiers` | TBD | Match on any field in the first |
38 | | | 80 bytes, Not measured |
39 +---------------------+-----------+-----------------------------------+
46 The FD.io VPP ACL Plugin
47 ~~~~~~~~~~~~~~~~~~~~~~~~
49 The plugin was originally developed as part of FD.io VPP and OpenStack
50 integration. The plugin needs to be enabled on specific interfaces.
52 Supports stateful and stateless ACLs on …
53 """"""""""""""""""""""""""""""""""""""""""
66 * Run before the IP flow classification.
70 * Run before interface output.
81 - Actions: permit+reflect
82 - Most heavily optimized, as are the most common use case.
83 - Faster because stateful uses a flow cache, it means the ACL hit is only taken once, up front for the flow and then becomes just look-up.
84 - Uses more memory, less deterministic as the flow cache makes it
85 more susceptible to the effects of the memory hierarchy and
91 - Actions : permit, drop
92 - Less optimized, less common use case.
93 - Slower as there is no flow-cache, every new packet incurs the same
94 amount ACL processing.
95 - Uses less memory, and are more deterministic (compared to
104 Test Case: 10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr
105 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
107 .. code-block:: console
110 Thread 0 vpp_main (lcore 1)
111 Time 3.8, average vectors/node 0.00, last 128 main loops 0.00 per node 0.00
112 vector rates in 0.0000e0, out 0.0000e0, drop 0.0000e0, punt 0.0000e0
113 Name State Calls Vectors Suspends Clocks Vectors/Call
114 acl-plugin-fa-cleaner-process any wait 0 0 14 1.29e3 0.00
115 acl-plugin-fa-worker-cleaner-pinterrupt wa 7 0 0 9.18e2 0.00
116 api-rx-from-ring active 0 0 52 8.96e4 0.00
117 dpdk-process any wait 0 0 1 1.35e4 0.00
118 fib-walk any wait 0 0 2 2.69e3 0.00
119 ip6-icmp-neighbor-discovery-ev any wait 0 0 4 1.32e3 0.00
120 lisp-retry-service any wait 0 0 2 2.90e3 0.00
121 unix-epoll-input polling 7037 0 0 1.25e6 0.00
122 vpe-oam-process any wait 0 0 2 2.28e3 0.00
124 Thread 1 vpp_wk_0 (lcore 2)
125 Time 3.8, average vectors/node 249.02, last 128 main loops 32.00 per node 273.07
126 vector rates in 6.1118e6, out 6.1118e6, drop 0.0000e0, punt 0.0000e0
127 Name State Calls Vectors Suspends Clocks Vectors/Call
128 TenGigabitEtherneta/0/0-output active 47106 11721472 0 9.47e0 248.83
129 TenGigabitEtherneta/0/0-tx active 47106 11721472 0 4.22e1 248.83
130 TenGigabitEtherneta/0/1-output active 47106 11721472 0 1.02e1 248.83
131 TenGigabitEtherneta/0/1-tx active 47106 11721472 0 4.18e1 248.83
132 acl-plugin-fa-worker-cleaner-pinterrupt wa 7 0 0 1.39e3 0.00
133 acl-plugin-in-ip4-fa active 94107 23442944 0 1.75e2 249.11
134 dpdk-input polling 47106 23442944 0 4.64e1 497.66
135 ethernet-input active 94212 23442944 0 1.55e1 248.83
136 ip4-input-no-checksum active 94107 23442944 0 3.23e1 249.11
137 ip4-lookup active 94107 23442944 0 2.91e1 249.11
138 ip4-rewrite active 94107 23442944 0 2.48e1 249.11
139 unix-epoll-input polling 46 0 0 1.54e3 0.00
144 Test Case: 64b-1t1c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr
145 """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
147 .. code-block:: console
150 Thread 0 vpp_main (lcore 1)
151 Time 3.9, average vectors/node 0.00, last 128 main loops 0.00 per node 0.00
152 vector rates in 0.0000e0, out 0.0000e0, drop 0.0000e0, punt 0.0000e0
153 Name State Calls Vectors Suspends Clocks Vectors/Call
154 acl-plugin-fa-cleaner-process any wait 0 0 16 1.40e3 0.00
155 acl-plugin-fa-worker-cleaner-pinterrupt wa 8 0 0 8.97e2 0.00
156 api-rx-from-ring active 0 0 52 7.12e4 0.00
157 dpdk-process any wait 0 0 1 1.69e4 0.00
158 fib-walk any wait 0 0 2 2.55e3 0.00
159 ip4-reassembly-expire-walk any wait 0 0 1 1.27e4 0.00
160 ip6-icmp-neighbor-discovery-ev any wait 0 0 4 1.09e3 0.00
161 ip6-reassembly-expire-walk any wait 0 0 1 2.57e3 0.00
162 lisp-retry-service any wait 0 0 2 1.18e4 0.00
163 statseg-collector-process time wait 0 0 1 6.38e3 0.00
164 unix-epoll-input polling 6320 0 0 1.41e6 0.00
165 vpe-oam-process any wait 0 0 2 7.53e3 0.00
167 Thread 1 vpp_wk_0 (lcore 2)
168 Time 3.9, average vectors/node 252.74, last 128 main loops 32.00 per node 273.07
169 vector rates in 7.5833e6, out 7.5833e6, drop 0.0000e0, punt 0.0000e0
170 Name State Calls Vectors Suspends Clocks Vectors/Call
171 TenGigabitEtherneta/0/0-output active 58325 14738944 0 9.41e0 252.70
172 TenGigabitEtherneta/0/0-tx active 58325 14738944 0 4.32e1 252.70
173 TenGigabitEtherneta/0/1-output active 58323 14738944 0 1.02e1 252.71
174 TenGigabitEtherneta/0/1-tx active 58323 14738944 0 4.31e1 252.71
175 acl-plugin-fa-worker-cleaner-pinterrupt wa 8 0 0 1.62e3 0.00
176 acl-plugin-in-ip4-fa active 116628 29477888 0 1.01e2 252.75
177 dpdk-input polling 58325 29477888 0 4.63e1 505.41
178 ethernet-input active 116648 29477888 0 1.53e1 252.71
179 ip4-input-no-checksum active 116628 29477888 0 3.21e1 252.75
180 ip4-lookup active 116628 29477888 0 2.90e1 252.75
181 ip4-rewrite active 116628 29477888 0 2.48e1 252.75
182 unix-epoll-input polling 57 0 0 2.39e3 0.00
187 Test Case: 64b-1t1c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr
188 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
190 .. code-block:: console
193 Thread 0 vpp_main (lcore 1)
194 Time 3.8, average vectors/node 0.00, last 128 main loops 0.00 per node 0.00
195 vector rates in 0.0000e0, out 0.0000e0, drop 0.0000e0, punt 0.0000e0
196 Name State Calls Vectors Suspends Clocks Vectors/Call
197 acl-plugin-fa-cleaner-process any wait 0 0 14 1.43e3 0.00
198 acl-plugin-fa-worker-cleaner-pinterrupt wa 7 0 0 9.23e2 0.00
199 api-rx-from-ring active 0 0 52 8.01e4 0.00
200 dpdk-process any wait 0 0 1 1.59e6 0.00
201 fib-walk any wait 0 0 2 6.81e3 0.00
202 ip6-icmp-neighbor-discovery-ev any wait 0 0 4 2.81e3 0.00
203 lisp-retry-service any wait 0 0 2 3.64e3 0.00
204 unix-epoll-input polling 4842 0 0 1.81e6 0.00
205 vpe-oam-process any wait 0 0 1 2.24e4 0.00
207 Thread 1 vpp_wk_0 (lcore 2)
208 Time 3.8, average vectors/node 249.29, last 128 main loops 36.00 per node 271.06
209 vector rates in 5.9196e6, out 5.9196e6, drop 0.0000e0, punt 0.0000e0
210 Name State Calls Vectors Suspends Clocks Vectors/Call
211 TenGigabitEtherneta/0/0-output active 45595 11363584 0 9.22e0 249.23
212 TenGigabitEtherneta/0/0-tx active 45595 11363584 0 4.25e1 249.23
213 TenGigabitEtherneta/0/1-output active 45594 11363584 0 9.75e0 249.23
214 TenGigabitEtherneta/0/1-tx active 45594 11363584 0 4.21e1 249.23
215 acl-plugin-fa-worker-cleaner-pinterrupt wa 7 0 0 1.28e3 0.00
216 acl-plugin-out-ip4-fa active 91155 22727168 0 1.78e2 249.32
217 dpdk-input polling 45595 22727168 0 4.64e1 498.46
218 ethernet-input active 91189 22727168 0 1.56e1 249.23
219 interface-output active 91155 22727168 0 1.13e1 249.32
220 ip4-input-no-checksum active 91155 22727168 0 1.95e1 249.32
221 ip4-lookup active 91155 22727168 0 2.88e1 249.32
222 ip4-rewrite active 91155 22727168 0 3.53e1 249.32
223 unix-epoll-input polling 44 0 0 1.53e3 0.00
228 Test Case: 64b-1t1c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr
229 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
231 .. code-block:: console
234 Thread 0 vpp_main (lcore 1)
235 Time 3.8, average vectors/node 0.00, last 128 main loops 0.00 per node 0.00
236 vector rates in 0.0000e0, out 0.0000e0, drop 0.0000e0, punt 0.0000e0
237 Name State Calls Vectors Suspends Clocks Vectors/Call
238 acl-plugin-fa-cleaner-process any wait 0 0 16 1.47e3 0.00
239 acl-plugin-fa-worker-cleaner-pinterrupt wa 8 0 0 8.51e2 0.00
240 api-rx-from-ring active 0 0 50 7.24e4 0.00
241 dpdk-process any wait 0 0 2 1.93e4 0.00
242 fib-walk any wait 0 0 2 2.02e3 0.00
243 ip4-reassembly-expire-walk any wait 0 0 1 3.96e3 0.00
244 ip6-icmp-neighbor-discovery-ev any wait 0 0 4 9.84e2 0.00
245 ip6-reassembly-expire-walk any wait 0 0 1 3.76e3 0.00
246 lisp-retry-service any wait 0 0 2 1.49e4 0.00
247 statseg-collector-process time wait 0 0 1 4.98e3 0.00
248 unix-epoll-input polling 5653 0 0 1.55e6 0.00
249 vpe-oam-process any wait 0 0 2 1.90e3 0.00
251 Thread 1 vpp_wk_0 (lcore 2)
252 Time 3.8, average vectors/node 250.85, last 128 main loops 36.00 per node 271.06
253 vector rates in 7.2686e6, out 7.2686e6, drop 0.0000e0, punt 0.0000e0
254 Name State Calls Vectors Suspends Clocks Vectors/Call
255 TenGigabitEtherneta/0/0-output active 55639 13930752 0 9.33e0 250.38
256 TenGigabitEtherneta/0/0-tx active 55639 13930752 0 4.27e1 250.38
257 TenGigabitEtherneta/0/1-output active 55636 13930758 0 9.81e0 250.39
258 TenGigabitEtherneta/0/1-tx active 55636 13930758 0 4.33e1 250.39
259 acl-plugin-fa-worker-cleaner-pinterrupt wa 8 0 0 1.62e3 0.00
260 acl-plugin-out-ip4-fa active 110988 27861510 0 1.04e2 251.03
261 dpdk-input polling 55639 27861510 0 4.62e1 500.76
262 ethernet-input active 111275 27861510 0 1.55e1 250.38
263 interface-output active 110988 27861510 0 1.21e1 251.03
264 ip4-input-no-checksum active 110988 27861510 0 1.95e1 251.03
265 ip4-lookup active 110988 27861510 0 2.89e1 251.03
266 ip4-rewrite active 110988 27861510 0 3.55e1 251.03
267 unix-epoll-input polling 54 0 0 2.43e3 0.00
272 +---------------------------------------+-------+-------------------+
273 | Test Case | MPPS | Cycles per packet |
274 +---------------------------------------+-------+-------------------+
275 | ethip4-ip4base | 18.26 | 136 |
276 +---------------------------------------+-------+-------------------+
277 | ethip4ip4udp-ip4base-iacl1sl-10kflows | 9.134 | 273 |
278 +---------------------------------------+-------+-------------------+
279 | ethip4ip4udp-ip4base-iacl1sf-10kflows | 11.06 | 226 |
280 +---------------------------------------+-------+-------------------+
285 .. figure:: /_images/ip4-2n-iacl.png
290 .. figure:: /_images/ip4-3n-oacl.png
298 .. code-block:: console
300 $ sudo vppctl ip_add_del_route 20.20.20.0/24 via 1.1.1.2 sw_if_index 1 resolve-attempts 10 count 1
301 $ sudo vppctl acl_add_replace ipv4 permit src 30.30.30.1/32 dst 40.40.40.1/32 sport 1000 dport 1000, ipv4 permit+reflect src 10.10.10.0/24, ipv4 permit+reflect src 20.20.20.0/24
302 $ sudo vppctl acl_interface_set_acl_list sw_if_index 2 input 0
303 $ sudo vppctl acl_interface_set_acl_list sw_if_index 1 input 0
308 .. code-block:: console
310 $ sudo vppctl ip_add_del_route 20.20.20.0/24 via 1.1.1.2 sw_if_index 1 resolve-attempts 10 count 1
311 $ sudo vppctl acl_add_replace ipv4 permit src 30.30.30.1/32 dst 40.40.40.1/32 sport 1000 dport 1000, ipv4 permit src 10.10.10.0/24, ipv4 permit src 20.20.20.0/24
312 $ sudo vppctl acl_interface_set_acl_list sw_if_index 2 input 0
313 $ sudo vppctl acl_interface_set_acl_list sw_if_index 1 input 0
318 - `FD.io Security Groups overview <https://wiki.fd.io/view/VPP/SecurityGroups>`__
319 - `Reflexive Access Control Lists <https://packetlife.net/blog/2008/nov/25/reflexive-access-lists/>`__
320 - `Andrew Yuort's Blog on ACLs <http://stdio.be/blog/2017-12-09-Debugging-VPP-MACIP-ACLs/>`__
327 IPv4/IPv6 white-lists using the FD.io VPP FIB, with support for multiple
333 - The cop graph nodes (input & white-list) make reuse of the FD.io VPP in FIB 2.0 implementation. Essentially
334 a successful lookup in the FIB, indicates that a packet has been white-listed and may be forwarded.
336 - cop-input: Determines if the frame is IPv4 or IPv6, and forwards to ipN-copwhitelist graph node.
338 - ipN-copwhitelist: uses the ip4_fib_[mtrie,lookup] functions to confirm the packet's ip matches a route in the white-list fib.
340 - Match: if it matches, it is then either sent to the next whitelist or to the ip layer.
342 - No Match: if it there is not match, it is sent to error-drop.
347 Note: the double-pass of the ip4-lookup and ip4-rewrite.
349 .. code-block:: console
352 Thread 0 vpp_main (lcore 1)
353 Time 3.9, average vectors/node 0.00, last 128 main loops 0.00 per node 0.00
354 vector rates in 0.0000e0, out 0.0000e0, drop 0.0000e0, punt 0.0000e0
355 Name State Calls Vectors Suspends Clocks Vectors/Call
356 api-rx-from-ring active 0 0 53 4.20e4 0.00
357 dpdk-process any wait 0 0 1 1.75e4 0.00
358 fib-walk any wait 0 0 2 1.59e3 0.00
359 ip4-reassembly-expire-walk any wait 0 0 1 2.20e3 0.00
360 ip6-icmp-neighbor-discovery-ev any wait 0 0 4 1.14e3 0.00
361 ip6-reassembly-expire-walk any wait 0 0 1 1.50e3 0.00
362 lisp-retry-service any wait 0 0 2 2.19e3 0.00
363 statseg-collector-process time wait 0 0 1 2.48e3 0.00
364 unix-epoll-input polling 2800 0 0 3.15e6 0.00
365 vpe-oam-process any wait 0 0 2 7.00e2 0.00
367 Thread 1 vpp_wk_0 (lcore 2)
368 Time 3.9, average vectors/node 220.84, last 128 main loops 20.87 per node 190.86
369 vector rates in 1.0724e7, out 1.0724e7, drop 0.0000e0, punt 0.0000e0
370 Name State Calls Vectors Suspends Clocks Vectors/Call
371 TenGigabitEtherneta/0/0-output active 94960 20698112 0 1.03e1 217.97
372 TenGigabitEtherneta/0/0-tx active 94960 20698112 0 3.97e1 217.97
373 TenGigabitEtherneta/0/1-output active 92238 20698112 0 9.92e0 224.39
374 TenGigabitEtherneta/0/1-tx active 92238 20698112 0 4.26e1 224.39
375 cop-input active 94960 20698112 0 1.98e1 217.97
376 dpdk-input polling 95154 41396224 0 4.58e1 435.04
377 ethernet-input active 92238 20698112 0 1.59e1 224.39
378 ip4-cop-whitelist active 94960 20698112 0 3.24e1 217.97
379 ip4-input active 94960 20698112 0 3.13e1 217.97
380 ip4-input-no-checksum active 92238 20698112 0 2.23e1 224.39
381 ip4-lookup active 187198 41396224 0 3.08e1 221.14
382 ip4-rewrite active 187198 41396224 0 2.47e1 221.14
383 unix-epoll-input polling 93 0 0 1.35e3 0.00
388 +-------------------------------+-------+-------------------+
389 | Test Case | MPPS | Cycles per packet |
390 +-------------------------------+-------+-------------------+
391 | ethip4-ip4base | 18.81 | 132 |
392 +-------------------------------+-------+-------------------+
393 | ethip4-ip4base-copwhtlistbase | 15.12 | 165 |
394 +-------------------------------+-------+-------------------+
396 .. figure:: /_images/ip4-acl-features-ndr.png
401 Note: a new VRF 1 is created which holds the whitelist, which then
402 applied to the interface 1.
404 .. code-block:: console
406 $ sudo vppctl ip_add_del_route 10.10.10.0/24 via 1.1.1.1 sw_if_index 2 resolve-attempts 10 count 1
407 $ sudo vppctl ip_table_add_del table 1
408 $ sudo vppctl ip_add_del_route 20.20.20.0/24 vrf 1 resolve-attempts 10 count 1 local
409 $ sudo vppctl cop_whitelist_enable_disable sw_if_index 1 ip4 fib-id 1
410 $ sudo vppctl cop_interface_enable_disable sw_if_index 1
415 - `FIB 2.0: Hierarchical, Protocol Independent. <https://wiki.fd.io/images/7/71/FIB_2.0_-_Hierarchical,_Protocol_Independent..pdf>`__
422 FD.io VPP Flow adds the ability for FD.io VPP to support matching of
423 flows and taking an associated action. This information is then used to
424 program hardware accelerations such as those available on network cards,
425 e.g. Intel® Ethernet Flow Director technology on the Intel® Ethernet
426 Controller X710/XXV710/XL710.
434 - Count: don't now what this does, presume it count's matches.
435 - Mark: Associate a matched flow with arbitrary data such as vxlan tunnel, for a lookup in the redirect graph node.
436 - Buffer Advance: Can be used advance to an encapsulated ethernet or ip header.
437 - Redirect to node: When you see a packet from flow xyz, the next node in FD.io VPP is the indicated graph node.
438 - Redirect to queue: When you see a packet from flow xyz, is to redirect to rx queue n.
439 - Drop: When you see a packet from flow xyz, drop the packet (next node is error drop).
444 - Currently the only place in FD.io VPP that this is used, is to accelerate VXLAN bypassing the Ethernet and IP Layers.
445 - Flow uses DPDK rte_flow API under the hood for those network interfaces programmed through DPDK.
446 - Redirect to node: worth remember that if you are bypassing a graph, you are bypassing all the checks in the graph node, e.e time-to-live, crcs and the like.
451 FD.io CSIT numbers for VXLan do not use FD.io Flow support.
456 FD.io CSIT numbers for VXLan do not use FD.io Flow support.
461 - `Flow API <https://git.fd.io/vpp/tree/src/vnet/flow/flow.h>`__
465 FD.io VPP Classifiers
466 ---------------------
468 The most flexible form of ACLs in FD.io VPP enable the user to match anywhere in the first
469 80 bytes of the packet header.
476 .. code-block:: console
478 $ sudo vppctl classify table mask l3 ip6 dst buckets 64
479 $ sudo vppctl classify session hit-next 0 table-index 0 match l3 ip6 dst 2001:db8:1::2 opaque-index 42
480 $ sudo vppctl set interface l2 input classify intfc host-s0_s1 ip6-table 0
485 - `Overview of classifiers <https://wiki.fd.io/view/VPP/SecurityGroups#Existing_functionality>`__
486 - `FD.io VPP Classifiers Overview <https://wiki.fd.io/view/VPP/Introduction_To_N-tuple_Classifiers>`__
487 - `FD.io VPP Classifiers CLI <https://docs.fd.io/vpp/19.04/clicmd_src_vnet_classify.html>`__
488 - `Sample Code from Andrew Yourt <http://stdio.be/vpp/t/aytest-bridge-tap-py.txt>`__