6 The only limiting factor on FIB scale is the amount of memory
7 allocated to each heap the FIB uses, and there are 2:
16 The main heap is used to allocate all memory needed for the FIB
17 data-structures. Each table, created by the user, i.e. with;
19 .. code-block:: console
23 or the default table, comprises 2 *ip4_fib_t* objects.
24 The 'non-forwarding' *ip4_fib_t* contains all the entries in the table
25 and, the 'forwarding' contains the entries that are matched against in
26 the data-plane. The difference between the two sets are the entries
27 that should not be matched in the data-plane.
28 Each *ip4_fib_t* comprises an mtrie (for fast lookup in the data-plane)
29 and a hash table per-prefix length (for lookup in the control plane).
31 To see the amount of memory consumed by the IPv4 tables use:
33 .. code-block:: console
36 ipv4-VRF:0 mtrie:335744 hash:4663
37 ipv4-VRF:1 mtrie:333056 hash:3499
38 totals: mtrie:668800 hash:8162 all:676962
40 this output shows two 'empty' (i.e. no added routes) tables. Each
41 mtrie uses about 150k of memory, so each table about 300k.
44 Below the output having added 1M, 2M and 4M routes respectively:
46 .. code-block:: console
49 ipv4-VRF:0 mtrie:335744 hash:4695
50 totals: mtrie:335744 hash:4695 all:340439
52 .. code-block:: console
55 ipv4-VRF:0 mtrie:5414720 hash:41177579
56 totals: mtrie:5414720 hash:41177579 all:46592299
58 .. code-block:: console
61 ipv4-VRF:0 mtrie:22452608 hash:168544508
62 totals: mtrie:22452608 hash:168544508 all:190997116
65 IPv6 also has the concept of forwarding and non-forwarding entries,
66 however for IPv6 all the forwarding entries are stored in a single
67 hash table (same goes for the non-forwarding). The key to the hash
68 table includes the IPv6 table-id.
70 To see the amount of memory consumed by the IPv4 tables use:
72 .. code-block:: console
75 IPv6 Non-Forwarding Hash Table:
76 Hash table ip6 FIB non-fwding table
77 7 active elements 7 active buckets
79 0 linear search buckets
80 arena: base 7f2fe28bf000, next 803c0
81 used 525248 b (0 Mbytes) of 33554432 b (32 Mbytes)
83 IPv6 Forwarding Hash Table:
84 Hash table ip6 FIB fwding table
85 7 active elements 7 active buckets
87 0 linear search buckets
88 arena: base 7f2fe48bf000, next 803c0
89 used 525248 b (0 Mbytes) of 33554432 b (32 Mbytes)
91 as we scale to 128k IPv6 entries:
93 .. code-block:: console
96 IPv6 Non-Forwarding Hash Table:
97 Hash table ip6 FIB non-fwding table
98 131079 active elements 32773 active buckets
101 0 linear search buckets
102 arena: base 7fed7a514000, next 4805c0
103 used 4720064 b (4 Mbytes) of 1073741824 b (1024 Mbytes)
105 IPv6 Forwarding Hash Table:
106 Hash table ip6 FIB fwding table
107 131079 active elements 32773 active buckets
110 0 linear search buckets
111 arena: base 7fedba514000, next 4805c0
112 used 4720064 b (4 Mbytes) of 1073741824 b (1024 Mbytes)
116 .. code-block:: console
119 IPv6 Non-Forwarding Hash Table:
120 Hash table ip6 FIB non-fwding table
121 262151 active elements 65536 active buckets
124 0 linear search buckets
125 arena: base 7fed7a514000, next 880840
126 used 8915008 b (8 Mbytes) of 1073741824 b (1024 Mbytes)
128 IPv6 Forwarding Hash Table:
129 Hash table ip6 FIB fwding table
130 262151 active elements 65536 active buckets
133 0 linear search buckets
134 arena: base 7fedba514000, next 880840
135 used 8915008 b (8 Mbytes) of 1073741824 b (1024 Mbytes)
139 .. code-block:: console
142 IPv6 Non-Forwarding Hash Table:
143 Hash table ip6 FIB non-fwding table
144 1048583 active elements 65536 active buckets
146 [len 1] 65533 free elts
147 [len 2] 65531 free elts
149 0 linear search buckets
150 arena: base 7fed7a514000, next 3882740
151 used 59254592 b (56 Mbytes) of 1073741824 b (1024 Mbytes)
153 IPv6 Forwarding Hash Table:
154 Hash table ip6 FIB fwding table
155 1048583 active elements 65536 active buckets
157 [len 1] 65533 free elts
158 [len 2] 65531 free elts
160 0 linear search buckets
161 arena: base 7fedba514000, next 3882740
162 used 59254592 b (56 Mbytes) of 1073741824 b (1024 Mbytes)
164 as can be seen from the output the IPv6 hash-table in this case was scaled
165 to 1GB and 1million prefixes has used 56MB of it.
167 The main heap is also used to allocate objects that represent the FIB
168 entries in the control and data plane (see :ref:`controlplane` and
169 :ref:`dataplane`) such as *fib_entry_t* and *load_balance_t*. These come
170 from the main heap because they are not protocol specific
171 (i.e. they are used to represent either IPv4, IPv6 or MPLS
174 With 1M prefixes allocated the memory usage is:
176 .. code-block:: console
182 IPv4 unicast 1 33619968
183 IPv6 unicast 2 118502784
185 IPv4 multicast 1 1175
186 IPv6 multicast 1 525312
188 Name Size in-use /allocated totals
189 Entry 72 1048589/ 1048589 75498408/75498408
190 Entry Source 40 1048589/ 1048589 41943560/41943560
191 Entry Path-Extensions 76 0 / 0 0/0
192 multicast-Entry 192 6 / 6 1152/1152
193 Path-list 40 18 / 18 720/720
194 uRPF-list 16 14 / 14 224/224
195 Path 72 22 / 22 1584/1584
196 Node-list elements 20 1048602/ 1048602 20972040/20972040
197 Node-list heads 8 24 / 24 192/192
201 .. code-block:: console
207 IPv4 unicast 1 33619968
208 IPv6 unicast 2 252743040
210 IPv4 multicast 1 1175
211 IPv6 multicast 1 525312
213 Name Size in-use /allocated totals
214 Entry 72 2097165/ 2097165 150995880/150995880
215 Entry Source 40 2097165/ 2097165 83886600/83886600
216 Entry Path-Extensions 76 0 / 0 0/0
217 multicast-Entry 192 6 / 6 1152/1152
218 Path-list 40 18 / 19 720/760
219 uRPF-list 16 18 / 18 288/288
220 Path 72 22 / 23 1584/1656
221 Node-list elements 20 2097178/ 2097178 41943560/41943560
222 Node-list heads 8 24 / 24 192/192
224 However, the situation is not a simple as that. All of the 1M prefixes
225 added above were reachable via the same next-hop, so the path-list
226 (and path) they use is shared. As prefixes are added that use
227 different (sets of) next-hops, the number of path-lists and paths
228 requires will increase.
234 VPP collects statistics for each route. For each route VPP collects
235 byte and packet counters for packets sent to the prefix (i.e. the
236 route was matched in the data-plane) and packets sent via the prefix (i.e. the
237 matching prefix is reachable through it - like a BGP peer). This
238 requires 4 counters per route in the stats segment.
240 Below shows the size of the stats segment with 1M, 2M and 4M routes.
242 .. code-block:: console
244 total: 1023.99M, used: 127.89M, free: 896.10M, trimmable: 830.94M
245 total: 1023.99M, used: 234.14M, free: 789.85M, trimmable: 668.15M
246 total: 1023.99M, used: 456.83M, free: 567.17M, trimmable: 388.91M