2 * Copyright (c) 2017 Cisco and/or its affiliates.
3 * Licensed under the Apache License, Version 2.0 (the "License");
4 * you may not use this file except in compliance with the License.
5 * You may obtain a copy of the License at:
7 * http://www.apache.org/licenses/LICENSE-2.0
9 * Unless required by applicable law or agreed to in writing, software
10 * distributed under the License is distributed on an "AS IS" BASIS,
11 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 * See the License for the specific language governing permissions and
13 * limitations under the License.
23 #include "vom/client_db.hpp"
28 The VPP Object Model (VOM) library.
30 Before we begin, a glossary of terms:
31 - Agent or client: A user mode process that links to and uses the VOM library
33 - VPP: A running instance of VPP
34 - High Availability (HA): Scenarios where the client and/or VPP restart with
35 minimal service interruption.
36 - CReate, Update, Delete (CRUD): An API style where the producer issues
37 notifications to changes to objects
39 The VOM is a C++ library that models entities in VPP as C++ classes. The
40 relationships between VOM objects and VPP entities is not always 1:1. Some
41 effort has been made to construct a higher level, more abstract API to VPP
43 The client programming model is simple (or at least I intended it to be..). The
44 client deals in ‘desired’ state, that is, it expresses the objects it wants to
45 exists (in VPP) and the properties that the object should have, i.e**;
46 Interface af1(“my-af-packet-1”, AFPACKET, admin::UP);
47 Then the client ‘writes’ this object into the ‘model’
48 OM::write(“clients-thing-1”, af1);
50 “clients-thing-1” is a description of the entity within the client’s domain that
51 ‘owns’ (or has locked or has a reference to) the VOM object. There can be many
52 owners of each VOM object. It will be the last owner’s update that will be
53 programmed in VPP. This model means that the client is not burdened with
54 maintaining which of its objects have created which VOM objects. If the client
55 is itself driven by a CRUD API, then create notifications are implemented as
56 above. Update notifications add two extra statements;
57 OM::mark(“clients-thing-1”);
59 OM::sweep(“clients-thing-1”);
60 These ‘mark’ and ‘sweep’ statements are indications to OM that firstly, indicate
61 that all the objects owned by “clients-thing-1” are now stale, i.e that the
62 client may no longer need them. If one of the subsequent writes should update a
63 stale object, then it is no longer stale. The sweep statement will ‘remove’ all
64 the remaining stale objects. In this model, the client does not need to maintain
65 the mapping of VOM objects to its own objects – it can simply express what it
67 The delete notification is simply:
68 OM::remove(“clients-thing-1”);
69 Which will remove all the objects in VOM that are owned by “clients-thing-1”.
70 Where ‘remove’ in this sense means unlock and unreference, the VOM object, and
71 VPP state, will only be truly removed once there are no more owners. This is
72 equivalent to a mark & sweep with no intermediate writes.
74 To provide this client side model the VOM is a stateful library, meaning that
75 for each entity it creates in VPP, VOM maintains its own representation of that
76 object. VOM can therefore be memory hungry. The desired state is expressed by
77 the client, the ‘actual’ state is maintained by VOM. VOM will consolidate the
78 two states when the client writes to the OM and thus issue VPP only the changes
81 The concepts of ownership and statefulness also allow the support for HA
83 VPP restart: When VPP restarts, VOM will reconnect and ‘replay’ its state, in
84 dependency order, to VPP. The client does not need to regenerate its desired
86 Client restart: when the client restarts, VOM will read/dump the current state
87 of all VPP objects and store them in the OM owned by the special owner “boot”.
88 As the client reprogrammes its desired state, objects will become owned by both
89 the boot process and the client. At the point in time, as determined by the
90 client, all stale state, that owned only by boot, can be purged. Hence the
91 system reaches the correct final state, with no interruption to VPP forwarding.
96 Each object in VOM (i.e. an interface, route, bridge-domain, etc) is stored in a
97 per-type object database, with an object-type specific key. This ‘singular’ DB
98 has a value-type of a weak pointer to the object. I use the term ‘singular’ to
99 refer to the instance of the object stored in these databases, to be distinct
100 from the instances the client constructs to represent desired state.
101 The ‘client’ DB maintains the mapping of owner to object. The value type of the
102 client DB is a shared pointer to the singular instance of the owned object.
103 Once all the owners are gone, and all the shared pointers are destroyed, the
104 singular instance is also destroyed.
106 Each VOM object has some basic behaviour:
107 update: issue to VPP an update to this object’s state. This could include the
109 sweep: delete the VPP entity – called when the object is destroyed.
110 replay: issue to VPP all the commands needed to re-programme (VPP restart HA
112 populate: read state from VPP and add it to the OM (client restart HA
115 The object code is boiler-plate, in some cases (like the ACLs) even template.
116 The objects are purposefully left as simple, functionality free as possible.
118 Communication with VPP is through a ‘queue’ of ‘commands’. A command is
119 essentially an object wrapper around a VPP binary API call (although we do use
120 the VAPI C++ bindings too). Commands come in three flavours:
122 DUMP: give me all of these things; here you go
123 EVENT; tell me about these events; here’s one …. Here’s one…. Oh here’s
126 RPC and DUMP commands are handled synchronously. Therefore on return from
127 OM::write(…) VPP has been issued with the request and responded. EVENTs are
128 asynchronous and will be delivered to the listeners in a different thread – so
131 * As such VOM provides some level of insulation to the changes to the VPP
133 ** some of the type names are shorten for brevity’s sake.
138 * The interface to writing objects into VPP OM.
144 * A class providing the RAII pattern for mark and sweep
150 * Constructor - will call mark on the key
152 mark_n_sweep(const client_db::key_t& key);
155 * Destructor - will call sweep on the key
163 mark_n_sweep(const mark_n_sweep& ms) = delete;
166 * The client whose state we are guarding.
168 client_db::key_t m_key;
177 * populate the OM with state read from HW.
179 static void populate(const client_db::key_t& key);
182 * Mark all state owned by this key as stale
184 static void mark(const client_db::key_t& key);
187 * Sweep all the key's objects that are stale
189 static void sweep(const client_db::key_t& key);
192 * Replay all of the objects to HW.
194 static void replay(void);
197 * Make the State in VPP reflect the expressed desired state.
198 * But don't call the HW - use this whilst processing dumped
201 template <typename OBJ>
202 static rc_t commit(const client_db::key_t& key, const OBJ& obj)
207 rc = OM::write(key, obj);
214 * Make the State in VPP reflect the expressed desired state.
215 * After processing all the objects in the queue, in FIFO order,
216 * any remaining state owned by the client_db::key_t is purged.
217 * This is a template function so the object's update() function is
218 * always called with the derived type.
220 template <typename OBJ>
221 static rc_t write(const client_db::key_t& key, const OBJ& obj)
226 * Find the singular instance another owner may have created.
227 * this always returns something.
229 std::shared_ptr<OBJ> inst = obj.singular();
232 * Update the existing object with the new desired state
237 * Find if the object already stored on behalf of this key.
238 * and mark them stale
240 object_ref_list& objs = m_db->find(key);
243 * Iterate through this list to find a matchin' object
244 * to the one requested.
246 auto match_ptr = [inst](const object_ref& oref) {
247 return (inst == oref.obj());
249 auto it = std::find_if(objs.begin(), objs.end(), match_ptr);
251 if (it != objs.end()) {
253 * yes, this key already owns this object.
258 * Add the singular instance to the owners list
260 objs.insert(object_ref(inst));
263 return (HW::write());
267 * Remove all object in the OM referenced by the key
269 static void remove(const client_db::key_t& key);
272 * Print each of the object in the DB into the stream provided
274 static void dump(const client_db::key_t& key, std::ostream& os);
277 * Print each of the KEYS
279 static void dump(std::ostream& os);
282 * Class definition for listeners to OM events
287 listener() = default;
288 virtual ~listener() = default;
291 * Handle a populate event
293 virtual void handle_populate(const client_db::key_t& key) = 0;
296 * Handle a replay event
298 virtual void handle_replay() = 0;
301 * Get the sortable Id of the listener
303 virtual dependency_t order() const = 0;
306 * less than operator for set sorting
308 bool operator<(const listener& listener) const
310 return (order() < listener.order());
315 * Register a listener of events
317 static bool register_listener(listener* listener);
321 * Database of object state created for each key
323 static client_db* m_db;
326 * Comparator to keep the pointers to listeners in sorted order
328 struct listener_comparator_t
330 bool operator()(const listener* l1, const listener* l2) const
332 return (l1->order() < l2->order());
337 * convenient typedef for the sorted set of listeners
339 typedef std::multiset<listener*, listener_comparator_t> listener_list;
342 * The listeners for events
344 static std::unique_ptr<listener_list> m_listeners;
349 * fd.io coding-style-patch-verification: ON
352 * eval: (c-set-style "mozilla")