Fix mutex/condvar sleep when adding API msgs, VPP-461

Fix mutex/condvar sleep when adding API msgs, VPP-461

Simply put, threads cannot sleep waiting for the vlib memory api main
input queue to drain. If, say, thread i (i !=0) fills the vlib api
main input queue with rpc requests - and then blocks trying to add
another request - the game is over.

RPCs attempt a barrier synchronization, which fails with Pr = {1.0}
because thread i is in a mutex/condvar sleep.

Add a main-thread cut-through path, which directly invokes the RPC
callback function when called on the main thread.

Change-Id: Ib036f0cc43b5738455c3a111cff64a132537152e
Signed-off-by: Dave Barach <dave@barachs.net>