Clean up multi-thread barrier-sync hold-down timer

Clean up multi-thread barrier-sync hold-down timer

Main thread: don't bother with the barrier sync hold-down timer if
none of the worker threads are busy.

Worker threads: avoid epoll_pwait (10ms timeout) when the
control-plane has been active in the last half-second.

Cherry-pick a recent dangling reference fix: pool_elt_at_index after
e.g. rx callback is required, in case the unix file pool expands.

Manual feature backport to 18.07

Change-Id: I745fbb8a12aeda34b0ec7b6dcda66c0e25c3eee1
Signed-off-by: Dave Barach <[email protected]>