Clean up multi-thread barrier-sync hold-down timer

Clean up multi-thread barrier-sync hold-down timer

Main thread: don't bother with the barrier sync hold-down timer if
none of the worker threads are busy.

Worker threads: avoid epoll_pwait (10ms timeout) when the
control-plane has been active in the last half-second.

Cherry-pick a recent dangling reference fix: pool_elt_at_index after
e.g. rx callback is required, in case the unix file pool expands.

DO NOT cherry-pick the patch into master!

Change-Id: I130dc72fd95c8a22716fa2f0f98ce35cdccc52d7
Signed-off-by: Dave Barach <[email protected]>