vlib: queue_hi_thresh fix to avoid deadlock

Adapt queue_hi_thresh value using num_threads to avoid risk of deadlock
between threads which could happen for example when different NAT
threads try to handoff work to each other at the same time when their
frame queues are congested. This change ensures that each thread can
reserve a queue entry without causing problems even in the most extreme
case when all threads attempt to add to the same queue simultaneously
when the queue is nearly full.

Type: fix

Signed-off-by: Elias Rudberg <elias.rudberg@bahnhof.net>
Change-Id: I9e02f753bd00833d8dd500d181b0d4f9a454d703

diff --git a/src/vlib/threads.c b/src/vlib/threads.c
index a6915c5..8226379 100644 (file)
--- a/src/vlib/threads.c
+++ b/src/vlib/threads.c
@@ -1814,17 +1814,19 @@ vlib_frame_queue_main_init (u32 node_index, u32 frame_queue_nelts)
   vlib_frame_queue_main_t *fqm;
   vlib_frame_queue_t *fq;
   int i;
+  u32 num_threads;
 
   if (frame_queue_nelts == 0)
     frame_queue_nelts = FRAME_QUEUE_MAX_NELTS;
 
-  ASSERT (frame_queue_nelts >= 8);
+  num_threads = 1 /* main thread */  + tm->n_threads;
+  ASSERT (frame_queue_nelts >= 8 + num_threads);
 
   vec_add2 (tm->frame_queue_mains, fqm, 1);
 
   fqm->node_index = node_index;
   fqm->frame_queue_nelts = frame_queue_nelts;
-  fqm->queue_hi_thresh = frame_queue_nelts - 2;
+  fqm->queue_hi_thresh = frame_queue_nelts - num_threads;
 
   vec_validate (fqm->vlib_frame_queues, tm->n_vlib_mains - 1);
   vec_validate (fqm->per_thread_data, tm->n_vlib_mains - 1);