vlib: queue_hi_thresh fix to avoid deadlock

Adapt queue_hi_thresh value using num_threads to avoid risk of deadlock
between threads which could happen for example when different NAT
threads try to handoff work to each other at the same time when their
frame queues are congested. This change ensures that each thread can
reserve a queue entry without causing problems even in the most extreme
case when all threads attempt to add to the same queue simultaneously
when the queue is nearly full.

Type: fix

Signed-off-by: Elias Rudberg <elias.rudberg@bahnhof.net>
Change-Id: I9e02f753bd00833d8dd500d181b0d4f9a454d703
(cherry picked from commit 368104d06ad6d667a8cce152426916fc654b6627)

diff --git a/src/vlib/threads.c b/src/vlib/threads.c
index 3488d70..fe3a874 100644 (file)
--- a/src/vlib/threads.c
+++ b/src/vlib/threads.c
@@ -1837,17 +1837,19 @@ vlib_frame_queue_main_init (u32 node_index, u32 frame_queue_nelts)
   vlib_frame_queue_main_t *fqm;
   vlib_frame_queue_t *fq;
   int i;
+  u32 num_threads;
 
   if (frame_queue_nelts == 0)
     frame_queue_nelts = FRAME_QUEUE_MAX_NELTS;
 
-  ASSERT (frame_queue_nelts >= 8);
+  num_threads = 1 /* main thread */  + tm->n_threads;
+  ASSERT (frame_queue_nelts >= 8 + num_threads);
 
   vec_add2 (tm->frame_queue_mains, fqm, 1);
 
   fqm->node_index = node_index;
   fqm->frame_queue_nelts = frame_queue_nelts;
-  fqm->queue_hi_thresh = frame_queue_nelts - 2;
+  fqm->queue_hi_thresh = frame_queue_nelts - num_threads;
 
   vec_validate (fqm->vlib_frame_queues, tm->n_vlib_mains - 1);
   vec_validate (fqm->per_thread_data, tm->n_vlib_mains - 1);