9 VPP can work in 2 different modes:
12 - multi-thread with worker threads
17 In a single-thread mode there is one main thread which handles both
18 packet processing and other management functions (Command-Line Interface
19 (CLI), API, stats). This is the default setup. There is no special
20 startup config needed.
22 Multi-thread with Worker Threads
23 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
25 In this mode, the main threads handles management functions(debug CLI,
26 API, stats collection) and one or more worker threads handle packet
27 processing from input to output of the packet.
29 Each worker thread polls input queues on subset of interfaces.
31 With RSS (Receive Side Scaling) enabled multiple threads can service one
32 physical interface (RSS function on NIC distributes traffic between
33 different queues which are serviced by different worker threads).
38 Thread placement is defined in the startup config under the cpu { … }
41 The VPP platform can place threads automatically or manually. Automatic
42 placement works in the following way:
44 - if “skip-cores X” is defined first X cores will not be used
45 - if “main-core X” is defined, VPP main thread will be placed on core
46 X, otherwise 1st available one will be used
47 - if “workers N” is defined vpp will allocate first N available cores
48 and it will run threads on them
49 - if “corelist-workers A,B1-Bn,C1-Cn” is defined vpp will automatically
50 assign those CPU cores to worker threads
52 User can see active placement of cores by using the VPP debug CLI
55 .. code-block:: console
58 ID Name Type LWP lcore Core Socket State
59 0 vpe_main 59723 2 2 0 wait
60 1 vpe_wk_0 workers 59755 4 4 0 running
61 2 vpe_wk_1 workers 59756 5 5 0 running
62 3 vpe_wk_2 workers 59757 6 0 1 running
63 4 vpe_wk_3 workers 59758 7 1 1 running
67 The sample output above shows the main thread running on core 2 (2nd
68 core on the CPU socket 0), worker threads running on cores 4-7.
73 By default, at start-up VPP uses
74 configuration values from: ``/etc/vpp/startup.conf``
76 The following sections describe some of the additional changes that can be made to this file.
77 This file is initially populated from the files located in the following directory ``/vpp/vpp/conf/``
82 Manual placement places the main thread on core 1, workers on cores
85 .. code-block:: console
89 corelist-workers 4-5,20-21
95 Auto placement is likely to place the main thread on core 1 and workers
98 .. code-block:: console
105 Buffer Memory Allocation
106 ~~~~~~~~~~~~~~~~~~~~~~~~
108 The VPP platform is NUMA aware. It can allocate memory for buffers on
109 different CPU sockets (NUMA nodes). The amount of memory allocated can
110 be defined in the startup config for each CPU socket by using the
111 socket-mem A[[,B],C] statement inside the dpdk { … } section.
115 .. code-block:: console
121 The above configuration allocates 1GB of memory on NUMA#0 and 1GB on
122 NUMA#1. Each worker thread uses buffers which are local to itself.
124 Buffer memory is allocated from hugepages. VPP prefers 1G pages if they
125 are available. If not 2MB pages will be used.
127 VPP takes care of mounting/unmounting hugepages file-system
128 automatically so there is no need to do that manually.
130 ’‘’NOTE’’’: If you are running latest VPP release, there is no need for
131 specifying socket-mem manually. VPP will discover all NUMA nodes and it
132 will allocate 512M on each by default. socket-mem is only needed if
133 bigger number of mbufs is required (default is 16384 per socket and can
134 be changed with num-mbufs startup config command).
136 Interface Placement in Multi-thread Setup
137 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
139 On startup, the VPP platform assigns interfaces (or interface, queue
140 pairs if RSS is used) to different worker threads in round robin
143 The following example shows debug CLI commands to show and change
146 .. code-block:: console
148 vpd# sh dpdk interface placement
149 Thread 1 (vpp_wk_0 at lcore 5):
150 TenGigabitEthernet2/0/0 queue 0
151 TenGigabitEthernet2/0/1 queue 0
152 Thread 2 (vpp_wk_1 at lcore 6):
153 TenGigabitEthernet2/0/0 queue 1
154 TenGigabitEthernet2/0/1 queue 1
156 The following shows an example of moving TenGigabitEthernet2/0/1 queue 1
157 processing to 1st worker thread:
159 .. code-block:: console
161 vpd# set interface placement TenGigabitEthernet2/0/1 queue 1 thread 1
163 vpp# sh dpdk interface placement
164 Thread 1 (vpp_wk_0 at lcore 5):
165 TenGigabitEthernet2/0/0 queue 0
166 TenGigabitEthernet2/0/1 queue 0
167 TenGigabitEthernet2/0/1 queue 1
168 Thread 2 (vpp_wk_1 at lcore 6):
169 TenGigabitEthernet2/0/0 queue 1