3 SELinux - VPP Custom SELinux Policy
4 ===================================
9 Security-enhanced Linux (SELinux) is a security feature in the Linux
10 kernel. At a very high level, SELinux implements mandatory access
11 controls (MAC), as opposed to discretionary access control (DAC)
12 implemented in standard Linux. MAC defines how processes can interact
13 with other system components (Files, Directories, Other Processes,
14 Pipes, Sockets, Network Ports). Each system component is assigned a
15 label, and then the SELinux Policy defines which labels and which
16 actions on each label a process is able to perform. The VPP Custom
17 SELinux Policy defines the actions VPP is allowed to perform on which
20 The VPP Custom SELinux Policy is intended to be installed on RPM based
21 platforms (tested on CentOS 7 and RHEL 7). Though SELinux can run on
22 Debian platforms, it typically is not and therefore is not currently
23 being built for Debian.
25 The VPP Custom SELinux Policy does not enable or disable SELinux, only
26 allows VPP to run when SELinux is enabled. A fresh install of either
27 Fedora, CentOS or RHEL will have SELinux enabled by default. To
28 determine if SELinux is enabled on a given system and enable it if
41 To make the change persistent, modify the following file to set
42 ``SELINUX=enforcing``:
46 $ sudo vi /etc/selinux/config
48 # This file controls the state of SELinux on the system.
49 # SELINUX= can take one of these three values:
50 # enforcing - SELinux security policy is enforced.
51 # permissive - SELinux prints warnings instead of enforcing.
52 # disabled - No SELinux policy is loaded.
59 To install VPP, see the installation instructions on the VPP Wiki
60 (https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages). The
61 VPP Custom SELinux Policy is packaged in its own RPM starting in 18.04,
62 ``vpp-selinux-policy-<VERSION>-<RELEASE>.rpm``. It is packaged and
63 installed along with the other VPP RPMs.
68 If VPP has never been installed on a system, then starting in 18.04, the
69 VPP Custom SELinux Policy will be installed with the other RPMs and all
70 the system components managed by VPP will be labeled properly.
72 Fix SELinux Labels for VPP
73 ~~~~~~~~~~~~~~~~~~~~~~~~~~
75 In the case where the VPP Custom Policy is being installed for the first
76 time, either because VPP has been upgraded or packages were removed and
77 then reinstalled, several directories and files will not not be properly
78 labeled. The labels on these files will need to be fixed for VPP to run
79 properly with SELinux enabled. After the VPP Custom SELinux Policy is
80 installed, run the following commands to fix the labels. If VPP is
81 already running, make sure to restart VPP after the labels are fixed.
82 This change is persistent for the life of the file. Once the VPP Custom
83 Policy is installed on the system, subsequent files created by VPP will
84 be labeled properly. This is only to fix files created by VPP prior to
85 the VPP Custom Policy being installed.
89 $ sudo restorecon -Rv /etc/vpp/
90 $ sudo restorecon -Rv /usr/lib/vpp_api_test_plugins/
91 $ sudo restorecon -Rv /usr/lib/vpp_plugins/
92 $ sudo restorecon -Rv /usr/share/vpp/
93 $ sudo restorecon -Rv /var/run/vpp/
95 $ sudo chcon -t vpp_tmp_t /tmp/vpp_*
96 $ sudo chcon -t vpp_var_run_t /var/run/.vpp_*
98 **NOTE:** Because the VPP APIs allow custom filenames in certain
99 scenarios, the above commands may not handle all files. Inspect your
100 system and correct any files that are mislabeled. For example, to verify
101 all VPP files in ``/tmp/`` are labeled properly, run:
107 Any files not properly labeled with ``vpp_tmp_t``, run:
111 $ sudo chcon -t vpp_tmp_t /tmp/<filename>
116 Recommended Default File Directories
117 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
119 Documentation in the VPP Wiki (https://wiki.fd.io/view/VPP/) and doxygen
120 generated documentation have examples with files located in certain
121 directories. Some of the recommend file locations have been moved to
122 satisfy SELinux. Most of the documentation has been updated, but links
123 to older documentation still exist and there may have been instances
124 that were missed. Use the file locations described below to allow
125 SELinux to properly label the given files.
127 File locations that have changed: \* VPP Debug CLI Script Files \* vHost
128 Sockets \* VPP Log Files
130 VPP Debug CLI Script Files
131 ^^^^^^^^^^^^^^^^^^^^^^^^^^
133 The VPP Debug CLI, ``vppctl``, allows a sequence of CLI commands to be
134 read from a file and executed. To avoid from having to grant VPP access
135 to all of ``/tmp/`` and possibly ``/home/`` sub-directories, it is
136 recommended that any VPP Debug CLI script files be placed in a common
137 directory such as ``/usr/share/vpp/``.
143 $ cat /usr/share/vpp/scripts/gigup.txt
144 set interface state GigabitEthernet0/8/0 up
145 set interface state GigabitEthernet0/9/0 up
151 $ vppctl exec /usr/share/vpp/scripts/gigup.txt
158 _______ _ _ _____ ___
159 __/ __/ _ \ (_)__ | | / / _ \/ _ \
160 _/ _// // / / / _ \ | |/ / ___/ ___/
161 /_/ /____(_)_/\___/ |___/_/ /_/
163 vpp# exec /usr/share/vpp/scripts/gigup.txt
166 If the file is not labeled properly, you will see something similar to:
170 $ vppctl exec /home/<user>/dev/vpp/scripts/vppctl/gigup.txt
171 exec: failed to open `/home/<user>/dev/vpp/scripts/vppctl/gigup.txt': Permission denied
174 drwxrwxr-x. <user> <user> unconfined_u:object_r:user_home_t:s0 .
175 drwxrwxr-x. <user> <user> unconfined_u:object_r:user_home_t:s0 ..
176 -rw-r--r--. <user> <user> unconfined_u:object_r:user_home_t:s0 gigup.txt
178 Original Documentation
179 ''''''''''''''''''''''
181 Some of the original documentation showed script files being executed
182 out of ``/tmp/``. Convenience also may lead to script files being placed
183 in ``/home/<user>/`` subdirectories. If a file is generated by the VPP
184 process in ``/tmp/``, for example a trace file or pcap file, it will get
185 properly labeled with the SELinux label ``vpp_tmp_t``. When a file is
186 created, unless a rule is in place for the process that created it, the
187 file will inherit the SELinux label of the parent directory. So if a
188 user creates a file themselves in ``/tmp/``, it will get the SELinux
189 label ``tmp_t``, which VPP does not have permission to access. Therefore
190 it is recommended that script files are located as described above.
195 vHost sockets are created from VPP perspective in either Server or
196 Client mode. In Server mode, the socket name is provided to VPP and VPP
197 creates the socket. In Client mode, the socket name is provided to VPP
198 and the hypervisor creates the socket. In order for VPP and hypervisor
199 to share the socket resource with SELinux enabled, a rule in the VPP
200 Custom SELinux Policy has been added. This rules allows processes with
201 the ``svirt_t`` label (the hypervisor) to access sockets with the
202 ``vpp_var_run_t`` label. As such, when SELinux is enabled, vHost sockets
203 should be created in the directory ``/var/run/vpp/``.
205 .. _original-documentation-1:
207 Original Documentation
208 ''''''''''''''''''''''
210 Some of the original documentation showed vHost sockets being created in
211 the directory ``/tmp/``. To work properly with SELinux enabled, vHost
212 sockets should be created as described above.
217 The VPP log file location is set by updating the
218 ``/etc/vpp/startup.conf`` file:
222 vi /etc/vpp/startup.conf
225 log /var/log/vpp/vpp.log
229 By moving the log file to ``/var/log/vpp/``, it will get the label
230 ``vpp_log_t``, which indicates that the files are log files so they
231 benefit from the associated rules (for example granting rights to
232 logrotate so that it can manipulate them).
234 .. _original-documentation-2:
236 Original Documentation
237 ''''''''''''''''''''''
239 The default ``startup.conf`` file creates the VPP log file in
240 ``/tmp/vpp.log``. By leaving the log file in ``/tmp/``, it will get the
241 label ``vpp_tmp_t``. Moving it to ``/var/log/vpp/``, it will get the
244 Use of Non-default File Directories
245 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
247 VPP installs multiple files on the system. Some files have fixed
248 directory and file names: - /etc/bash_completion.d/vppctl_completion -
249 /etc/sysctl.d/80-vpp.conf - /usr/lib/systemd/system/vpp.service
251 Others files have default directory and file names but the default can
252 be overwritten: - /etc/vpp/startup.conf - Can be changed via the
253 ``/usr/lib/systemd/system/vpp.service`` file by changing the -c option
254 on the VPP command line:
258 ExecStart=/usr/bin/vpp -c /etc/vpp/startup.conf
262 - Can be changed via the ``/etc/vpp/startup.conf`` file by changing
263 the cli-listen setting:
269 cli-listen /run/vpp/cli.sock
273 - /var/log/vpp/vpp.log
275 - Can be changed via the ``/etc/vpp/startup.conf`` file by changing
282 log /var/log/vpp/vpp.log
286 If the directory of any VPP installed files is changed from the default,
287 ensure that the proper SELiunx label is applied. The SELinux label can
288 be determined by passing the -Z option to many common Linux commands:
293 drwxr-xr-x. root vpp system_u:object_r:vpp_var_run_t:s0 .
294 drwxr-xr-x. root root system_u:object_r:var_run_t:s0 ..
295 srwxrwxr-x. root vpp system_u:object_r:vpp_var_run_t:s0 cli.sock
300 The following SELinux types are created by the VPP Custom SELinux
301 Policy: - ``vpp_t`` - Applied to: - VPP process and spawned threads.
303 - ``vpp_config_rw_t`` - Applied to:
307 - ``vpp_tmp_t`` - Applied to:
311 - ``vpp_exec_t`` - Applied to:
315 - ``vpp_lib_t`` - Applied to:
317 - ``/usr/lib/vpp_api_test_plugins/*``
318 - ``/usr/lib/vpp_plugins/*``
320 - ``vpp_unit_file_t`` - Applied to:
322 - ``/usr/lib/systemd/system/vpp.*``
324 - ``vpp_log_t`` - Applied to:
328 - ``vpp_var_run_t`` - Applied to:
335 If SELinux issues are suspected, there are a few steps that can be taken
336 to debug the issue. This section provides a few pointers on on those
337 steps. Any SELinux JIRAs will need this information to properly address
340 Additional SELinux Packages and Setup
341 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
343 First, install the SELinux troubleshooting packages:
347 $ sudo yum -y install setroubleshoot setroubleshoot-server setools-console
349 $ sudo dnf -y install setroubleshoot setroubleshoot-server setools-console
351 To enable proper logging, restart auditd:
355 $ sudo service auditd restart
357 While debugging issues, it is best to set SELinux to ``Permissive``
358 mode. In ``Permissive`` mode, SELinux will still detect and flag errors,
359 but will allow processes to continue normal operation. This allows
360 multiple errors to be collected at once as opposed to breaking on each
361 individual error. To set SELinux to ``Permissive`` mode (until next
362 reboot or it is set back), use:
371 After debugging, to set SELinux back to ``Enforcing`` mode, use:
383 Once the SELinux troubleshooting packages are installed, perform the
384 actions that are suspected to be blocked by SELinux. Either ``tail`` the
385 log during these actions or ``grep`` the log for additional SELinux
390 sudo tail -f /var/log/messages
394 Below are some examples of SELinux logs that are generated:
398 May 14 11:28:34 svr-22 setroubleshoot: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt. For complete SELinux messages run: sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
399 May 14 11:28:34 svr-22 python: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed read access on the hostCreate.txt file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012
400 May 14 11:28:34 svr-22 setroubleshoot: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt. For complete SELinux messages run: sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
401 May 14 11:28:34 svr-22 python: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed read access on the hostCreate.txt file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012
402 May 14 11:28:37 svr-22 setroubleshoot: SELinux is preventing vpp_main from map access on the packet_socket packet_socket. For complete SELinux messages run: sealert -l ab6667d9-3f14-4dbd-96a0-7a655f7b4eb1
403 May 14 11:28:37 svr-22 python: SELinux is preventing vpp_main from map access on the packet_socket packet_socket.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp_main should be allowed map access on the packet_socket packet_socket by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012
404 May 14 11:28:51 svr-22 setroubleshoot: SELinux is preventing vpp_main from map access on the packet_socket packet_socket. For complete SELinux messages run: sealert -l ab6667d9-3f14-4dbd-96a0-7a655f7b4eb1
405 May 14 11:28:51 svr-22 python: SELinux is preventing vpp_main from map access on the packet_socket packet_socket.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp_main should be allowed map access on the packet_socket packet_socket by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012
407 From the logs above, there are two sets of commands that are recommended
408 to be run. The first is to run the ``sealert`` command. The second is to
409 run the ``ausearch | audit2allow`` commands and the ``semodule``
415 This ``sealert`` command provides a more detailed output for the given
420 $ sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
421 SELinux is preventing /usr/bin/vpp from 'read, write' accesses on the chr_file noiommu-0.
423 ***** Plugin device (91.4 confidence) suggests ****************************
425 If you want to allow vpp to have read write access on the noiommu-0 chr_file
426 Then you need to change the label on noiommu-0 to a type of a similar device.
428 # semanage fcontext -a -t SIMILAR_TYPE 'noiommu-0'
429 # restorecon -v 'noiommu-0'
431 ***** Plugin catchall (9.59 confidence) suggests **************************
433 If you believe that vpp should be allowed read write access on the noiommu-0 chr_file by default.
434 Then you should report this as a bug.
435 You can generate a local policy module to allow this access.
437 allow this access for now by executing:
438 # ausearch -c 'vpp' --raw | audit2allow -M my-vpp
439 # semodule -i my-vpp.pp
442 Additional Information:
443 Source Context system_u:system_r:vpp_t:s0
444 Target Context system_u:object_r:device_t:s0
445 Target Objects noiommu-0 [ chr_file ]
447 Source Path /usr/bin/vpp
449 Host vpp_centos7_selinux
450 Source RPM Packages vpp-19.01.2-rc0~17_gcfd3086.x86_64
452 Policy RPM selinux-policy-3.13.1-229.el7_6.12.noarch
455 Enforcing Mode Permissive
456 Host Name vpp_centos7_selinux
457 Platform Linux vpp_centos7_selinux
458 3.10.0-957.12.1.el7.x86_64 #1 SMP Mon Apr 29
459 14:59:59 UTC 2019 x86_64 x86_64
461 First Seen 2019-05-13 18:10:50 EDT
462 Last Seen 2019-05-13 18:10:50 EDT
463 Local ID a418f869-f470-4c8a-b8e9-bdd41f2dd60b
466 type=AVC msg=audit(1557785450.964:257): avc: denied { read write } for pid=5273 comm="vpp" name="noiommu-0" dev="devtmpfs" ino=36022 scontext=system_u:system_r:vpp_t:s0 tcontext=system_u:object_r:device_t:s0 tclass=chr_file permissive=1
469 type=AVC msg=audit(1557785450.964:257): avc: denied { open } for pid=5273 comm="vpp" path="/dev/vfio/noiommu-0" dev="devtmpfs" ino=36022 scontext=system_u:system_r:vpp_t:s0 tcontext=system_u:object_r:device_t:s0 tclass=chr_file permissive=1
472 type=SYSCALL msg=audit(1557785450.964:257): arch=x86_64 syscall=open success=yes exit=ENOTBLK a0=7fb395ffd7f0 a1=2 a2=7fb395ffd803 a3=7fb395ffe2a0 items=0 ppid=1 pid=5273 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=993 sgid=0 fsgid=993 tty=(none) ses=4294967295 comm=vpp exe=/usr/bin/vpp subj=system_u:system_r:vpp_t:s0 key=(null)
474 Hash: vpp,vpp_t,device_t,chr_file,read,write
476 In general, this command pumps out too much info and is only needed for
477 additional debugging for tougher issues. Also note that once the process
478 being tested is restarted, this command loses it’s context and will not
479 provide any information:
483 $ sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
485 query_alerts error (1003): id (a418f869-f470-4c8a-b8e9-bdd41f2dd60b) not found
487 ausearch \| audit2allow and semodule Commands
488 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
490 These set of commands are more useful for basic debugging. The
491 ``ausearch | audit2allow`` commands generate a set files. It may be
492 worthwhile to run the commands in a temporary subdirectory:
496 $ mkdir test-01/; cd test-01/
498 $ sudo ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain
509 class packet_socket map;
510 class file { open read };
513 #============= vpp_t ==============
514 allow vpp_t self:packet_socket map;
515 allow vpp_t user_home_t:file { open read };
517 As shown above, the file ``my-vpp.te`` has been generated. This file
518 shows possible changes to the SELinux policy that may fix the issue. If
519 an SELinux policy was being created from scratch, this policy could be
520 applied using the ``semodule -i my-vpp.pp`` command. HOWEVER, VPP
521 already has a policy in place. So these changes need to be incorporated
522 into the existing policy. The VPP SELinux policy is located in the
528 selinux_doc.md vpp-custom.fc vpp-custom.if vpp-custom.te
530 In this example, ``map`` needs to be added to the ``packet_socket``
531 class. If the ``vpp-custom.te`` is examined (prior to this fix), then
532 one would see that the ``packet_socket`` class is already defined and
533 just needs to be updated:
537 $ vi extras/selinux/vpp-custom.te
539 allow vpp_t self:process { execmem execstack setsched signal }; # too benevolent
540 allow vpp_t self:packet_socket { bind create setopt ioctl }; <---
541 allow vpp_t self:tun_socket { create relabelto relabelfrom };
544 Before blindly applying the changes proposed by the
545 ``ausearch | audit2allow`` commands, try to determine what is being
546 allowed by the policy and determine if this is desired, or if the code
547 can be reworked to no longer require the suggested permission. In the
548 ``my-vpp.te`` file from above, it is suggested to allow ``vpp_t``
549 (i.e. the VPP process) access to all files in the home directory
550 (``allow vpp_t user_home_t:file { open read };``). This was because a
551 ``vppctl exec`` command was executed calling a script located in the
552 ``/home/<user>/`` directory. Once this script was run from the
553 ``/usr/share/vpp/`` directory as described in a section above, these
554 permissions were no longer needed.