X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Freport%2Fintroduction%2Fmethodology_kvm_vms_vhost_user.rst;h=216d461911cd5e3b21a04fbd1023097bc76a460a;hp=34e3bc0447502c1251138201123124d47fb3348f;hb=07dc2213e6c0e95147e7025b02e91ed89ea07209;hpb=124101d22151239b0411a73ae4d2bf8d70970937 diff --git a/docs/report/introduction/methodology_kvm_vms_vhost_user.rst b/docs/report/introduction/methodology_kvm_vms_vhost_user.rst index 34e3bc0447..216d461911 100644 --- a/docs/report/introduction/methodology_kvm_vms_vhost_user.rst +++ b/docs/report/introduction/methodology_kvm_vms_vhost_user.rst @@ -1,23 +1,81 @@ KVM VMs vhost-user ------------------ -FD.io CSIT performance lab is testing VPP vhost with KVM VMs using +QEMU is used for KVM VM vhost-user testing enviroment. By default, +standard QEMU version is used, preinstalled from OS repositories +(qemu-2.11.1 for Ubuntu 18.04). The path +to the QEMU binary can be adjusted in `Constants.py`. + +FD.io CSIT performance lab is testing VPP vhost-user with KVM VMs using following environment settings: -- Tests with varying Qemu virtio queue (a.k.a. vring) sizes: [vr256] - default 256 descriptors, [vr1024] 1024 descriptors to optimize for - packet throughput. -- Tests with varying Linux :abbr:`CFS (Completely Fair Scheduler)` - settings: [cfs] default settings, [cfsrr1] CFS RoundRobin(1) policy - applied to all data plane threads handling test packet path including - all VPP worker threads and all Qemu testpmd poll-mode threads. -- Resulting test cases are all combinations with [vr256,vr1024] and - [cfs,cfsrr1] settings. -- Adjusted Linux kernel :abbr:`CFS (Completely Fair Scheduler)` - scheduler policy for data plane threads used in CSIT is documented in - `CSIT Performance Environment Tuning wiki - `_. -- The purpose is to verify performance impact (MRR and NDR/PDR - throughput) and same test measurements repeatability, by making VPP - and VM data plane threads less susceptible to other Linux OS system - tasks hijacking CPU cores running those data plane threads. +CSIT supports two types of VMs: + +- **Image-VM**: used for all functional, VPP_device, and regular + performance tests except NFV density tests. + +- **Kernel-VM**: new VM type introduced for NFV density tests to provide + greater in-VM application install flexibility and to further reduce + test execution time by simpler VM lifecycle management. + +Image-VM +~~~~~~~~ + +CSIT can use a pre-created VM image. The path to the image can be +adjusted in `Constants.py`. For convenience and full compatibility CSIT +repository contains a set of scripts to prepare `Built-root +`_ based embedded Linux image with all the +dependencies needed to run DPDK Testpmd, DPDK L3Fwd, Linux bridge or +Linux IPv4 forwarding. + +Built-root was chosen for a VM image to make it lightweight and with +fast booting time to limit impact on tests duration. + +In order to execute CSIT tests, VM image must have following software +installed: qemu-guest-agent, sshd, bridge-utils, VirtIO support and DPDK +Testpmd/L3fwd applications. Username/password for the VM must be +``cisco``/``cisco`` and ``NOPASSWD`` sudo access. The interface naming +is based on the driver (management interface type is Intel E1000), all +E1000 interfaces will be named ``mgmt`` and all VirtIO interfaces +will be named ``virtio``. In VM ``/etc/init.d/qemu-guest-agent`` must +be set to ``TRANSPORT=isa-serial:/dev/ttyS1`` because ttyS0 is used by +serial console and ttyS1 is dedicated for qemu-guest-agent in QEMU +setup. + +Kernel-VM +~~~~~~~~~ + +CSIT can use a kernel KVM image as a boot kernel, as an alternative to +image VM. This option allows better configurability of what application +is running in VM userspace. Using root9p filesystem allows mapping the +host-OS filesystem as read only guest-OS filesystem. + +Example of custom init script for the kernel-VM: + +:: + + #!/bin/bash + mount -t sysfs -o "nodev,noexec,nosuid" sysfs /sys + mount -t proc -o "nodev,noexec,nosuid" proc /proc + mkdir /dev/pts + mkdir /dev/hugepages + mount -t devpts -o "rw,noexec,nosuid,gid=5,mode=0620" devpts /dev/pts || true + mount -t tmpfs -o "rw,noexec,nosuid,size=10%,mode=0755" tmpfs /run + mount -t tmpfs -o "rw,noexec,nosuid,size=10%,mode=0755" tmpfs /tmp + mount -t hugetlbfs -o "rw,relatime,pagesize=2M" hugetlbfs /dev/hugepages + echo 0000:00:06.0 > /sys/bus/pci/devices/0000:00:06.0/driver/unbind + echo 0000:00:07.0 > /sys/bus/pci/devices/0000:00:07.0/driver/unbind + echo vfio-pci > /sys/bus/pci/devices/0000:00:06.0/driver_override + echo vfio-pci > /sys/bus/pci/devices/0000:00:07.0/driver_override + echo 0000:00:06.0 > /sys/bus/pci/drivers/vfio-pci/bind + echo 0000:00:07.0 > /sys/bus/pci/drivers/vfio-pci/bind + $vnf_bin + poweroff -f + +QemuUtils library during runtime replaces the ``$vnf_bin`` variable by +the path to NF binary and its parameters. This allows CSIT to run any +application installed on host OS, for example the same version of VPP +as running on the host-OS. + +Kernel-VM image must be available in the host filesystem as a +prerequisite. The path to kernel-VM image is defined in `Constants.py`.