CSIT-1426 Update main repo README

Change-Id: Ibd10baca840c42cc6f27f0452a7162559a2b34ae
Signed-off-by: Peter Mikus <pmikus@cisco.com>

diff --git a/README b/README
deleted file mode 100644 (file)
index b14834f..0000000
--- a/README
+++ /dev/null
@@ -1,51 +0,0 @@
-# STEPS TO START DEVELOPING TESTS LOCALLY
- - install virtualenv
- - generate environment using virtualenv:
-    # cd $ROOT
-    # virtualenv env
-    # source env/bin/activate
- - install python requirements for this project by executing:
-    # pip install -r requirements.txt
- - make sure user mentioned in topology.py has NOPASSWD sudo access to
-    vpp_api_test
-
-Note:
-You can alternatively create the virtualenv with the flag --system-site-packages
-It give access to the global site-packages dir to the virtual environment, which
-is faster, but you may end up with some conflicts.
-
-
- Done.
-
-# STEPS TO START THE TESTS
-export PYTHONPATH=.
-
-# create topology, edit ip addresses
-cp topologies/available/3_node_hw_topo1.yaml.example topologies/available/topology.yaml
-ln -s ../available/topology.yaml topologies/enabled/topology.yaml
-
-pybot -L TRACE -v TOPOLOGY_PATH:topologies/enabled/topology.yaml tests
- or
-./main.py -t topologies/enabled/topology.yaml -i test_tag
- or
-./main.py
-
-
-# Dependencies on Nodes
-
- - virtualenv
- - pip
- - python2.7
- - python-dev package
- - gcc (pycrypto)
- - libpcap-devel (pypcap)
-
-``` bash
-# on fedora26
-yum install -y python-virtualenv python-pip python python-devel libpcap-devel gcc
-
-# if you have the following error during pycrypto
-# gcc: error: /usr/lib/rpm/redhat/redhat-hardened-cc1 No such file or directory
-yum install -y redhat-rpm-config
-```
-

diff --git a/README.md b/README.md
new file mode 100644 (file)
index 0000000..f878980
--- /dev/null
+++ b/README.md
@@ -0,0 +1,201 @@
+# CSIT - Continuous System Integration Testing
+
+1. [Architecture](#architecture)
+1. [Directory Structure](#directory-structure)
+   1. [Tests](#tests)
+   1. [Keywords](#keywords)
+   1. [Other Resources](#other-resources)
+1. [Quickstart](#quick-start)
+   1. [Vagrant](#vagrant)
+   1. [Physical Testbed](#physical-testbed)
+1. [Report](#report)
+1. [Trending](#trending)
+1. [Code Documentation](#code-documentation)
+1. [Coding Guidelines](#coding-guidelines)
+
+## Architecture
+
+FD.io CSIT system design needs to meet continuously expanding requirements of
+FD.io projects including VPP, related sub-systems (e.g. plugin applications,
+DPDK drivers) and FD.io applications (e.g. DPDK applications), as well as
+growing number of compute platforms running those applications. With CSIT
+project scope and charter including both FD.io continuous testing AND
+performance trending/comparisons, those evolving requirements further amplify
+the need for CSIT framework modularity, flexibility and usability.
+
+CSIT follows a hierarchical system design with SUTs and DUTs at the bottom level
+of the hierarchy, presentation level at the top level and a number of functional
+layers in-between. The current CSIT system design including CSIT framework is
+depicted in the figure below.
+
+![csit design](docs/report/csit_framework_documentation/csit_design_picture.svg "CSIT architecture")
+
+A brief bottom-up description is provided here:
+
+1. SUTs, DUTs, TGs
+   - SUTs - Systems Under Test;
+   - DUTs - Devices Under Test;
+   - TGs - Traffic Generators;
+1. Level-1 libraries - Robot and Python
+   - Lowest level CSIT libraries abstracting underlying test environment, SUT,
+     DUT and TG specifics;
+   - Used commonly across multiple L2 KWs;
+   - Performance and functional tests:
+     - L1 KWs (KeyWords) are implemented as RF libraries and Python
+       libraries;
+   - Performance TG L1 KWs:
+     - All L1 KWs are implemented as Python libraries:
+       - Support for TRex only today;
+   - Performance data plane traffic profiles:
+     - TG-specific stream profiles provide full control of:
+       - Packet definition – layers, MACs, IPs, ports, combinations thereof
+         e.g. IPs and UDP ports;
+       - Stream definitions - different streams can run together, delayed,
+         one after each other;
+       - Stream profiles are independent of CSIT framework and can be used
+         in any T-rex setup, can be sent anywhere to repeat tests with
+         exactly the same setup;
+       - Easily extensible – one can create a new stream profile that meets
+         tests requirements;
+       - Same stream profile can be used for different tests with the same
+         traffic needs;
+   - Functional data plane traffic scripts:
+     - Scapy specific traffic scripts;
+1. Level-2 libraries - Robot resource files
+   - Higher level CSIT libraries abstracting required functions for executing
+     tests;
+   - L2 KWs are classified into the following functional categories:
+     - Configuration, test, verification, state report;
+     - Suite setup, suite teardown;
+     - Test setup, test teardown;
+1. Tests - Robot
+   - Test suites with test cases;
+   - Functional tests using VIRL environment:
+     - VPP;
+     - Honeycomb;
+     - NSH_SFC;
+     - DMM;
+     - TLDK;
+   - Performance tests using physical testbed environment:
+     - VPP;
+     - DPDK-Testpmd;
+     - DPDK-L3Fwd;
+     - Honeycomb;
+     - VPP Container K8s orchestrated topologies;
+   - Tools:
+     - Documentation generator;
+     - Report generator;
+     - Testbed environment setup ansible playbooks;
+     - Operational debugging scripts;
+
+## Directory Structure
+
+### Tests
+
+```
+.
+└── tests
+    ├── dmm
+    │   └── func                    # DMM functional VIRL tests
+    ├── dpdk
+    │   ├── dpdk_scripts            # DPDK helper scripts
+    │   └── perf                    # DPDK performance tests
+    ├── honeycomb
+    │   ├── func                    # Honeycomb functional VIRL tests
+    │   └── perf                    # Honeycomb functional performance tests
+    ├── kubernetes
+    │   └── perf                    # VPP K8S orchestration performance tests
+    ├── nsh_sfc
+    │   ├── func                    # NSH_SFC functional tests
+    │   └── sfc_scripts             # NSH_SFC helper scripts
+    ├── tldk
+    │   ├── func                    # TLDK functional VIRL tests
+    │   ├── tldk_scripts            # TLDK helper scripts
+    │   └── tldk_testconfig         # TLDK test configuration
+    └── vpp
+        ├── device                  # VPP device tests
+        ├── func                    # VPP functional VIRL tests
+        └── perf                    # VPP performance tests
+```
+
+### Keywords
+
+```
+.
+resources
+└── libraries
+    ├── bash
+    │   ├── config
+    │   ├── entry                   # Main bootstrap entry directory
+    │   ├── function                # Bootstrap function library
+    │   ├── qemu_patches            # Custom QEMU patches (see KVM methodology)
+    │   └── shell                   # Various functions
+    ├── python                      # Python L1 KWs
+    └── robot                       # Robot Framework L2 KWs
+```
+
+### Other Resources
+
+```
+.
+├── docs                            # Main documentaion
+├── PyPI                            # PyPI packages provided by CSIT
+│   ├── jumpavg
+│   └── MLRsearch
+├── resources
+│   ├── templates                   # Templates (vpp_api_test, kubernetes, ...)
+│   ├── test_data                   # Robot Test configuration
+│   ├── tools
+│   │   ├── disk-image-builder      # Utilities for building (DCR, VM) images
+│   │   ├── doc_gen                 # Code documentation generator
+│   │   ├── papi                    # PAPI driver
+│   │   ├── presentation            # Report generator
+│   │   ├── scripts                 # Various tools
+│   │   ├── testbed-setup           # Physical testbed setup scripts
+│   │   ├── topology                # Helper scripts for topology manipulation
+│   │   ├── trex                    # TRex driver
+│   │   ├── vagrant                 # VPP device vagrant environment
+│   │   ├── virl                    # VIRL helper scripts
+│   │   └── wrk                     # WRK driver
+│   ├── topology_schemas
+│   ├── traffic_profiles            # Performance tests traffic profiles
+│   │   ├── trex
+│   │   └── wrk
+│   └── traffic_scripts             # Functional tests traffic profiles
+│       ├── dhcp
+│       ├── honeycomb
+│       └── lisp
+└── topologies                      # Linux Foundation topology files
+    ├── available
+    └── enabled
+```
+
+## Quickstart
+
+### Vagrant
+
+[Vagrant environment preparation](docs/testing_in_vagrant.rst) documentaion is
+describing local VPP Device functional testing.
+
+### Physical Testbed
+
+[Physical testbed preparation](resources/tools/testbed-setup/README.md)
+documentation is describing PXE and Ansible setup process. All the software
+requirements for running Performance Teste are part of Ansible playbooks.
+
+## Report
+
+[CSIT Report](https://docs.fd.io/csit/master/report/).
+
+## Trending
+
+[CSIT Trending](https://docs.fd.io/csit/master/trending/).
+
+## Code Documentation
+
+[CSIT Code Documentation](https://docs.fd.io/csit/master/doc/).
+
+## Coding Guidelines
+
+If you are interested in contributing, please see the
+[coding guidelines](docs/test_code_guidelines.rst).

diff --git a/docs/directory_structure.md b/docs/directory_structure.md
deleted file mode 100644 (file)
index b406598..0000000
--- a/docs/directory_structure.md
+++ /dev/null
@@ -1,537 +0,0 @@
-
-# Directory structure re-organization
-
-## Table of contents
-
-1. [The new structure](#the-new-structure)
-1. [Tests](#tests)
-   1. [Performance](#performance)
-   1. [Functional](#functional)
-1. [Keywords](#keywords)
-   1. [L2 Robot keywords](#l2-robot-keywords)
-   1. [L1 Python keywords](#l1-python-keywords)
-
-## The new structure
-
-### Tests
-```
-$CSIT/
-    tests/
-        vpp/
-            func/
-                l2bd/
-                l2xc/
-                ip4/
-                ip6/
-                ip4_tunnels/
-                ip6_tunnels/
-                vm_vhost/
-                crypto/
-                interfaces/
-                telemetry/
-                honeycomb/
-            perf/
-                l2/
-                ip4/
-                ip6/
-                ip4_tunnels/
-                ip6_tunnels/
-                vm_vhost/
-                crypto/
-        dpdk/
-            func/
-            perf/
-        nsh_sfc/
-            func/
-            perf/
-        tldk/
-            func/
-            perf/
-```
-
-### Keywords
-```
-$CSIT/
-    resources/
-        libraries/
-            bash/
-            python/
-                packages/ (dirs): feature | area
-                    modules/
-            robot/
-                shared/
-                l2/
-                ip/
-                overlay/
-                vm/
-                crypto/
-                dpdk/
-                nsh_sfc/
-                tldk/
-                honeycomb/
-                performance/
-                telemetry/
-                features/
-                fds/
-```
-
-*Notes:*
-1. **l1/** - L1 will not be created, L1 KWs will be temporarily kept in L2 (their
-   current place) and asap refactored using Python
-1. **l2/** - Not needed if we do not have l1/
-
-
-### Other resources
-```
-$CSIT/
-    resources/
-        templates/
-            vat/
-            honeycomb/
-        test_data/
-            honeycomb/
-            lisp/
-            softwire/
-        tools/
-            disk_image_builder/
-            doc_gen/
-            report_gen/
-            scripts/
-            testbed_setup/
-            topology/
-            trex/
-            vagrant/
-            virl/
-        topology_schemas/
-        traffic_scripts/
-        traffic_profiles/
-            trex/
-            ixia/
-```
-
-## Tests
-
-### Performance
-
-#### L2 Ethernet Switching
-```
-ls | grep -E "(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-(ndrpdrdisc|ndrchk|pdrchk)"
-```
-```
-l2/
-    10ge2p1vic1227-eth-l2bdbasemaclrn-ndrpdrdisc.robot
-    10ge2p1x520-dot1ad-l2xcbase-ndrchk.robot
-    10ge2p1x520-dot1ad-l2xcbase-ndrpdrdisc.robot
-    10ge2p1x520-dot1q-l2xcbase-ndrchk.robot
-    10ge2p1x520-dot1q-l2xcbase-ndrpdrdisc.robot
-    10ge2p1x520-eth-l2bdbasemaclrn-ndrchk.robot
-    10ge2p1x520-eth-l2bdbasemaclrn-ndrpdrdisc.robot
-    10ge2p1x520-eth-l2bdbasemaclrn-pdrchk.robot
-    10ge2p1x520-eth-l2xcbase-ndrchk.robot
-    10ge2p1x520-eth-l2xcbase-ndrpdrdisc.robot
-    10ge2p1x520-eth-l2xcbase-pdrchk.robot
-    10ge2p1x710-eth-l2bdbasemaclrn-ndrpdrdisc.robot
-    40ge2p1vic1385-eth-l2bdbasemaclrn-ndrpdrdisc.robot
-    40ge2p1xl710-eth-l2bdbasemaclrn-ndrpdrdisc.robot
-    40ge2p1xl710-eth-l2xcbase-ndrpdrdisc.robot
-```
-
-#### IPv4 Routed-Forwarding
-```
-ls | grep -P 'ethip4(udp|)-ip4(base|scale)[a-z0-9]*(?!-eth-[0-9]vhost).*-(ndrpdrdisc|ndrchk|pdrchk)'
-```
-```
-ip4/
-    10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrchk.robot
-    10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrpdrdisc.robot
-    10ge2p1x520-ethip4-ip4base-iacldstbase-ndrchk.robot
-    10ge2p1x520-ethip4-ip4base-iacldstbase-ndrpdrdisc.robot
-    10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrchk.robot
-    10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrpdrdisc.robot
-    10ge2p1x520-ethip4-ip4base-ndrchk.robot
-    10ge2p1x520-ethip4-ip4base-ndrpdrdisc.robot
-    10ge2p1x520-ethip4-ip4base-pdrchk.robot
-    10ge2p1x520-ethip4-ip4base-snat-ndrpdrdisc.robot
-    10ge2p1x520-ethip4-ip4scale200k-ndrchk.robot
-    10ge2p1x520-ethip4-ip4scale200k-ndrpdrdisc.robot
-    10ge2p1x520-ethip4-ip4scale20k-ndrchk.robot
-    10ge2p1x520-ethip4-ip4scale20k-ndrpdrdisc.robot
-    10ge2p1x520-ethip4-ip4scale2m-ndrchk.robot
-    10ge2p1x520-ethip4-ip4scale2m-ndrpdrdisc.robot
-    10ge2p1x520-ethip4-ip4scale2m-pdrchk.robot
-    10ge2p1x520-ethip4udp-ip4base-udpsrcscale15-snat-ndrpdrdisc.robot
-    10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-snat-ndrpdrdisc.robot
-    10ge2p1x520-ethip4udp-ip4scale100-udpsrcscale15-snat-ndrpdrdisc.robot
-    10ge2p1x520-ethip4udp-ip4scale10-udpsrcscale15-snat-ndrpdrdisc.robot
-    10ge2p1x520-ethip4udp-ip4scale2000-udpsrcscale15-snat-ndrpdrdisc.robot
-    10ge2p1x520-ethip4udp-ip4scale4000-udpsrcscale15-snat-ndrpdrdisc.robot
-    40ge2p1xl710-ethip4-ip4base-ndrpdrdisc.robot
-```
-
-#### IPv6 Routed-Forwarding
-```
-ls | grep -E "ethip6-ip6(base|scale)[-a-z0-9]*-(ndrpdrdisc|ndrchk|pdrchk)"
-```
-```
-ip6/
-    10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrchk.robot
-    10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrpdrdisc.robot
-    10ge2p1x520-ethip6-ip6base-iacldstbase-ndrchk.robot
-    10ge2p1x520-ethip6-ip6base-iacldstbase-ndrpdrdisc.robot
-    10ge2p1x520-ethip6-ip6base-ndrchk.robot
-    10ge2p1x520-ethip6-ip6base-ndrpdrdisc.robot
-    10ge2p1x520-ethip6-ip6base-pdrchk.robot
-    10ge2p1x520-ethip6-ip6scale200k-ndrchk.robot
-    10ge2p1x520-ethip6-ip6scale200k-ndrpdrdisc.robot
-    10ge2p1x520-ethip6-ip6scale20k-ndrchk.robot
-    10ge2p1x520-ethip6-ip6scale20k-ndrpdrdisc.robot
-    10ge2p1x520-ethip6-ip6scale2m-ndrchk.robot
-    10ge2p1x520-ethip6-ip6scale2m-ndrpdrdisc.robot
-    10ge2p1x520-ethip6-ip6scale2m-pdrchk.robot
-    40ge2p1xl710-ethip6-ip6base-ndrpdrdisc.robot
-```
-
-#### IPv4 Overlay Tunnels
-```
-ls | grep -E "ethip4[a-z0-9]+-[a-z0-9]*-(ndrpdrdisc|ndrchk|pdrchk)"
-```
-```
-ip4_tunnels/
-    10ge2p1x520-ethip4lispip4-ip4base-ndrchk.robot
-    10ge2p1x520-ethip4lispip4-ip4base-ndrpdrdisc.robot
-    10ge2p1x520-ethip4lispip4-ip4base-pdrchk.robot
-    10ge2p1x520-ethip4lispip6-ip4base-ndrchk.robot
-    10ge2p1x520-ethip4lispip6-ip4base-ndrpdrdisc.robot
-    10ge2p1x520-ethip4lispip6-ip4base-pdrchk.robot
-    10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrpdrdisc.robot
-    10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-pdrchk.robot
-    10ge2p1x520-ethip4vxlan-l2xcbase-ndrchk.robot
-    10ge2p1x520-ethip4vxlan-l2xcbase-ndrpdrdisc.robot
-    10ge2p1x520-ethip4vxlan-l2xcbase-pdrchk.robot
-```
-
-#### IPv6 Overlay Tunnels
-```
-ls | grep -E "ethip6[a-z0-9]+-[a-z0-9]*-(ndrpdrdisc|ndrchk|pdrchk)"
-```
-```
-ip6_tunnels/
-    10ge2p1x520-ethip6lispip4-ip6base-ndrchk.robot
-    10ge2p1x520-ethip6lispip4-ip6base-ndrpdrdisc.robot
-    10ge2p1x520-ethip6lispip4-ip6base-pdrchk.robot
-    10ge2p1x520-ethip6lispip6-ip6base-ndrchk.robot
-    10ge2p1x520-ethip6lispip6-ip6base-ndrpdrdisc.robot
-    10ge2p1x520-ethip6lispip6-ip6base-pdrchk.robot
-```
-
-#### VM vhost Connections
-```
-ls | grep -E ".*vhost.*-(ndrpdrdisc|ndrchk|pdrchk)"
-```
-```
-vm_vhost/
-    10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot
-    10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrchk.robot
-    10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot
-    10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-pdrchk.robot
-    10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot
-    10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-pdrchk.robot
-    10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot
-    10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot
-    10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot
-    10ge2p1x520-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot
-    10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot
-    10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-pdrchk.robot
-    10ge2p1x520-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot
-    10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot
-    40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot
-    40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot
-    40ge2p1xl710-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot
-    40ge2p1xl710-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot
-```
-
-#### IPSec Crypto HW: IP4 Routed-Forwarding
-```
-ls | grep -E ".*ipsec.*-(ndrpdrdisc|ndrchk|pdrchk)"
-```
-```
-crypto/
-    40ge2p1xl710-ethip4ipsecbasetnl-ip4base-int-aes-gcm-ndrpdrdisc.robot
-    40ge2p1xl710-ethip4ipsecbasetnl-ip4base-int-cbc-sha1-ndrpdrdisc.robot
-    40ge2p1xl710-ethip4ipsecbasetnl-ip4base-tnl-aes-gcm-ndrpdrdisc.robot
-    40ge2p1xl710-ethip4ipsecbasetnl-ip4base-tnl-cbc-sha1-ndrpdrdisc.robot
-    40ge2p1xl710-ethip4ipsecscale1000tnl-ip4base-int-aes-gcm-ndrpdrdisc.robot
-    40ge2p1xl710-ethip4ipsecscale1000tnl-ip4base-int-cbc-sha1-ndrpdrdisc.robot
-    40ge2p1xl710-ethip4ipsecscale1000tnl-ip4base-tnl-aes-gcm-ndrpdrdisc.robot
-    40ge2p1xl710-ethip4ipsecscale1000tnl-ip4base-tnl-cbc-sha1-ndrpdrdisc.robot
-    40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot
-```
-
-### Functional
-
-#### L2 Ethernet Switching
-```
-l2bd/
-    eth2p-dot1ad--dot1q-l2bdbasemaclrn-vlantrans21-func.robot
-    eth2p-dot1ad-l2bdbasemaclrn-vlantrans22-func.robot
-    eth2p-dot1q--dot1ad-l2bdbasemaclrn-vlantrans12-func.robot
-    eth2p-dot1q-l2bdbasemaclrn-vlantrans11-func.robot
-    eth2p-eth-l2bdbasemaclrn-func.robot
-    eth2p-eth-l2bdbasemacstc-func.robot
-    eth4p-eth-l2bdbasemaclrn-l2shg-func.robot
-```
-
-```
-l2xc/
-    eth2p-dot1ad--dot1q-l2xcbase-vlantrans21-func.robot
-    eth2p-dot1ad-l2xcbase-func.robot
-    eth2p-dot1ad-l2xcbase-vlantrans22-func.robot
-    eth2p-dot1q--dot1ad-l2xcbase-vlantrans12-func.robot
-    eth2p-dot1q-l2xcbase-vlantrans11-func.robot
-    eth2p-eth-l2xcbase-func.robot
-    eth2p-eth-l2xcbase-iaclbase-func.robot
-```
-
-#### IPv4 Routed-Forwarding
-```
-ip4/
-    eth2p-dot1q-ip4base-func.robot
-    eth2p-ethip4-ip4base-copblklistbase-func.robot
-    eth2p-ethip4-ip4base-copwhlistbase-func.robot
-    eth2p-ethip4-ip4base-func.robot
-    eth2p-ethip4-ip4base-iaclbase-func.robot
-    eth2p-ethip4-ip4base-ip4arp-func.robot
-    eth2p-ethip4-ip4base-ip4dhcpclient-func.robot
-    eth2p-ethip4-ip4base-ip4dhcpproxy-func.robot
-    eth2p-ethip4-ip4base-ip4ecmp-func.robot
-    eth2p-ethip4-ip4base-ip4proxyarp-func.robot
-    eth2p-ethip4-ip4base-ipolicemarkbase-func.robot
-    eth2p-ethip4-ip4base-rpf-func.robot
-    eth2p-ethip4-ip4basevrf-func.robot
-```
-
-#### IPv6 Routed-Forwarding
-```
-ip6/
-    eth2p-ethip6-ip6base-copblklistbase-func.robot
-    eth2p-ethip6-ip6base-copwhlistbase-func.robot
-    eth2p-ethip6-ip6base-func.robot
-    eth2p-ethip6-ip6base-iaclbase-func.robot
-    eth2p-ethip6-ip6base-ip6dhcpproxy-func.robot
-    eth2p-ethip6-ip6base-ip6ecmp-func.robot
-    eth2p-ethip6-ip6base-ip6ra-func.robot
-    eth2p-ethip6-ip6base-ipolicemarkbase-func.robot
-    eth2p-ethip6-ip6basevrf-func.robot
-```
-
-#### IPv4 Overlay Tunnels
-```
-ip4_tunnels/
-    gre/
-        eth2p-ethip4gre-ip4base-func.robot
-    lisp/
-        api-crud-lisp-func.robot
-        eth2p-ethip4lispgpe-ip4base-func.robot
-        eth2p-ethip4lispgpe-ip4basevrf-func.robot
-        eth2p-ethip4lispgpe-ip6base-func.robot
-        eth2p-ethip4lispgpe-ip6basevrf-func.robot
-        eth2p-ethip4lisp-ip4base-func.robot
-        eth2p-ethip4lisp-l2bdbasemaclrn-func.robot
-    softwire/
-        eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremapt-func.robot
-        eth2p-ethip4--ethip6ip4-ip4base--ip6base-swirelw46-func.robot
-        eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremape-func.robot
-    vxlan/
-        eth2p-dot1qip4vxlan-l2bdbasemaclrn-func.robot
-        eth4p-ethip4vxlan-l2bdbasemaclrn-l2shg-func.robot
-        eth2p-ethip4vxlan-l2bdbasemaclrn-func.robot
-        eth2p-ethip4vxlan-l2xcbase-func.robot
-```
-
-#### IPv6 Overlay Tunnels
-```
-ip6_tunnels/
-    lisp/
-        eth2p-ethip6lispgpe-ip6base-func.robot
-        eth2p-ethip6lispgpe-ip6basevrf-func.robot
-        eth2p-ethip6lispgpe-ip4base-func.robot
-        eth2p-ethip6lisp-l2bdbasemaclrn-func.robot
-    vxlan/
-        eth2p-ethip6vxlan-l2bdbasemaclrn-func.robot
-        eth4p-ethip6vxlan-l2bdbasemaclrn-l2shg-func.robot
-```
-
-#### VM vhost Connections
-```
-vm_vhost/
-    l2bd/
-        eth2p-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-func.robot
-        eth2p-dot1q-l2bdbasemaclrn-eth-4vhost-2vm-fds-provider-nets-func.robot
-        eth2p-ethip4-l2bdbase-vhost-client-reconnect-2vm-func.robot
-        eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot
-        eth2p-ethip4vxlan-l2bdbasemaclrn--eth-4vhost-2vm-fds-tenant-nets-func.robot
-        eth2p-ethip6vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot
-        eth2p-eth-l2bdbasemaclrn-eth-2vhost-1vm-func.robot
-        eth2p-eth-l2bdbasemacstc-eth-2vhost-1vm-func.robot
-    l2xc/
-        eth2p-eth-l2xcbase-eth-2vhost-1vm-func.robot
-    ip4/
-        eth2p-ethip4-ip4base-eth-2vhost-1vm.robot
-        eth2p-ethip4ipsectptlispgpe-ip4base-eth-2vhost-1vm-func.robot
-        eth2p-ethip4ipsectptlispgpe-ip6base-eth-2vhost-1vm-func.robot
-        eth2p-ethip4lispgpe-ip4base-eth-2vhost-1vm-func.robot
-        eth2p-ethip4lispgpe-ip4basevrf-eth-2vhost-1vm-func.robot
-        eth2p-ethip4lispgpe-ip6base-eth-2vhost-1vm-func.robot
-    ip6/
-        eth2p-ethip6ipsectptlispgpe-ip4base-eth-2vhost-1vm-func.robot
-        eth2p-ethip6ipsectptlispgpe-ip6base-eth-2vhost-1vm-func.robot
-        eth2p-ethip6lispgpe-ip6base-eth-2vhost-1vm-func.robot
-        eth2p-ethip6lispgpe-ip6basevrf-eth-2vhost-1vm-func.robot
-```
-
-#### Crypto HW: IP4 Routed-Forwarding
-```
-crypto/
-    eth2p-ethip4ipsectnl-ip4base-func.robot
-    eth2p-ethip4ipsectpt-ip4base-func.robot
-    eth2p-ethip4ipsectptlispgpe-ip4base-func.robot
-    eth2p-ethip4ipsectptlispgpe-ip6base-func.robot
-    eth2p-ethip4ipsectptlispgpe-ip6basevrf-func.robot
-    eth2p-ethip6ipsectnl-ip6base-func.robot
-    eth2p-ethip6ipsectpt-ip6base-func.robot
-    eth2p-ethip6ipsectptlispgpe-ip4base-func.robot
-    eth2p-ethip6ipsectptlispgpe-ip6base-func.robot
-```
-
-#### Honecomb
-```
-honeycomb/
-    __init__.robot
-    mgmt-cfg-l2fib-apihc-apivat-func.robot
-    mgmt-cfg-slaac-apihc-func.robot
-    mgmt-cfg-l2bd-apihc-apivat-func.robot
-    mgmt-cfg-lisp-apihc-apivat-func.robot
-    mgmt-cfg-intip4-intip6-apihc-apivat-func.robot
-    mgmt-cfg-nsh-apihc-apivat-func.robot
-    mgmt-cfg-proxyarp-apihc-func.robot
-    mgmt-cfg-int-subint-apihc-apivat-func.robot
-    mgmt-cfg-snat44-apihc-apivat-func.robot
-    mgmt-cfg-vxlangpe-apihc-apivat-func.robot
-    mgmt-cfg-pluginacl-apihc-apivat-func.robot
-    mgmt-cfg-dhcp-apihc-apivat-func.robot
-    mgmt-cfg-inttap-apihc-apivat-func.robot
-    mgmt-cfg-routing-apihc-apivat-func.robot
-    mgmt-cfg-spanrx-apihc-apivat-func.robot
-    mgmt-cfg-vxlan-apihc-apivat-func.robot
-    mgmt-cfg-policer-apihc-func.robot
-    mgmt-cfg-intvhost-apihc-apivat-func.robot
-    mgmt-notif-apihcnc-func.robot
-    mgmt-cfg-proxynd6-apihc-func.robot
-    mgmt-cfg-pbb-apihc-apivat-func.robot
-    mgmt-statepersist-apihc-func.robot
-    mgmt-cfg-int-apihcnc-func.robot
-    mgmt-cfg-acl-apihc-apivat-func.robot
-```
-
-#### Telemetry
-```
-telemetry/
-    eth2p-ethip4-ip4base-ip4ipfixbase-func.robot
-    eth2p-ethip4-ip4base-ip4ipfixscale-func.robot
-    eth2p-ethip4-ip4base-spanrx-func.robot
-    eth2p-ethip6-ip6base-ip6ipfixbase-func.robot
-    eth2p-ethip6-ip6base-ip6ipfixscale-func.robot
-    eth2p-ethip6-ip6base-spanrx-func.robot
-```
-
-#### Interface
-```
-interfaces/
-    api-crud-tap-func.robot
-    eth2p-ethip4-ip4base-eth-1tap-func.robot
-    eth2p-eth-l2bdbasemaclrn-eth-2tap-func.robot
-    eth2p-eth-l2bdbasemaclrn-l2shg-eth-2tap-func.robot
-```
-
-
-## Keywords
-
-### L2 Robot keywords
-```
-shared/
-    counters.robot
-    default.robot
-    interfaces.robot
-    traffic.robot
-    testing_path.robot
-    lxc.robot
-l2/
-    bridge_domain.robot
-    l2_traffic.robot
-    l2_xconnect.robot
-    tagging.robot
-ip/
-    ipv4.robot
-    ipv6.robot
-    snat.robot
-    map.robot
-overlay/
-    gre.robot
-    lisp_static_adjacency.robot
-    lispgpe.robot
-    l2lisp.robot
-    lisp_api.robot
-    vxlan.robot
-vm/
-    double_qemu_setup.robot
-    qemu.robot
-crypto/
-    ipsec.robot
-dpdk/
-    default.robot
-nsh_sfc/
-    default.robot
-tldk/
-    TLDKUtils.robot
-honeycomb/
-    policer.robot
-    nat.robot
-    port_mirroring.robot
-    vhost_user.robot
-    netconf.robot
-    nsh.robot
-    vxlan.robot
-    dhcp.robot
-    slaac.robot
-    notifications.robot
-    routing.robot
-    access_control_lists.robot
-    tap.robot
-    interfaces.robot
-    honeycomb.robot
-    persistence.robot
-    sub_interface.robot
-    provider_backbone_bridge.robot
-    vxlan_gpe.robot
-    proxyarp.robot
-    bridge_domain.robot
-    lisp.robot
-    l2_fib.robot
-performance/
-    performance_configuration.robot
-    performance_utils.robot
-    performance_setup.robot
-telemetry/
-    span.robot
-    ipfix.robot
-features/
-    policer.robot
-    dhcp_client.robot
-    dhcp_proxy.robot
-fds/
-    default.robot
-```
-
-### L1 Python keywords
-
-No changes at this stage.

diff --git a/docs/qemu_vpp_vm b/docs/qemu_vpp_vm
deleted file mode 100644 (file)
index 7f99231..0000000
--- a/docs/qemu_vpp_vm
+++ /dev/null
@@ -1,49 +0,0 @@
-# Copyright (c) 2016 Cisco and/or its affiliates.
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at:
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-QEMU is used for VPP-VM testing enviroment. You need to run qemu-2.2.1 or newer
-in order to Vhos-user support. In CSIT VIRL setup DUT has preinstalled QEMU and
-have small VM image "/var/lib/vm/vhost-nested.img". QEMU binary path must
-be "/opt/qemu/bin/qemu-system-x86_64". You can use following script to replicate
-QEMU setup on DUT "resources/libraries/bash/qemu_build.sh" for local testing,
-out of LF's VM setup. VM image must have installed at least qemu-guest-agent,
-sshd, bridge-utils and VirtIO support. Note that VPP must be installed before
-starting QEMU, because VPP will do the hugepages configuration as part of it's
-installation process. Username/password for the VM must be cisco/cisco and
-NOPASSWD sudo access. The interface naming is based on driver (management
-interface type is Intel E1000), all E1000 interfaces will be named mgmt<n> and
-all VirtIO interfaces will be named virtio<n>. In VM
-"/etc/init.d/qemu-guest-agent" you must set "TRANSPORT=isa-serial:/dev/ttyS1"
-because ttyS0 is used by serial console and ttyS1 is dedicated for
-qemu-guest-agent in QEMU setup.
-There is python library for QEMU setup, start and some utilities
-"resources/libraries/python/QemuUtils.py" and keyword "Stop and Clear QEMU" for
-teardown in resources/libraries/robot/vm/qemu.robot. "Qemu Start" setup one
-management interface by default. You can add Vhost-user interfaces by
-"Qemu Add Vhost User If" keyword.
-Example usage in robot framework test:
-
-*** Settings ***
-| Resource | resources/libraries/robot/vm/qemu.robot
-
-*** Test Cases ***
-| VM test
-| | [Tags] | VPP_VM_ENV
-| | Qemu Set Node | ${nodes['DUT1']}
-| | Qemu Add Vhost User If | /tmp/vhost_sock
-| | ${vm}= | Set Variable | ${None}
-| | ${vm}= | Qemu Start
-| | [Teardown] | Stop and Clear QEMU | ${dut1} | ${vm}
-
-Note that VPP must be started and configured before starting the VM when using
-Vhost-user interfaces.

diff --git a/docs/report/introduction/methodology_kvm_vms_vhost_user.rst b/docs/report/introduction/methodology_kvm_vms_vhost_user.rst
index 0a0ea1b..0a465cf 100644 (file)
--- a/docs/report/introduction/methodology_kvm_vms_vhost_user.rst
+++ b/docs/report/introduction/methodology_kvm_vms_vhost_user.rst
@@ -1,6 +1,24 @@
 KVM VMs vhost-user
 ------------------
 
+QEMU is used for VPP-VM testing enviroment. By default, standard QEMU version
+preinstalled from OS repositories is used on VIRL/vpp_device functional testing
+(qemu-2.11.x for Ubuntu 18.04, qemu-2.5.0 for Ubuntu 16.04). For perfomance
+testing QEMU is downloaded from `project homepage <qemu.org>`_ and compiled
+during testing. This allows framework to easily inject QEMU patches in case of
+need. In QEMU version <2.8 we used it for increasing QEMU virtion queue size.
+In CSIT setup DUTs have small VM image `/var/lib/vm/vhost-nested.img`. QEMU
+binary can be adjusted in global settings. VM image must have installed at least
+qemu-guest-agent, sshd, bridge-utils, VirtIO support and Testpmd/L3fwd
+application. Username/password for the VM must be cisco/cisco and
+NOPASSWD sudo access. The interface naming is based on driver (management
+interface type is Intel E1000), all E1000 interfaces will be named mgmt<n> and
+all VirtIO interfaces will be named virtio<n>. In VM
+"/etc/init.d/qemu-guest-agent" you must set "TRANSPORT=isa-serial:/dev/ttyS1"
+because ttyS0 is used by serial console and ttyS1 is dedicated for
+qemu-guest-agent in QEMU setup. There is python library for QEMU setup, start
+and some utilities "resources/libraries/python/QemuUtils.py"
+
 FD.io CSIT performance lab is testing VPP vhost with KVM VMs using
 following environment settings:
 

diff --git a/docs/tg_interface_driver b/docs/tg_interface_driver
deleted file mode 100644 (file)
index a45999a..0000000
--- a/docs/tg_interface_driver
+++ /dev/null
@@ -1,14 +0,0 @@
-If using traffic scripts in test add "| Suite Setup | Setup all TGs before
-traffic script" to test suite robot file, this bind TG interfaces to the kernel
-driver specified in topology. Also add kernel driver name for TG interfaces to
-topology YAML file. You can find driver name with following command where you
-specify interface PCI address:
-# lspci -vmmks 0000:00:07.0
-Slot:   00:07.0
-Class:  Ethernet controller
-Vendor: Red Hat, Inc
-Device: Virtio network device
-SVendor:        Red Hat, Inc
-SDevice:        Device 0001
-PhySlot:        7
-Driver: virtio-pci

diff --git a/docs/topology_schemas b/docs/topology_schemas
deleted file mode 100644 (file)
index d25e99a..0000000
--- a/docs/topology_schemas
+++ /dev/null
@@ -1,2 +0,0 @@
-http://www.kuwata-lab.com/kwalify/ruby/users-guide.html
-http://www.kuwata-lab.com/kwalify/ruby/users-guide.02.html#tips-merge