**Prerequisites**:
-* Tests use *hs-test*'s own docker image, so building it before starting tests is a prerequisite. Run ``sudo make`` to do so
+* Install hs-test dependencies with ``make install-deps``
+* Tests use *hs-test*'s own docker image, so building it before starting tests is a prerequisite. Run ``make build[-debug]`` to do so
* Docker has to be installed and Go has to be in path of both the running user and root
* Root privileges are required to run tests as it uses Linux ``ip`` command for configuring topology
**Action flow when running a test case**:
-#. It starts with running ``./test``. This script is basically a wrapper for ``go test`` and accepts its parameters,
- for example following runs a specific test: ``./test -run TestNs/TestHttpTps``
+#. It starts with running ``make test``. Optional arguments are VERBOSE, PERSIST (topology configuration isn't cleaned up after test run),
+ and TEST=<test-name> to run specific test.
+#. ``make list-tests`` (or ``make help``) shows all test names.
#. ``go test`` compiles package ``main`` along with any files with names matching the file pattern ``*_test.go``
and then runs the resulting test binaries
#. The go test framework runs each function matching :ref:`naming convention<test-convention>`. Each of these corresponds to a `test suite`_
#. Declare method whose name starts with ``Test`` and specifies its receiver as a pointer to the suite's struct (defined in ``framework_test.go``)
#. Implement test behaviour inside the test method. This typically includes the following:
- #. Start docker container(s) as needed. Function ``dockerRun(instance, args string)``
- from ``utils.go`` serves this purpose. Alternatively use suite struct's ``NewContainer(name string)`` method to create
- an object representing a container and start it with ``run()`` method
- #. Execute *hs-test* action(s) inside any of the running containers.
- Function ``hstExec`` from ``utils.go`` does this by using ``docker exec`` command to run ``hs-test`` executable.
- For starting an VPP instance inside a container, the ``VppInstance`` struct can be used as a forward-looking alternative
- #. Run arbitrary commands inside the containers with ``dockerExec(cmd string, instance string)``
+ #. Retrieve a running container in which to run some action. Method ``getContainerByName``
+ from ``HstSuite`` struct serves this purpose
+ #. Interact with VPP through the ``VppInstance`` struct embedded in container. It provides ``vppctl`` method to access debug CLI
+ #. Run arbitrary commands inside the containers with ``exec`` method
#. Run other external tool with one of the preexisting functions in the ``utils.go`` file.
- For example, use ``wget`` with ``startWget(..)`` function
+ For example, use ``wget`` with ``startWget`` function
#. Use ``exechelper`` or just plain ``exec`` packages to run whatever else
- #. ``defer func() { exechelper.Run("docker stop <container-name>) }()`` inside the method body,
- to stop the running container(s). It's not necessary to do this if containers were created
- with suite's ``NewContainer(..)`` method
+ #. Verify results of your tests using ``assert`` methods provided by the test suite,
+ implemented by HstSuite struct
**Example test case**
-Two docker containers, each with its own VPP instance running. One VPP then pings the other.
-This can be put in file ``extras/hs-test/my_test.go`` and run with command ``./test -run TestMySuite``.
+Assumed are two docker containers, each with its own VPP instance running. One VPP then pings the other.
+This can be put in file ``extras/hs-test/my_test.go`` and run with command ``./test -run TestMySuite/TestMyCase``.
::
import (
"fmt"
- "github.com/edwarnicke/exechelper"
)
func (s *MySuite) TestMyCase() {
- t := s.T()
+ clientVpp := s.getContainerByName("client-vpp").vppInstance
- vpp1Instance := "vpp-1"
- vpp2Instance := "vpp-2"
+ serverVethAddress := s.netInterfaces["server-iface"].AddressString()
- err := dockerRun(vpp1Instance, "")
- if err != nil {
- t.Errorf("%v", err)
- return
- }
- defer func() { exechelper.Run("docker stop " + vpp1Instance) }()
-
- err = dockerRun(vpp2Instance, "")
- if err != nil {
- t.Errorf("%v", err)
- return
- }
- defer func() { exechelper.Run("docker stop " + vpp2Instance) }()
-
- _, err = hstExec("Configure2Veths srv", vpp1Instance)
- if err != nil {
- t.Errorf("%v", err)
- return
- }
-
- _, err = hstExec("Configure2Veths cln", vpp2Instance)
- if err != nil {
- t.Errorf("%v", err)
- return
- }
-
- // ping one VPP from the other
- //
- // not using dockerExec because it executes in detached mode
- // and we want to capture output from ping and show it
- command := "docker exec --detach=false vpp-1 vppctl -s /tmp/2veths/var/run/vpp/cli.sock ping 10.10.10.2"
- output, err := exechelper.CombinedOutput(command)
- if err != nil {
- t.Errorf("ping failed: %v", err)
- }
- fmt.Println(string(output))
+ result := clientVpp.vppctl("ping " + serverVethAddress)
+ s.assertNotNil(result)
+ s.log(result)
}
Modifying the framework
.. _test-convention:
-#. Adding a new suite takes place in ``framework_test.go``
+#. Adding a new suite takes place in ``framework_test.go`` and by creating a new file for the suite.
+ Naming convention for the suite files is ``suite_name_test.go`` where *name* will be replaced
+ by the actual name
-#. Make a ``struct`` with at least ``HstSuite`` struct as its member.
+#. Make a ``struct``, in the suite file, with at least ``HstSuite`` struct as its member.
HstSuite provides functionality that can be shared for all suites, like starting containers
::
HstSuite
}
-#. Implement SetupSuite method which testify runs before running the tests.
- It's important here to call ``setupSuite(s *suite.Suite, topologyName string)`` and assign its result to the suite's ``teardownSuite`` member.
- Pass the topology name to the function in the form of file name of one of the *yaml* files in ``topo`` folder.
- Without the extension. In this example, *myTopology* corresponds to file ``extras/hs-test/topo/myTopology.yaml``
+#. In suite file, implement ``SetupSuite`` method which testify runs once before starting any of the tests.
+ It's important here to call ``configureNetworkTopology`` method,
+ pass the topology name to the function in a form of file name of one of the *yaml* files in ``topo-network`` folder.
+ Without the extension. In this example, *myTopology* corresponds to file ``extras/hs-test/topo-network/myTopology.yaml``
+ This will ensure network topology, such as network interfaces and namespaces, will be created.
+ Another important method to call is ``loadContainerTopology()`` which will load
+ containers and shared volumes used by the suite. This time the name passed to method corresponds
+ to file in ``extras/hs-test/topo-containers`` folder
::
func (s *MySuite) SetupSuite() {
// Add custom setup code here
- s.teardownSuite = setupSuite(&s.Suite, "myTopology")
+ s.configureNetworkTopology("myTopology")
+ s.loadContainerTopology("2peerVeth")
}
-#. In order for ``go test`` to run this suite, we need to create a normal test function and pass our suite to ``suite.Run``
+#. In suite file, implement ``SetupTest`` method which gets executed before each test. Starting containers and
+ configuring VPP is usually placed here
+
+ ::
+
+ func (s *MySuite) SetupTest() {
+ s.SetupVolumes()
+ s.SetupContainers()
+ }
+
+#. In order for ``go test`` to run this suite, we need to create a normal test function and pass our suite to ``suite.Run``.
+ These functions are placed at the end of ``framework_test.go``
::
**Adding a topology element**
-Topology configuration exists as ``yaml`` files in the ``extras/hs-test/topo`` folder.
-Processing of a file for a particular test suite is started by the ``setupSuite`` function depending on which file's name is passed to it.
-Specified file is loaded by ``LoadTopology()`` function and converted into internal data structures which represent various elements of the topology.
-After parsing the configuration, ``Configure()`` method loops over array of topology elements and configures them one by one.
+Topology configuration exists as ``yaml`` files in the ``extras/hs-test/topo-network`` and
+``extras/hs-test/topo-containers`` folders. Processing of a network topology file for a particular test suite
+is started by the ``configureNetworkTopology`` method depending on which file's name is passed to it.
+Specified file is loaded and converted into internal data structures which represent various elements of the topology.
+After parsing the configuration, framework loops over the elements and configures them one by one on the host system.
-These are currently supported types of elements.
+These are currently supported types of network elements.
* ``netns`` - network namespace
* ``veth`` - veth network interface, optionally with target network namespace or IPv4 address
* ``bridge`` - ethernet bridge to connect created interfaces, optionally with target network namespace
* ``tap`` - tap network interface with IP address
+Similarly, container topology is started by ``loadContainerTopology()``, configuration file is processed
+so that test suite retains map of defined containers and uses that to start them at the beginning
+of each test case and stop containers after the test finishes. Container configuration can specify
+also volumes which allow to share data between containers or between host system and containers.
+
Supporting a new type of topology element requires adding code to recognize the new element type during loading.
-And adding code to set up the element in the host system with some Linux tool, such as *ip*. This should be implemented in ``netconfig.go``.
+And adding code to set up the element in the host system with some Linux tool, such as *ip*.
+This should be implemented in ``netconfig.go`` for network and in ``container.go`` for containers and volumes.
**Communicating between containers**
* Shared folders. Containers are being created with ``-v`` option to create shared `volumes`_ between host system and containers
or just between containers
-**Adding a hs-test action**
-
-Executing more complex or long running jobs is made easier by *hs-test* actions.
-These are functions that compartmentalize configuration and execution together for a specific task.
-For example, starting up VPP or running VCL echo client.
-
-The actions are located in ``extras/hs-test/actions.go``. To add one, create a new method that has its receiver as a pointer to ``Actions`` struct.
-
-Run it from test case with ``hstExec(args, instance)`` where ``args`` is the action method's name and ``instance`` is target Docker container's name.
-This then executes the ``hs-test`` binary inside of the container and it then runs selected action.
-Action is specified by its name as first argument for the binary.
-
-*Note*: When ``hstExec(..)`` runs some action from a test case, the execution of ``hs-test`` inside the container
-is asynchronous. The action might take many seconds to finish, while the test case execution context continues to run.
-To mitigate this, ``hstExec(..)`` waits pre-defined arbitrary number of seconds for a *sync file* to be written by ``hs-test``
-at the end of its run. The test case context and container use Docker volume to share the file.
+Host system connects to VPP instances running in containers using a shared folder
+where binary API socket is accessible by both sides.
**Adding an external tool**
* Linux tools ``ip``, ``brctl``
* Standalone programs ``wget``, ``iperf3`` - since these are downloaded when Docker image is made,
they are reasonably up-to-date automatically
-* Programs in Docker images - see ``envoyproxy/envoy-contrib`` in ``utils.go``
-* ``http_server`` - homegrown application that listens on specified address and sends a test file in response
+* Programs in Docker images - ``envoyproxy/envoy-contrib`` and ``nginx``
+* ``http_server`` - homegrown application that listens on specified port and sends a test file in response
* Non-standard Go libraries - see ``extras/hs-test/go.mod``
Generally, these will be updated on a per-need basis, for example when a bug is discovered
Code Review / vpp.git / blobdiff