.. _virtualenv: http://docs.python-guide.org/en/latest/dev/virtualenvs/
.. _scapy: http://www.secdev.org/projects/scapy/
.. _logging: https://docs.python.org/2/library/logging.html
+.. _process: https://docs.python.org/2/library/multiprocessing.html#the-process-class
+.. _pipes: https://docs.python.org/2/library/multiprocessing.html#multiprocessing.Pipe
+.. _managed: https://docs.python.org/2/library/multiprocessing.html#managers
.. |vtf| replace:: VPP Test Framework
make test V=1 # moderate verbosity
make test V=2 # maximum verbosity
+Parallel test execution
+#######################
+
+|vtf| test suites can be run in parallel. Each test suite is executed
+in a separate process spawned by Python multiprocessing process_.
+
+The results from child test suites are sent to parent through pipes_, which are
+aggregated and summarized at the end of the run.
+
+Stdout, stderr and logs logged in child processes are redirected to individual
+parent managed_ queues. The data from these queues are then emitted to stdout
+of the parent process in the order the test suites have finished. In case there
+are no finished test suites (such as at the beginning of the run), the data
+from last started test suite are emitted in real time.
+
+To enable parallel test run, specify the number of parallel processes:
+
+.. code-block:: shell
+
+ make test TEST_JOBS=n # at most n processes will be spawned
+ make test TEST_JOBS=auto # chosen based on the number of cores
+ # and the size of shared memory
+
Test temporary directory and VPP life cycle
###########################################
* local_ip4n - Local IPv4 address - raw, suitable as API parameter.
These addresses need to be configured in VPP to be usable using e.g.
-`config_ip4` API. Please see the documentation to `VppInterface` for more
-details.
+`VppInterface.config_ip4` API. Please see the documentation to
+`VppInterface` for more details.
By default, there is one remote address of each kind created for L3:
remote_ip4 and remote_ip6. If the test needs more addresses, because it's
Packets are written into a temporary .pcap file, which is then read by the VPP
and the packets are injected into the VPP world.
-To add a list of packets to an interface, call the `add_stream` method on that
-interface. Once everything is prepared, call `pg_start` method to start
-the packet generator on the VPP side.
+To add a list of packets to an interface, call the `VppPGInterface.add_stream`
+method on that interface. Once everything is prepared, call `pg_start` method to
+start the packet generator on the VPP side.
VPP -> test framework
~~~~~~~~~~~~~~~~~~~~~
The following APIs are available to the test case for reading pcap files.
-* `get_capture`: this API is suitable for bulk & batch style of test, where
- a list of packets is prepared & sent, then the received packets are read
- and verified. The API needs the number of packets which are expected to
- be captured (ignoring filtered packets - see below) to know when the pcap
- file is completely written by the VPP. If using packet infos for verifying
- packets, then the counts of the packet infos can be automatically used
- by `get_capture` to get the proper count (in this case the default value
- None can be supplied as expected_count or ommitted altogether).
-* `wait_for_packet`: this API is suitable for interactive style of test,
- e.g. when doing session management, three-way handsakes, etc. This API waits
- for and returns a single packet, keeping the capture file in place
- and remembering context. Repeated invocations return following packets
- (or raise Exception if timeout is reached) from the same capture file
- (= packets arriving on the same interface).
-
-*NOTE*: it is not recommended to mix these APIs unless you understand how they
-work internally. None of these APIs rotate the pcap capture file, so calling
-e.g. `get_capture` after `wait_for_packet` will return already read packets.
-It is safe to switch from one API to another after calling `enable_capture`
-as that API rotates the capture file.
+* `VppPGInterface.get_capture`: this API is suitable for bulk & batch
+ style of test, where a list of packets is prepared & sent, then the
+ received packets are read and verified. The API needs the number of
+ packets which are expected to be captured (ignoring filtered
+ packets - see below) to know when the pcap file is completely
+ written by the VPP. If using packet infos for verifying packets,
+ then the counts of the packet infos can be automatically used by
+ `VppPGInterface.get_capture` to get the proper count (in this case
+ the default value None can be supplied as expected_count or ommitted
+ altogether).
+* `VppPGInterface.wait_for_packet`: this API is suitable for
+ interactive style of test, e.g. when doing session management,
+ three-way handshakes, etc. This API waits for and returns a single
+ packet, keeping the capture file in place and remembering
+ context. Repeated invocations return following packets (or raise
+ Exception if timeout is reached) from the same capture file (=
+ packets arriving on the same interface).
+
+*NOTE*: it is not recommended to mix these APIs unless you understand
+how they work internally. None of these APIs rotate the pcap capture
+file, so calling e.g. `VppPGInterface.get_capture` after
+`VppPGInterface.wait_for_packet` will return already read packets. It
+is safe to switch from one API to another after calling
+`VppPGInterface.enable_capture` as that API rotates the capture file.
Automatic filtering of packets:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Both APIs (`get_capture` and `wait_for_packet`) by default filter the packet
-capture, removing known uninteresting packets from it - these are IPv6 Router
-Advertisments and IPv6 Router Alerts. These packets are unsolicitated
-and from the point of |vtf| are random. If a test wants to receive these
-packets, it should specify either None or a custom filtering function
-as the value to the 'filter_out_fn' argument.
+Both APIs (`VppPGInterface.get_capture` and
+`VppPGInterface.wait_for_packet`) by default filter the packet
+capture, removing known uninteresting packets from it - these are IPv6
+Router Advertisments and IPv6 Router Alerts. These packets are
+unsolicitated and from the point of |vtf| are random. If a test wants
+to receive these packets, it should specify either None or a custom
+filtering function as the value to the 'filter_out_fn' argument.
Common API flow for sending/receiving packets:
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We will describe a simple scenario, where packets are sent from pg0 to pg1
interface, assuming that the interfaces were created using
class IP4FwdTestCase(VppTestCase):
""" IPv4 simple forwarding test case """
-2. Add a setUpClass function containing the setup needed for our test to run::
+3. Add a setUpClass function containing the setup needed for our test to run::
@classmethod
def setUpClass(self):
i.config_ip4() # configure IPv4 address on the interface
i.resolve_arp() # resolve ARP, so that we know VPP MAC
-3. Create a helper method to create the packets to send::
+4. Create a helper method to create the packets to send::
def create_stream(self, src_if, dst_if, count):
packets = []
# return the created packet list
return packets
-4. Create a helper method to verify the capture::
+5. Create a helper method to verify the capture::
def verify_capture(self, src_if, dst_if, capture):
packet_info = None
ip = packet[IP]
udp = packet[UDP]
# convert the payload to packet info object
- payload_info = self.payload_to_info(str(packet[Raw]))
+ payload_info = self.payload_to_info(packet[Raw])
# make sure the indexes match
self.assert_equal(payload_info.src, src_if.sw_if_index,
"source sw_if_index")
"Interface %s: Packet expected from interface "
"%s didn't arrive" % (dst_if.name, src_if.name))
-5. Add the test code to test_basic function::
+6. Add the test code to test_basic function::
def test_basic(self):
count = 10
# verify capture
self.verify_capture(self.pg0, self.pg1, capture)
-6. Run the test by issuing 'make test'.
+7. Run the test by issuing 'make test' or, to run only this specific
+ test, issue 'make test TEST=test_ip4_fwd'.
Code Review / vpp.git / blobdiff