-# Copyright (c) 2018 Cisco and/or its affiliates.
+# Copyright (c) 2019 Cisco and/or its affiliates.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at:
| Library | resources.libraries.python.DUTSetup
| Library | resources.tools.wrk.wrk
| Resource | resources/libraries/robot/performance/performance_configuration.robot
+| Resource | resources/libraries/robot/performance/performance_limits.robot
| Resource | resources/libraries/robot/performance/performance_utils.robot
| Resource | resources/libraries/robot/tcp/tcp_setup.robot
| Documentation | Performance suite keywords - Suite and test setups and
| | ... | _NOTE:_ This KW sets following suite variables:
| | ... | - tg - TG node
| | ... | - tg_if1 - 1st TG interface towards DUT.
+| | ... | - tg_if1 - 1st TG interface MAC address.
| | ... | - tg_if2 - 2nd TG interface towards DUT.
+| | ... | - tg_if2 - 2nd TG interface MAC address.
| | ... | - dut1 - DUT1 node
| | ... | - dut1_if1 - 1st DUT interface towards TG.
| | ... | - dut1_if2 - 2nd DUT interface towards TG.
| | ${dut1_if1} | ${dut1}= | First Ingress Interface
| | ${dut1_if2} | ${dut1}= | Last Egress Interface
| | ${tg_if2} | ${tg}= | Last Interface
+| | ${tg_if1_mac}= | Get Interface MAC | ${tg} | ${tg_if1}
+| | ${tg_if2_mac}= | Get Interface MAC | ${tg} | ${tg_if2}
| | Set Suite Variable | ${tg}
| | Set Suite Variable | ${tg_if1}
+| | Set Suite Variable | ${tg_if1_mac}
| | Set Suite Variable | ${tg_if2}
+| | Set Suite Variable | ${tg_if2_mac}
| | Set Suite Variable | ${dut1}
| | Set Suite Variable | ${dut1_if1}
| | Set Suite Variable | ${dut1_if2}
| | ...
| | ... | _NOTE:_ This KW sets following suite variables:
| | ... | - tg - TG node
-| | ... | - tg_if1 - TG interface towards DUT1.
-| | ... | - tg_if2 - TG interface towards DUT2.
+| | ... | - tg_if1 - 1st TG interface towards DUT.
+| | ... | - tg_if1 - 1st TG interface MAC address.
+| | ... | - tg_if2 - 2nd TG interface towards DUT.
+| | ... | - tg_if2 - 2nd TG interface MAC address.
| | ... | - dut1 - DUT1 node
| | ... | - dut1_if1 - DUT1 interface towards TG.
| | ... | - dut1_if2 - DUT1 interface towards DUT2.
| | ${dut2_if1} | ${dut2}= | Next Interface
| | ${dut2_if2} | ${dut2}= | Next Interface
| | ${tg_if2} | ${tg}= | Next Interface
+| | ${tg_if1_mac}= | Get Interface MAC | ${tg} | ${tg_if1}
+| | ${tg_if2_mac}= | Get Interface MAC | ${tg} | ${tg_if2}
| | Set Suite Variable | ${tg}
| | Set Suite Variable | ${tg_if1}
+| | Set Suite Variable | ${tg_if1_mac}
| | Set Suite Variable | ${tg_if2}
+| | Set Suite Variable | ${tg_if2_mac}
| | Set Suite Variable | ${dut1}
| | Set Suite Variable | ${dut1_if1}
| | Set Suite Variable | ${dut1_if2}
| | ...
| | ... | _NOTE:_ This KW sets following suite variables:
| | ... | - tg - TG node
-| | ... | - tg_if1 - TG interface towards DUT1.
-| | ... | - tg_if2 - TG interface towards DUT2.
+| | ... | - tg_if1 - 1st TG interface towards DUT.
+| | ... | - tg_if1 - 1st TG interface MAC address.
+| | ... | - tg_if2 - 2nd TG interface towards DUT.
+| | ... | - tg_if2 - 2nd TG interface MAC address.
| | ... | - dut1 - DUT1 node
| | ... | - dut1_if1 - DUT1 interface towards TG.
| | ... | - dut1_if2_1 - DUT1 interface 1 towards DUT2.
| | ${dut1_if2_2} | ${dut1}= | Last Interface
| | ${dut2_if1_1} | ${dut2}= | First Ingress Interface
| | ${dut2_if1_2} | ${dut2}= | Last Egress Interface
+| | ${tg_if1_mac}= | Get Interface MAC | ${tg} | ${tg_if1}
+| | ${tg_if2_mac}= | Get Interface MAC | ${tg} | ${tg_if2}
| | # Set suite variables
| | Set Suite Variable | ${tg}
| | Set Suite Variable | ${tg_if1}
+| | Set Suite Variable | ${tg_if1_mac}
| | Set Suite Variable | ${tg_if2}
+| | Set Suite Variable | ${tg_if2_mac}
| | Set Suite Variable | ${dut1}
| | Set Suite Variable | ${dut1_if1}
| | Set Suite Variable | ${dut1_if2_1}
| | Set Suite Variable | ${dut2_if1_2}
| | Set Suite Variable | ${dut2_if2}
+| Tear down guest VM
+| | [Documentation]
+| | ... | Stop all qemu processes running on nodes.
+| | ...
+| | Run Keyword | vnf_manager.Kill All VMs
+
| Tear down guest VM with dpdk-testpmd
| | [Documentation]
| | ... | Stop all qemu processes with dpdk-testpmd running on ${dut_node}.
| | | ${vm}= | Get From Dictionary | ${dut_vm_refs} | ${vm_name}
| | | ${index}= | Evaluate | ${index} + 1
| | | Dpdk Testpmd Stop | ${vm}
-| | | Run Keyword | ${vm_name}.Qemu Set Node | ${dut_node}
-| | | Run Keyword | ${vm_name}.Qemu Clear Socks
-| | | Run Keyword If | '${index}' == '${vms_number}' | ${vm_name}.Qemu Kill All
-
-| Tear down guest VM
-| | [Documentation]
-| | ... | Stop all qemu processes running on ${dut_node}.
-| | ... | Argument is dictionary of all qemu nodes running with its names.
-| | ...
-| | ... | *Arguments:*
-| | ... | - dut_node - Node where to clean qemu. Type: dictionary
-| | ... | - dut_vm_refs - VM references on node. Type: dictionary
-| | ...
-| | ... | *Example:*
-| | ...
-| | ... | \| Tear down guest VM \| ${node['DUT1']} \
-| | ... | \| ${dut_vm_refs} \|
-| | ...
-| | [Arguments] | ${dut_node} | ${dut_vm_refs}
-| | ${vms_number}= | Get Length | ${dut_vm_refs}
-| | ${index}= | Set Variable | ${0}
-| | :FOR | ${vm_name} | IN | @{dut_vm_refs}
-| | | ${vm}= | Get From Dictionary | ${dut_vm_refs} | ${vm_name}
-| | | ${index}= | Evaluate | ${index} + 1
-| | | Run Keyword | ${vm_name}.Qemu Set Node | ${dut_node}
-| | | Run Keyword | ${vm_name}.Qemu Clear Socks
| | | Run Keyword If | '${index}' == '${vms_number}' | ${vm_name}.Qemu Kill All
# Suite setups
| | ... | argument. Initializes traffic generator.
| | ...
| | ... | *Arguments:*
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - nic_model - Interface model. Type: string
+| | ... | - osi_layer - OSI Layer type to initialize TG with. Type: string
+| | ... | - nic_name - Interface model. Type: string
| | ...
| | ... | *Example:*
| | ...
| | ... | \| 2-node Performance Suite Setup \| L2 \| Intel-X520-DA2 \|
| | ...
-| | [Arguments] | ${topology_type} | ${nic_model}
+| | [Arguments] | ${osi_layer} | ${nic_name}
| | ...
| | Set variables in 2-node circular topology with DUT interface model
-| | ... | ${nic_model}
+| | ... | ${nic_name}
| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
-| | ... | ${dut1} | ${dut1_if1} | ${dut1} | ${dut1_if2} | ${topology_type}
+| | ... | ${dut1} | ${dut1_if1} | ${dut1} | ${dut1_if2} | ${osi_layer}
| Set up 2-node-switched performance topology with DUT's NIC model
| | [Documentation]
| | ... | argument. Initializes traffic generator.
| | ...
| | ... | *Arguments:*
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - nic_model - Interface model. Type: string
+| | ... | - osi_layer - OSI Layer type to initialize TG with. Type: string
+| | ... | - nic_name - Interface model. Type: string
| | ... | - tg_if1_dest_mac - Interface 1 destination MAC address. Type: string
| | ... | - tg_if2_dest_mac - Interface 2 destination MAC address. Type: string
| | ...
| | ... | \| 2-node Performance Suite Setup \| L2 \| Intel-X520-DA2 \
| | ... | \| 22:22:33:44:55:66 \| 22:22:33:44:55:55 \|
| | ...
-| | [Arguments] | ${topology_type} | ${nic_model} | ${tg_if1_dest_mac}
+| | [Arguments] | ${osi_layer} | ${nic_name} | ${tg_if1_dest_mac}
| | ... | ${tg_if2_dest_mac}
| | ...
| | Set variables in 2-node circular topology with DUT interface model
-| | ... | ${nic_model}
+| | ... | ${nic_name}
| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
-| | ... | ${dut1} | ${dut1_if1} | ${dut1} | ${dut1_if2} | ${topology_type}
+| | ... | ${dut1} | ${dut1_if1} | ${dut1} | ${dut1_if2} | ${osi_layer}
| | ... | ${tg_if1_dest_mac} | ${tg_if2_dest_mac}
| Set up 3-node performance topology with DUT's NIC model
| | ... | argument. Initializes traffic generator.
| | ...
| | ... | *Arguments:*
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - nic_model - Interface model. Type: string
+| | ... | - osi_layer - OSI Layer type to initialize TG with. Type: string
+| | ... | - nic_name - Interface model. Type: string
| | ...
| | ... | *Example:*
| | ...
| | ... | \| Set up 3-node performance topology with DUT's NIC model \| L2 \
| | ... | \| Intel-X520-DA2 \|
| | ...
-| | [Arguments] | ${topology_type} | ${nic_model}
+| | [Arguments] | ${osi_layer} | ${nic_name}
| | ...
| | Set variables in 3-node circular topology with DUT interface model
-| | ... | ${nic_model}
+| | ... | ${nic_name}
| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
-| | ... | ${dut1} | ${dut1_if1} | ${dut2} | ${dut2_if2} | ${topology_type}
+| | ... | ${dut1} | ${dut1_if1} | ${dut2} | ${dut2_if2} | ${osi_layer}
| Set up 3-node performance topology with DUT's NIC model with double link between DUTs
| | [Documentation]
| | ... | argument. Initializes traffic generator.
| | ...
| | ... | *Arguments:*
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - nic_model - Interface model. Type: string
+| | ... | - osi_layer - OSI Layer type to initialize TG with. Type: string
+| | ... | - nic_name - Interface model. Type: string
| | ...
| | ... | *Example:*
| | ...
| | ... | \| Set up 3-node performance topology with DUT's NIC model with \
| | ... | double link between DUTs \| L2 \| Intel-X520-DA2 \|
| | ...
-| | [Arguments] | ${topology_type} | ${nic_model}
+| | [Arguments] | ${osi_layer} | ${nic_name}
| | ...
| | Set variables in 3-node circular topology with DUT interface model with double link between DUTs
-| | ... | ${nic_model}
+| | ... | ${nic_name}
| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
-| | ... | ${dut1} | ${dut1_if1} | ${dut2} | ${dut2_if2} | ${topology_type}
+| | ... | ${dut1} | ${dut1_if1} | ${dut2} | ${dut2_if2} | ${osi_layer}
| Set up DPDK 2-node performance topology with DUT's NIC model
| | [Documentation]
| | ... | environment.
| | ...
| | ... | *Arguments:*
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - nic_model - Interface model. Type: string
+| | ... | - osi_layer - OSI Layer type to initialize TG with. Type: string
+| | ... | - nic_name - Interface model. Type: string
| | ...
| | ... | *Example:*
| | ...
| | ... | \| Set up DPDK 2-node performance topology with DUT's NIC model \
| | ... | \| L2 \| Intel-X520-DA2 \|
| | ...
-| | [Arguments] | ${topology_type} | ${nic_model}
+| | [Arguments] | ${osi_layer} | ${nic_name}
| | ...
| | Set variables in 2-node circular topology with DUT interface model
-| | ... | ${nic_model}
+| | ... | ${nic_name}
| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
-| | ... | ${dut1} | ${dut1_if1} | ${dut1} | ${dut1_if2} | ${topology_type}
+| | ... | ${dut1} | ${dut1_if1} | ${dut1} | ${dut1_if2} | ${osi_layer}
| | Initialize DPDK Environment | ${dut1} | ${dut1_if1} | ${dut1_if2}
| Set up DPDK 3-node performance topology with DUT's NIC model
| | ... | environment.
| | ...
| | ... | *Arguments:*
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - nic_model - Interface model. Type: string
+| | ... | - osi_layer - OSI Layer type to initialize TG with. Type: string
+| | ... | - nic_name - Interface model. Type: string
| | ...
| | ... | *Example:*
| | ...
| | ... | \| 3-node Performance Suite Setup \| L2 \| Intel-X520-DA2 \|
| | ...
-| | [Arguments] | ${topology_type} | ${nic_model}
+| | [Arguments] | ${osi_layer} | ${nic_name}
| | ...
| | Set variables in 3-node circular topology with DUT interface model
-| | ... | ${nic_model}
+| | ... | ${nic_name}
| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
-| | ... | ${dut1} | ${dut1_if1} | ${dut2} | ${dut2_if2} | ${topology_type}
+| | ... | ${dut1} | ${dut1_if1} | ${dut2} | ${dut2_if2} | ${osi_layer}
| | Initialize DPDK Environment | ${dut1} | ${dut1_if1} | ${dut1_if2}
| | Initialize DPDK Environment | ${dut2} | ${dut2_if1} | ${dut2_if2}
| | ... | It configures PCI device with VFs on all DUTs.
| | ...
| | ... | *Arguments:*
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - nic_model - Interface model. Type: string
+| | ... | - osi_layer - OSI Layer type to initialize TG with. Type: string
+| | ... | - nic_name - Interface model. Type: string
| | ... | - vf_driver - Virtual function driver. Type: string
| | ... | - numvfs - Number of VFs. Type: integer
| | ...
| | ... | \| Set up SRIOV 2-node performance topology with DUT's NIC model \
| | ... | \| L2 \| Intel-X520-DA2 \| AVF \|
| | ...
-| | [Arguments] | ${topology_type} | ${nic_model} | ${vf_driver}
+| | [Arguments] | ${osi_layer} | ${nic_name} | ${vf_driver}
| | ... | ${numvfs}=${1}
| | ...
| | Set variables in 2-node circular topology with DUT interface model
-| | ... | ${nic_model}
+| | ... | ${nic_name}
| | Run Keyword If | '${vf_driver}' == 'AVF'
| | ... | Configure AVF interfaces on all DUTs | numvfs=${numvfs}
-| | ... | topology_type=${topology_type}
+| | ... | osi_layer=${osi_layer}
| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
| | ... | ${dut1} | ${dut1_if1_vf0} | ${dut1} | ${dut1_if2_vf0}
-| | ... | ${topology_type}
+| | ... | ${osi_layer}
| Set up SRIOV 3-node performance topology with DUT's NIC model
| | [Documentation]
| | ... | It configures PCI device with VFs on all DUTs.
| | ...
| | ... | *Arguments:*
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - nic_model - Interface model. Type: string
+| | ... | - osi_layer - OSI Layer type to initialize TG with. Type: string
+| | ... | - nic_name - Interface model. Type: string
| | ... | - vf_driver - Virtual function driver. Type: string
| | ... | - numvfs - Number of VFs. Type: integer
| | ...
| | ... | \| Set up SRIOV 3-node performance topology with DUT's NIC model \
| | ... | \| L2 \| Intel-X520-DA2 \| AVF \|
| | ...
-| | [Arguments] | ${topology_type} | ${nic_model} | ${vf_driver}
+| | [Arguments] | ${osi_layer} | ${nic_name} | ${vf_driver}
| | ... | ${numvfs}=${1}
| | ...
| | Set variables in 3-node circular topology with DUT interface model
-| | ... | ${nic_model}
+| | ... | ${nic_name}
| | Run Keyword If | '${vf_driver}' == 'AVF'
| | ... | Configure AVF interfaces on all DUTs | numvfs=${numvfs}
-| | ... | topology_type=${topology_type}
+| | ... | osi_layer=${osi_layer}
| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
| | ... | ${dut1} | ${dut1_if1_vf0} | ${dut2} | ${dut2_if2_vf0}
-| | ... | ${topology_type}
+| | ... | ${osi_layer}
| Set up IPSec performance test suite
| | [Documentation]
| | ... | argument. Initializes traffic generator.
| | ... | Then it configures crypto device and kernel module on all DUTs.
| | ...
+| | ... | TODO CSIT-1481: Crypto HW should be read from topology file instead.
+| | ...
| | ... | *Arguments:*
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - nic_model - Interface model. Type: string
-| | ... | - crypto_type - Crypto device type - HW_cryptodev or SW_cryptodev
-| | ... | (Optional). Type: string, default value: HW_cryptodev
+| | ... | - osi_layer - OSI Layer type to initialize TG with. Type: string
+| | ... | - nic_name - Interface model. Type: string
+| | ... | - crypto_type - Crypto device type - HW_DH895xcc or HW_C3xxx or
+| | ... | SW_cryptodev. Type: string, default value: HW_DH895xcc
| | ...
| | ... | *Example:*
| | ...
| | ... | \| Set up IPSec performance test suite \| L2 \
-| | ... | \| Intel-X520-DA2 \|
+| | ... | \| Intel-X520-DA2 \| HW_DH895xcc \|
| | ...
-| | [Arguments] | ${topology_type} | ${nic_model} | ${crypto_type}=HW_cryptodev
+| | [Arguments] | ${osi_layer} | ${nic_name} | ${crypto_type}=HW_DH895xcc
| | ...
| | Set up 3-node performance topology with DUT's NIC model
-| | ... | ${topology_type} | ${nic_model}
+| | ... | ${osi_layer} | ${nic_name}
+| | Return From Keyword If | '${crypto_type}' == 'SW_cryptodev'
| | ${numvfs}= | Set Variable If
-| | ... | '${crypto_type}' == 'HW_cryptodev' | ${32}
-| | ... | '${crypto_type}' == 'SW_cryptodev' | ${0}
-| | Configure crypto device on all DUTs | force_init=${True} | numvfs=${numvfs}
-| | Run Keyword If | '${crypto_type}' == 'HW_cryptodev'
-| | ... | Configure kernel module on all DUTs | vfio_pci | force_load=${True}
-
-| Set up performance topology with containers
-| | [Documentation]
-| | ... | Suite preparation phase that starts containers
-| | ...
-| | ... | *Arguments:*
-| | ... | - chains: Total number of chains. Type: integer
-| | ... | - nodeness: Total number of nodes per chain. Type: integer
-| | ...
-| | ... | *Example:*
-| | ...
-| | ... | \| Set up performance topology with containers \| 1 \| 1 \|
-| | ...
-| | [Arguments] | ${chains}=${1} | ${nodeness}=${1}
-| | ...
-| | Set Suite Variable | @{container_groups} | @{EMPTY}
-| | Construct chains of containers on all DUTs | ${chains} | ${nodeness}
-| | Acquire all 'CNF' containers
-| | Create all 'CNF' containers
-| | Configure VPP in all 'CNF' containers
-| | Stop VPP service on all DUTs | ${nodes}
-| | Install VPP in all 'CNF' containers
-| | Start VPP service on all DUTs | ${nodes}
+| | ... | '${crypto_type}' == 'HW_DH895xcc' | ${32}
+| | ... | '${crypto_type}' == 'HW_C3xxx' | ${16}
+| | Configure crypto device on all DUTs | ${crypto_type} | numvfs=${numvfs}
+| | ... | force_init=${True}
+| | Configure kernel module on all DUTs | vfio_pci | force_load=${True}
| Set up performance test suite with MEMIF
| | [Documentation]
| | Set Suite Variable | @{plugins_to_enable}
| | Append To List | ${plugins_to_enable} | lacp_plugin.so
+| Set up performance test suite with crypto ipsecmb
+| | [Documentation]
+| | ... | Append crypto_ipsecmb_plugin.so to the list of enabled plugins.
+| | ...
+| | Set Suite Variable | @{plugins_to_enable}
+| | Append To List | ${plugins_to_enable} | crypto_ia32_plugin.so
+| | Append To List | ${plugins_to_enable} | crypto_ipsecmb_plugin.so
+| | Append To List | ${plugins_to_enable} | crypto_openssl_plugin.so
+
| Set up 3-node performance topology with wrk and DUT's NIC model
| | [Documentation]
| | ... | Suite preparation phase that sets the default startup configuration of
| | ...
| | Teardown traffic generator | ${tg}
-| Tear down 2-node performance topology with container
-| | [Documentation]
-| | ... | Suite teardown phase with traffic generator teardown and container
-| | ... | destroy.
-| | ...
-| | Teardown traffic generator | ${tg}
-| | :FOR | ${group} | IN | @{container_groups}
-| | | Destroy all '${group}' containers
-
| Tear down 3-node performance topology
| | [Documentation]
| | ... | Suite teardown phase with traffic generator teardown.
| | ...
| | Teardown traffic generator | ${tg}
-| Tear down 3-node performance topology with container
-| | [Documentation]
-| | ... | Suite teardown phase with traffic generator teardown and container
-| | ... | destroy.
-| | ...
-| | Teardown traffic generator | ${tg}
-| | :FOR | ${group} | IN | @{container_groups}
-| | | Destroy all '${group}' containers
-
# Tests setups
| Set up performance test
| | [Documentation] | Common test setup for performance tests.
| | ...
-| | Reset VAT History On All DUTs | ${nodes}
+| | Reset PAPI History On All DUTs | ${nodes}
| | Create base startup configuration of VPP on all DUTs
| Set up tcp performance test
| | [Documentation] | Common test setup for TCP performance tests.
| | ...
-| | Reset VAT History On All DUTs | ${nodes}
+| | Reset PAPI History On All DUTs | ${nodes}
| | Create base startup configuration of VPP for TCP tests on all DUTs
| Set up performance test with Ligato Kubernetes
| | Apply Kubernetes resource on all duts | ${nodes}
| | ... | pods/contiv-vswitch.yaml
-# Tests teardowns
-
-| Tear down performance discovery test
-| | [Documentation] | Common test teardown for ndrdisc and pdrdisc performance \
-| | ... | tests.
+| Set up performance test with containers
+| | [Documentation]
+| | ... | Common test setup for performance tests with containers
| | ...
| | ... | *Arguments:*
-| | ... | - rate - Rate for sending packets. Type: string
-| | ... | - framesize - L2 Frame Size [B]. Type: integer
-| | ... | - topology_type - Topology type. Type: string
+| | ... | - chains: Total number of chains. Type: integer
+| | ... | - nodeness: Total number of nodes per chain. Type: integer
+| | ... | - auto_scale - If True, use same amount of Dataplane threads for
+| | ... | network function as DUT, otherwise use single physical core for
+| | ... | every network function. Type: boolean
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Tear down performance discovery test \| 4.0mpps \| 64 \
-| | ... | \| 3-node-IPv4 \|
+| | ... | \| Set up performance test with containers \| 1 \| 1 \|
| | ...
-| | [Arguments] | ${rate} | ${framesize} | ${topology_type}
+| | [Arguments] | ${nf_chains}=${1} | ${nf_nodes}=${1} | ${auto_scale}=${True}
| | ...
-| | Remove All Added Ports On All DUTs From Topology | ${nodes}
-| | Show VAT History On All DUTs | ${nodes}
-| | Run Keyword If Test Failed
-| | ... | Set Test Variable | ${pkt_trace} | ${True}
-| | Run Keyword If Test Failed
-| | ... | Traffic should pass with no loss | ${perf_trial_duration} | ${rate}
-| | ... | ${framesize} | ${topology_type} | fail_on_loss=${False}
+| | Set Test Variable | @{container_groups} | @{EMPTY}
+| | Set Test Variable | ${container_group} | CNF
+| | Set Test Variable | ${nf_nodes}
+| | Import Library | resources.libraries.python.ContainerUtils.ContainerManager
+| | ... | engine=${container_engine} | WITH NAME | ${container_group}
+| | Construct chains of containers on all DUTs | ${nf_chains} | ${nf_nodes}
+| | ... | auto_scale=${auto_scale}
+| | Acquire all '${container_group}' containers
+| | Create all '${container_group}' containers
+| | Configure VPP in all '${container_group}' containers
+| | Stop VPP service on all DUTs | ${nodes}
+| | Start VPP in all '${container_group}' containers
+| | Restart VPP service on all DUTs | ${nodes}
+| | Verify VPP on all DUTs | ${nodes}
+| | Append To List | ${container_groups} | ${container_group}
-| Tear down performance mrr test
-| | [Documentation] | Common test teardown for max-received-rate performance
-| | ... | tests.
-| | ...
-| | Remove All Added Ports On All DUTs From Topology | ${nodes}
-| | Show VAT History On All DUTs | ${nodes}
+# Tests teardowns
-| Tear down performance test with wrk
-| | [Documentation] | Common test teardown for ndrdisc and pdrdisc performance \
-| | ... | tests.
+| Tear down performance test
+| | [Documentation] | Common test teardown for performance tests.
+| | ...
+| | ... | TODO: Rename to "Tear down VPP performance test"?
+| | ... | most steps do not apply to DPDK nor other vswitch solutions.
+| | ... | TODO: Document at least some steps this keyword takes.
+| | ...
+| | ... | To save space in suites, this keyword does not accept
+| | ... | any explicit arguments. Instead, it requires few variables
+| | ... | to be defined in at least test scope.
+| | ...
+| | ... | *Test Variables needed:*
+| | ... | - frame_size - L2 Frame Size [B]. Type: integer
+| | ... | - traffic_profile - Profile name to run debug trial with. Type: string
+| | ... | - nodes - Parsed information object. Type: dict
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Tear down performance test with wrk \|
+| | ... | \| Set Test Variable \| \${frame_size} \| 64 \|
+| | ... | \| Tear down performance test \|
| | ...
| | Remove All Added Ports On All DUTs From Topology | ${nodes}
-| | Show VAT History On All DUTs | ${nodes}
-| | Show statistics on all DUTs | ${nodes}
+| | Show PAPI History On All DUTs | ${nodes}
+| | Get Core Files on All Nodes | ${nodes}
+| | Set Test Variable | ${pkt_trace} | ${True}
+| | Run Keyword If Test Failed
+| | ... | Traffic should pass with no loss | ${perf_trial_duration} | 10000pps
+| | ... | ${frame_size} | ${traffic_profile} | fail_on_loss=${False}
-| Tear down performance test with vhost and VM with dpdk-testpmd
-| | [Documentation] | Common test teardown for performance tests which use
-| | ... | vhost(s) and VM(s) with dpdk-testpmd.
-| | ...
-| | ... | *Arguments:*
-| | ... | - rate - Rate for sending packets. Type: string
-| | ... | - framesize - L2 Frame Size [B]. Type: integer
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - dut1_node - Node where to clean qemu. Type: dictionary
-| | ... | - dut1_vm_refs - VM references on node. Type: dictionary
-| | ... | - dut2_node - Node where to clean qemu. Type: dictionary
-| | ... | - dut2_vm_refs - VM references on node. Type: dictionary
+| Tear down performance test with container
+| | [Documentation]
+| | ... | Common test teardown for performance tests which uses containers.
| | ...
-| | ... | *Example:*
+| | ... | TODO: Call Tear down performance test?
| | ...
-| | ... | \| Tear down performance test with vhost and VM with dpdk-testpmd \
-| | ... | \| 4.0mpps \| 64 \| 3-node-IPv4 \| ${node['DUT1']} \| ${dut_vm_refs} \
-| | ... | \| ${node['DUT2']} \| ${dut_vm_refs} \|
-| | ...
-| | [Arguments] | ${rate} | ${framesize} | ${topology_type}
-| | ... | ${dut1_node}=${None} | ${dut1_vm_refs}=${None}
-| | ... | ${dut2_node}=${None} | ${dut2_vm_refs}=${None}
+| | :FOR | ${container_group} | IN | @{container_groups}
+| | | Destroy all '${container_group}' containers
+
+| Tear down performance test with vhost
+| | [Documentation] | Common test teardown for performance tests which use
+| | ... | vhost(s) and VM(s).
| | ...
-| | Remove All Added Ports On All DUTs From Topology | ${nodes}
-| | Show VAT History On All DUTs | ${nodes}
+| | Tear down performance test
| | Show VPP vhost on all DUTs | ${nodes}
-| | Show statistics on all DUTs | ${nodes}
-| | Run Keyword If Test Failed
-| | ... | Traffic should pass with no loss | ${perf_trial_duration} | ${rate}
-| | ... | ${framesize} | ${topology_type} | fail_on_loss=${False}
-| | Run keyword unless | ${dut1_node}==${None}
-| | ... | Tear down guest VM with dpdk-testpmd | ${dut1} | ${dut1_vm_refs}
-| | Run keyword unless | ${dut2_node}==${None}
-| | ... | Tear down guest VM with dpdk-testpmd | ${dut2} | ${dut2_vm_refs}
+| | Tear down guest VM
-| Tear down mrr test with vhost and VM with dpdk-testpmd
-| | [Documentation] | Common test teardown for mrr tests which use
+| Tear down performance test with vhost and VM with dpdk-testpmd
+| | [Documentation] | Common test teardown for performance tests which use
| | ... | vhost(s) and VM(s) with dpdk-testpmd.
| | ...
+| | ... | *Test Variables needed:*
+| | ... | - frame_size - L2 Frame Size [B]. Type: integer
+| | ... | - traffic_profile - Profile name to run debug trial with. Type: string
+| | ...
| | ... | *Arguments:*
| | ... | - dut1_node - Node where to clean qemu. Type: dictionary
| | ... | - dut1_vm_refs - VM references on node. Type: dictionary
| | [Arguments] | ${dut1_node}=${None} | ${dut1_vm_refs}=${None}
| | ... | ${dut2_node}=${None} | ${dut2_vm_refs}=${None}
| | ...
-| | Remove All Added Ports On All DUTs From Topology | ${nodes}
-| | Show VAT History On All DUTs | ${nodes}
+| | Tear down performance test
| | Show VPP vhost on all DUTs | ${nodes}
-| | Show statistics on all DUTs | ${nodes}
| | Run keyword unless | ${dut1_node}==${None}
| | ... | Tear down guest VM with dpdk-testpmd | ${dut1} | ${dut1_vm_refs}
| | Run keyword unless | ${dut2_node}==${None}
| | [Documentation] | Common test teardown for performance tests which use
| | ... | vhost(s) and VM(s) with ACL and dpdk-testpmd.
| | ...
-| | ... | *Arguments:*
-| | ... | - rate - Rate for sending packets. Type: string
-| | ... | - framesize - L2 Frame Size [B]. Type: integer
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - dut1_node - Node where to clean qemu. Type: dictionary
-| | ... | - dut1_vm_refs - VM references on node. Type: dictionary
-| | ... | - dut2_node - Node where to clean qemu. Type: dictionary
-| | ... | - dut2_vm_refs - VM references on node. Type: dictionary
-| | ...
-| | [Arguments] | ${rate} | ${framesize} | ${topology_type}
-| | ... | ${dut1_node}=${None} | ${dut1_vm_refs}=${None}
-| | ... | ${dut2_node}=${None} | ${dut2_vm_refs}=${None}
-| | ...
-| | Tear down performance test with vhost and VM with dpdk-testpmd
-| | ... | ${rate} | ${framesize} | ${topology_type}
-| | ... | ${dut1_node} | ${dut1_vm_refs}
-| | ... | ${dut2_node} | ${dut2_vm_refs}
-| | Run Keyword If Test Failed | Vpp Log Plugin Acl Settings | ${dut1}
-| | Run Keyword If Test Failed | Run Keyword And Ignore Error
-| | ... | Vpp Log Plugin Acl Interface Assignment | ${dut1}
-
-| Tear down mrr test with vhost and VM with dpdk-testpmd and ACL
-| | [Documentation] | Common test teardown for mrr tests which use
-| | ... | vhost(s) and VM(s) with ACL and dpdk-testpmd.
+| | ... | *Test Variables needed:*
+| | ... | - frame_size - L2 Frame Size [B]. Type: integer
+| | ... | - traffic_profile - Profile name to run debug trial with. Type: string
| | ...
| | ... | *Arguments:*
| | ... | - dut1_node - Node where to clean qemu. Type: dictionary
| | [Arguments] | ${dut1_node}=${None} | ${dut1_vm_refs}=${None}
| | ... | ${dut2_node}=${None} | ${dut2_vm_refs}=${None}
| | ...
-| | Tear down mrr test with vhost and VM with dpdk-testpmd
+| | Tear down performance test with vhost and VM with dpdk-testpmd
| | ... | ${dut1_node} | ${dut1_vm_refs}
| | ... | ${dut2_node} | ${dut2_vm_refs}
| | Run Keyword If Test Failed | Vpp Log Plugin Acl Settings | ${dut1}
| | Run Keyword If Test Failed | Run Keyword And Ignore Error
| | ... | Vpp Log Plugin Acl Interface Assignment | ${dut1}
-| Tear down performance pdrchk test with vhost and VM with dpdk-testpmd
-| | [Documentation] | Common test teardown for performance pdrchk tests which \
-| | ... | use vhost(s) and VM(s) with dpdk-testpmd.
-| | ...
-| | ... | *Arguments:*
-| | ... | - rate - Rate for sending packets. Type: string
-| | ... | - framesize - L2 Frame Size [B]. Type: integer
-| | ... | - topology_type - Topology type. Type: string
-| | ... | - dut1_node - Node where to clean qemu. Type: dictionary
-| | ... | - dut1_vm_refs - VM references on node. Type: dictionary
-| | ... | - dut2_node - Node where to clean qemu. Type: dictionary
-| | ... | - dut2_vm_refs - VM references on node. Type: dictionary
-| | ...
-| | ... | *Example:*
-| | ...
-| | ... | \| Tear down performance pdrchk test with vhost and VM with \
-| | ... | dpdk-testpmd \| 4.0mpps \| 64 \| 3-node-IPv4 \| ${node['DUT1']} \
-| | ... | \| ${dut_vm_refs} \| ${node['DUT2']} \| ${dut_vm_refs} \|
-| | ...
-| | [Arguments] | ${rate} | ${framesize} | ${topology_type}
-| | ... | ${dut1_node}=${None} | ${dut1_vm_refs}=${None}
-| | ... | ${dut2_node}=${None} | ${dut2_vm_refs}=${None}
-| | ...
-| | Remove All Added Ports On All DUTs From Topology | ${nodes}
-| | Show VAT History On All DUTs | ${nodes}
-| | Show VPP vhost on all DUTs | ${nodes}
-| | Show statistics on all DUTs | ${nodes}
-| | Run keyword unless | ${dut1_node}==${None}
-| | ... | Tear down guest VM with dpdk-testpmd | ${dut1} | ${dut1_vm_refs}
-| | Run keyword unless | ${dut2_node}==${None}
-| | ... | Tear down guest VM with dpdk-testpmd | ${dut2} | ${dut2_vm_refs}
-
-| Tear down performance mrr test with vhost and VM with dpdk-testpmd
-| | [Documentation] | Common test teardown for performance mrr tests which \
-| | ... | use vhost(s) and VM(s) with dpdk-testpmd.
-| | ...
-| | ... | *Arguments:*
-| | ... | - dut1_node - Node where to clean qemu. Type: dictionary
-| | ... | - dut1_vm_refs - VM references on node. Type: dictionary
-| | ... | - dut2_node - Node where to clean qemu. Type: dictionary
-| | ... | - dut2_vm_refs - VM references on node. Type: dictionary
-| | ...
-| | ... | *Example:*
-| | ...
-| | ... | \| Tear down performance mrr test with vhost and VM with \
-| | ... | dpdk-testpmd \| ${node['DUT1']} \| ${dut_vm_refs} \| ${node['DUT2']} \
-| | ... | \| ${dut_vm_refs} \|
-| | ...
-| | [Arguments] | ${dut1_node}=${None} | ${dut1_vm_refs}=${None}
-| | ... | ${dut2_node}=${None} | ${dut2_vm_refs}=${None}
-| | ...
-| | Remove All Added Ports On All DUTs From Topology | ${nodes}
-| | Show VAT History On All DUTs | ${nodes}
-| | Show VPP vhost on all DUTs | ${nodes}
-| | Run keyword unless | ${dut1_node}==${None}
-| | ... | Tear down guest VM with dpdk-testpmd | ${dut1} | ${dut1_vm_refs}
-| | Run keyword unless | ${dut2_node}==${None}
-| | ... | Tear down guest VM with dpdk-testpmd | ${dut2} | ${dut2_vm_refs}
-
| Tear down DPDK 2-node performance topology
| | [Documentation]
| | ... | Suite teardown phase with traffic generator teardown.
| | ... | Cleanup DPDK test environment.
| | ...
+| | ... | TODO: Add "Traffic should pass with no loss" with min rate
+| | ... | on failure, as in the VPP test teardown?
+| | ...
| | Teardown traffic generator | ${tg}
| | Cleanup DPDK Environment | ${dut1} | ${dut1_if1} | ${dut1_if2}
| | ... | Suite teardown phase with traffic generator teardown.
| | ... | Cleanup DPDK test environment.
| | ...
+| | ... | TODO: Add "Traffic should pass with no loss" with min rate
+| | ... | on failure, as in the VPP test teardown?
+| | ...
| | Teardown traffic generator | ${tg}
| | Cleanup DPDK Environment | ${dut1} | ${dut1_if1} | ${dut1_if2}
| | Cleanup DPDK Environment | ${dut2} | ${dut2_if1} | ${dut2_if2}
-| Tear down performance discovery test with NAT
-| | [Documentation] | Common test teardown for ndrdisc and pdrdisc performance \
+| Tear down performance test with NAT
+| | [Documentation] | Common test teardown for performance \
| | ... | tests with NAT feature used.
| | ...
-| | ... | *Arguments:*
-| | ... | - rate - Rate for sending packets. Type: string
-| | ... | - framesize - L2 Frame Size [B]. Type: integer
-| | ... | - traffic_profile - Traffic profile. Type: string
+| | ... | *Test Variables needed:*
+| | ... | - frame_size - L2 Frame Size [B]. Type: integer
+| | ... | - traffic_profile - Profile name to run debug trial with. Type: string
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Tear down performance discovery test with NAT \| 100000pps \| 64 \
-| | ... | \| ${traffic_profile} \|
+| | ... | \| Tear down performance test with NAT \|
| | ...
-| | [Arguments] | ${rate} | ${framesize} | ${traffic_profile}
-| | ...
-| | Tear down performance discovery test | ${rate} | ${framesize}
-| | ... | ${traffic_profile}
-| | Show NAT verbose | ${dut1}
-| | Show NAT verbose | ${dut2}
-
-| Tear down mrr test with NAT
-| | [Documentation] | Common test teardown for mrr performance \
-| | ... | tests with NAT feature used.
-| | ...
-| | ... | \| Tear down mrr test with NAT \|
-| | ...
-| | Tear down performance mrr test
+| | Tear down performance test
| | Show NAT verbose | ${dut1}
| | Show NAT verbose | ${dut2}
| Tear down performance test with ACL
-| | [Documentation] | Common test teardown for ndrdisc and pdrdisc performance \
+| | [Documentation] | Common test teardown for performance \
| | ... | tests with ACL feature used.
| | ...
-| | ... | *Arguments:*
-| | ... | - rate - Rate for sending packets. Type: string
-| | ... | - framesize - L2 Frame Size [B]. Type: integer
-| | ... | - traffic_profile - Traffic profile. Type: string
-| | ...
-| | ... | *Example:*
-| | ...
-| | ... | \| Tear down performance test with ACL \| 100000pps \| 64 \
-| | ... | \| ${traffic_profile} \|
-| | ...
-| | [Arguments] | ${rate} | ${framesize} | ${traffic_profile}
-| | ...
-| | Tear down performance discovery test | ${rate} | ${framesize}
-| | ... | ${traffic_profile}
-| | Run Keyword If Test Failed | Vpp Log Plugin Acl Settings | ${dut1}
-| | Run Keyword If Test Failed | Run Keyword And Ignore Error
-| | ... | Vpp Log Plugin Acl Interface Assignment | ${dut1}
-
-| Tear down mrr test with ACL
-| | [Documentation] | Common test teardown for mrr performance \
-| | ... | tests with ACL feature used.
+| | ... | *Test Variables needed:*
+| | ... | - frame_size - L2 Frame Size [B]. Type: integer
+| | ... | - traffic_profile - Profile name to run debug trial with. Type: string
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Tear down mrr test with ACL \|
+| | ... | \| Tear down performance test with ACL \|
| | ...
-| | Tear down performance mrr test
+| | Tear down performance test
| | Run Keyword If Test Failed | Vpp Log Plugin Acl Settings | ${dut1}
| | Run Keyword If Test Failed | Run Keyword And Ignore Error
| | ... | Vpp Log Plugin Acl Interface Assignment | ${dut1}
| Tear down performance test with MACIP ACL
-| | [Documentation] | Common test teardown for ndrdisc and pdrdisc performance \
+| | [Documentation] | Common test teardown for performance \
| | ... | tests with MACIP ACL feature used.
| | ...
-| | ... | *Arguments:*
-| | ... | - rate - Rate for sending packets. Type: string
-| | ... | - framesize - L2 Frame Size [B]. Type: integer
-| | ... | - traffic_profile - Traffic profile. Type: string
+| | ... | *Test Variables needed:*
+| | ... | - frame_size - L2 Frame Size [B]. Type: integer
+| | ... | - traffic_profile - Profile name to run debug trial with. Type: string
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Tear down performance test with MACIP ACL \| 100000pps \| 64 \
-| | ... | \| ${traffic_profile} \|
-| | ...
-| | [Arguments] | ${rate} | ${framesize} | ${traffic_profile}
+| | ... | \| Tear down performance test with MACIP ACL \|
| | ...
-| | Tear down performance discovery test | ${rate} | ${framesize}
-| | ... | ${traffic_profile}
-| | Run Keyword If Test Failed | Run Keyword And Ignore Error
-| | ... | Vpp Log Macip Acl Settings | ${dut1}
-| | Run Keyword And Ignore Error
-| | ... | Vpp Log Macip Acl Interface Assignment | ${dut1}
-
-| Tear down mrr test with MACIP ACL
-| | [Documentation] | Common test teardown for mrr performance \
-| | ... | tests with MACIP ACL feature used.
-| | ...
-| | ... | *Example:*
-| | ...
-| | ... | \| Tear down mrr test with MACIP ACL \|
-| | ...
-| | Tear down performance mrr test
+| | Tear down performance test
| | Run Keyword If Test Failed | Run Keyword And Ignore Error
| | ... | Vpp Log Macip Acl Settings | ${dut1}
| | Run Keyword And Ignore Error
| | ... | Vpp Log Macip Acl Interface Assignment | ${dut1}
| Tear down performance test with Ligato Kubernetes
-| | [Documentation] | Common test teardown for ndrdisc and pdrdisc performance \
+| | [Documentation] | Common test teardown for performance \
| | ... | tests with Ligato Kubernetes.
| | ...
+| | ... | TODO: Call Tear down performance test?
+| | ...
| | Run Keyword If Test Failed
| | ... | Get Kubernetes logs on all DUTs | ${nodes} | csit
| | Run Keyword If Test Failed
| | Delete Kubernetes resource on all DUTs | ${nodes} | csit
| Tear down performance test with SRv6 with encapsulation
-| | [Documentation] | Common test teardown for ndrdisc and pdrdisc performance \
+| | [Documentation] | Common test teardown for performance \
| | ... | tests with SRv6 with encapsulation feature used.
| | ...
-| | ... | *Arguments:*
-| | ... | - rate - Rate for sending packets. Type: string
-| | ... | - framesize - L2 Frame Size [B]. Type: integer/string
-| | ... | - traffic_profile - Traffic profile. Type: string
-| | ...
-| | ... | *Example:*
-| | ...
-| | ... | \| Tear down performance test with SRv6 with encapsulation \
-| | ... | \| 100000pps \| 64 \| ${traffic_profile} \|
-| | ...
-| | [Arguments] | ${rate} | ${framesize} | ${traffic_profile}
-| | ...
-| | Tear down performance discovery test | ${rate} | ${framesize}
-| | ... | ${traffic_profile}
-| | Run Keyword If Test Failed | Show SR Policies on all DUTs | ${nodes}
-| | Run Keyword If Test Failed
-| | ... | Show SR Steering Policies on all DUTs | ${nodes}
-| | Run Keyword If Test Failed | Show SR LocalSIDs on all DUTs | ${nodes}
-
-| Tear down mrr test with SRv6 with encapsulation
-| | [Documentation] | Common test teardown for mrr tests with SRv6 with \
-| | ... | encapsulation feature used.
+| | ... | *Test Variables needed:*
+| | ... | - frame_size - L2 Frame Size [B]. Type: integer
+| | ... | - traffic_profile - Profile name to run debug trial with. Type: string
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Tear down mrr test with SRv6 with encapsulation \|
+| | ... | \| Tear down performance test with SRv6 with encapsulation \|
| | ...
-| | Remove All Added Ports On All DUTs From Topology | ${nodes}
-| | Show VAT History On All DUTs | ${nodes}
+| | Tear down performance test
| | Run Keyword If Test Failed | Show SR Policies on all DUTs | ${nodes}
| | Run Keyword If Test Failed
| | ... | Show SR Steering Policies on all DUTs | ${nodes}
Code Review / csit.git / blobdiff