Long perf tests

- added new long tests for ipv4 and xconnect
- modified long bridge domain test
- added 64B, 1518B and 9000B frame sizes

Change-Id: I88f75c375345f2d708df3fd24dd50d77bc63c495
Signed-off-by: pmikus <pmikus@cisco.com>

diff --git a/resources/libraries/robot/default.robot b/resources/libraries/robot/default.robot
index 0c93415..d085825 100644 (file)
--- a/resources/libraries/robot/default.robot
+++ b/resources/libraries/robot/default.robot
@@ -15,6 +15,7 @@
 | Variables | resources/libraries/python/topology.py
 | Library | resources/libraries/python/DUTSetup.py
 | Library | resources/libraries/python/TGSetup.py
 | Variables | resources/libraries/python/topology.py
 | Library | resources/libraries/python/DUTSetup.py
 | Library | resources/libraries/python/TGSetup.py

+| Library | Collections

 
 *** Keywords ***
 | Setup all DUTs before test
 
 *** Keywords ***
 | Setup all DUTs before test


@@ -24,3 +25,10 @@
 | Setup all TGs before traffic script
 | | [Documentation] | Prepare all TGs before traffic scripts execution
 | | All TGs Set Interface Default Driver | ${nodes}
 | Setup all TGs before traffic script
 | | [Documentation] | Prepare all TGs before traffic scripts execution
 | | All TGs Set Interface Default Driver | ${nodes}

+
+| Show statistics on all DUTs
+| | [Documentation] | Show VPP statistics on all DUTs after the test failed
+| | Sleep | 10 | Waiting for statistics to be collected
+| | ${duts}= | Get Matches | ${nodes} | DUT*
+| | :FOR | ${dut} | IN | @{duts}
+| | | Vpp show stats | ${nodes['${dut}']}

diff --git a/resources/libraries/robot/performance.robot b/resources/libraries/robot/performance.robot
index a8e18b1..ca7ce30 100644 (file)
--- a/resources/libraries/robot/performance.robot
+++ b/resources/libraries/robot/performance.robot
@@ -12,6 +12,14 @@
 # limitations under the License.
 
 *** Settings ***
 # limitations under the License.
 
 *** Settings ***

+| Resource | resources/libraries/robot/default.robot
+| Resource | resources/libraries/robot/interfaces.robot
+| Resource | resources/libraries/robot/l2_xconnect.robot
+| Resource | resources/libraries/robot/ipv4.robot
+| Resource | resources/libraries/robot/bridge_domain.robot
+| Resource | resources/libraries/robot/counters.robot
+| Library | resources.libraries.python.TrafficGenerator
+| Library | resources.libraries.python.TrafficGenerator.TGDropRateSearchImpl

 | Documentation | Performance suite keywords
 
 *** Keywords ***
 | Documentation | Performance suite keywords
 
 *** Keywords ***


@@ -35,3 +43,67 @@
 | | Set Suite Variable | ${dut2_if1}
 | | Set Suite Variable | ${dut2_if2}
 
 | | Set Suite Variable | ${dut2_if1}
 | | Set Suite Variable | ${dut2_if2}
 

+| IPv4 forwarding initialized in a 3-node circular topology
+| | [Documentation] | Custom setup of IPv4 addresses on all DUT nodes and TG
+| | Set Interface State | ${dut1} | ${dut1_if1} | up
+| | Set Interface State | ${dut1} | ${dut1_if2} | up
+| | Set Interface State | ${dut2} | ${dut2_if1} | up
+| | Set Interface State | ${dut2} | ${dut2_if2} | up
+| | ${tg1_if1_mac}= | Get Interface MAC | ${tg} | ${tg_if1}
+| | ${tg1_if2_mac}= | Get Interface MAC | ${tg} | ${tg_if2}
+| | ${dut1_if1_mac}= | Get Interface MAC | ${dut1} | ${dut1_if1}
+| | ${dut1_if2_mac}= | Get Interface MAC | ${dut1} | ${dut1_if2}
+| | ${dut2_if1_mac}= | Get Interface MAC | ${dut2} | ${dut1_if1}
+| | ${dut2_if2_mac}= | Get Interface MAC | ${dut2} | ${dut1_if2}
+| | dut1_v4.set_arp | ${dut1_if1} | 10.10.10.2 | ${tg1_if1_mac}
+| | dut1_v4.set_arp | ${dut1_if2} | 1.1.1.2 | ${dut2_if1_mac}
+| | dut2_v4.set_arp | ${dut2_if1} | 1.1.1.1 | ${dut1_if2_mac}
+| | dut2_v4.set_arp | ${dut2_if2} | 20.20.20.2 | ${tg1_if2_mac}
+| | dut1_v4.set_ip | ${dut1_if1} | 10.10.10.1 | 24
+| | dut1_v4.set_ip | ${dut1_if2} | 1.1.1.1 | 30
+| | dut2_v4.set_ip | ${dut2_if1} | 1.1.1.2 | 30
+| | dut2_v4.set_ip | ${dut2_if2} | 20.20.20.1 | 24
+| | dut1_v4.set_route | 20.20.20.0 | 24 | 1.1.1.2 | ${dut1_if2}
+| | dut2_v4.set_route | 10.10.10.0 | 24 | 1.1.1.1 | ${dut2_if1}
+| | All Vpp Interfaces Ready Wait | ${nodes}
+
+| L2 xconnect initialized in a 3-node circular topology
+| | [Documentation] | Custom setup of L2 xconnect topology
+| | L2 setup xconnect on DUT | ${dut1} | ${dut1_if1} | ${dut1_if2}
+| | L2 setup xconnect on DUT | ${dut2} | ${dut2_if1} | ${dut2_if2}
+| | All Vpp Interfaces Ready Wait | ${nodes}
+
+| L2 bridge domain initialized in a 3-node circular topology
+| | [Documentation] | Custom setup of L2 bridge topology
+| | Vpp l2bd forwarding setup | ${dut1} | ${dut1_if1} | ${dut1_if2}
+| | Vpp l2bd forwarding setup | ${dut2} | ${dut2_if1} | ${dut2_if2}
+| | All Vpp Interfaces Ready Wait | ${nodes}
+
+| 3-node Performance Suite Setup
+| | 3-node circular Topology Variables Setup
+| | [Arguments] | ${topology_type}
+| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
+| | ...                          | ${dut1} | ${dut1_if1} | ${dut1_if2}
+| | ...                          | ${dut2} | ${dut2_if1} | ${dut2_if2}
+| | ...                          | ${topology_type}
+
+| 3-node Performance Suite Teardown
+| | Teardown traffic generator | ${tg}
+
+| Find NDR using linear search and pps
+| | [Arguments] | ${framesize} | ${start_rate} | ${step_rate}
+| | ...         | ${topology_type} | ${min_rate} | ${max_rate}
+| | Set Duration | 60
+| | Set Search Rate Boundaries | ${max_rate} | ${min_rate}
+| | Set Search Linear Step | ${step_rate}
+| | Set Search Frame Size | ${framesize}
+| | Set Search Rate Type pps
+| | Linear Search | ${start_rate} | ${topology_type}
+| | ${result_rate}= | Verify Search Result
+| | Set Test Message | FINAL_RATE: ${result_rate} pps
+
+| Traffic should pass with no loss
+| | [Arguments] | ${duration} | ${rate} | ${framesize} | ${topology_type}
+| | Send traffic on | ${tg} | ${duration}
+| | ...             | ${rate} | ${framesize} | ${topology_type}
+| | No traffic loss occured

diff --git a/tests/suites/performance/long_bridge_domain.robot b/tests/suites/performance/long_bridge_domain.robot
index 43076b3..9938cce 100644 (file)
--- a/tests/suites/performance/long_bridge_domain.robot
+++ b/tests/suites/performance/long_bridge_domain.robot
@@ -11,16 +11,10 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 *** Settings ***
 # See the License for the specific language governing permissions and
 # limitations under the License.
 *** Settings ***

-| Resource | resources/libraries/robot/default.robot
-| Resource | resources/libraries/robot/interfaces.robot
-| Resource | resources/libraries/robot/bridge_domain.robot

 | Resource | resources/libraries/robot/performance.robot
 | Resource | resources/libraries/robot/performance.robot

-| Resource | resources/libraries/robot/counters.robot
-| Library | resources.libraries.python.TrafficGenerator
-| Library | resources.libraries.python.TrafficGenerator.TGDropRateSearchImpl

 | Library | resources.libraries.python.NodePath
 | Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | PERFTEST_LONG
 | Library | resources.libraries.python.NodePath
 | Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | PERFTEST_LONG

-| Suite Setup | 3-node Performance Suite Setup
+| Suite Setup | 3-node Performance Suite Setup | L2

 | Suite Teardown | 3-node Performance Suite Teardown
 | Test Setup | Setup all DUTs before test
 | Test Teardown  | Run Keyword If Test Failed | Show statistics on all DUTs
 | Suite Teardown | 3-node Performance Suite Teardown
 | Test Setup | Setup all DUTs before test
 | Test Teardown  | Run Keyword If Test Failed | Show statistics on all DUTs


@@ -28,39 +22,31 @@
 
 *** Test Cases ***
 | Find NDR by using linear search and 64B frames through bridge domain in 3-node topology
 
 *** Test Cases ***
 | Find NDR by using linear search and 64B frames through bridge domain in 3-node topology

+| | ${framesize}= | Set Variable | 64
+| | ${start_rate}= | Set Variable | 5000000
+| | ${step_rate}= | Set Variable | 100000
+| | ${min_rate}= | Set Variable | 100000
+| | ${max_rate}= | Set Variable | 14880952

 | | Given L2 bridge domain initialized in a 3-node circular topology
 | | Given L2 bridge domain initialized in a 3-node circular topology

-| | Then Find NDR using linear search and pps | 64 | 4600000 | 100000
-| | ...                                       | 3-node-bridge | 14000000 | 100000
+| | Then Find NDR using linear search and pps | ${framesize} | ${start_rate} | ${step_rate}
+| | ...                                       | 3-node-bridge | ${min_rate} | ${max_rate}

 
 

-*** Keywords ***
-
-| 3-node Performance Suite Setup
-| | 3-node circular Topology Variables Setup
-| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
-| | ...                          | ${dut1} | ${dut1_if1} | ${dut1_if2}
-| | ...                          | ${dut2} | ${dut2_if1} | ${dut2_if2}
-| | ...                          | L2
-
-| 3-node Performance Suite Teardown
-| | Teardown traffic generator | ${tg}
-
-| L2 bridge domain initialized in a 3-node circular topology
-| | Vpp l2bd forwarding setup | ${dut1} | ${dut1_if1} | ${dut1_if2}
-| | Vpp l2bd forwarding setup | ${dut2} | ${dut2_if1} | ${dut2_if2}
-| | All Vpp Interfaces Ready Wait | ${nodes}
-
-| Find NDR using linear search and pps
-| | [Arguments] | ${framesize} | ${start_rate} | ${step_rate}
-| | ...         | ${topology_type} | ${min_rate} | ${max_rate}
-| | Set Duration | 60
-| | Set Search Rate Boundaries | 14000000 | 100000
-| | Set Search Linear Step | ${step_rate}
-| | Set Search Rate Type pps
-| | Linear Search | ${start_rate} | ${topology_type}
-| | ${result_rate}= | Verify Search Result
-| | Set Test Message | FINAL_RATE: ${result_rate} pps
+| Find NDR by using linear search and 1518B frames through bridge domain in 3-node topology
+| | ${framesize}= | Set Variable | 1518
+| | ${start_rate}= | Set Variable | 812743
+| | ${step_rate}= | Set Variable | 10000
+| | ${min_rate}= | Set Variable | 10000
+| | ${max_rate}= | Set Variable | 812743
+| | Given L2 bridge domain initialized in a 3-node circular topology
+| | Then Find NDR using linear search and pps | ${framesize} | ${start_rate} | ${step_rate}
+| | ...                                       | 3-node-bridge | ${min_rate} | ${max_rate}

 
 

-| Show statistics on all DUTs
-| | Sleep | 10 | Waiting for statistics to be collected
-| | Vpp show stats | ${dut1}
-| | Vpp show stats | ${dut2}
+| Find NDR by using linear search and 9000B frames through bridge domain in 3-node topology
+| | ${framesize}= | Set Variable | 9000
+| | ${start_rate}= | Set Variable | 138580
+| | ${step_rate}= | Set Variable | 5000
+| | ${min_rate}= | Set Variable | 5000
+| | ${max_rate}= | Set Variable | 138580
+| | Given L2 bridge domain initialized in a 3-node circular topology
+| | Then Find NDR using linear search and pps | ${framesize} | ${start_rate} | ${step_rate}
+| | ...                                       | 3-node-bridge | ${min_rate} | ${max_rate}

diff --git a/tests/suites/performance/long_ipv4.robot b/tests/suites/performance/long_ipv4.robot
new file mode 100644 (file)
index 0000000..108fc90
--- /dev/null
+++ b/tests/suites/performance/long_ipv4.robot
@@ -0,0 +1,56 @@
+# Copyright (c) 2016 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+*** Settings ***
+| Resource | resources/libraries/robot/performance.robot
+| Library | resources.libraries.python.topology.Topology
+| Library | resources.libraries.python.NodePath
+| Library | resources.libraries.python.InterfaceUtil
+| Library | resources.libraries.python.IPv4Setup.Dut | ${nodes['DUT1']} | WITH NAME | dut1_v4
+| Library | resources.libraries.python.IPv4Setup.Dut | ${nodes['DUT2']} | WITH NAME | dut2_v4
+| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | PERFTEST_LONG
+| Suite Setup | 3-node Performance Suite Setup | L3
+| Suite Teardown | 3-node Performance Suite Teardown
+| Test Setup | Setup all DUTs before test
+| Test Teardown  | Run Keyword If Test Failed | Show statistics on all DUTs
+| Documentation | Throughput search suite (long running test suite based on RFC2544).
+
+*** Test Cases ***
+| Find NDR by using linear search and 64B frames through IPv4 forwarding in 3-node topology
+| | ${framesize}= | Set Variable | 64
+| | ${start_rate}= | Set Variable | 5000000
+| | ${step_rate}= | Set Variable | 100000
+| | ${min_rate}= | Set Variable | 100000
+| | ${max_rate}= | Set Variable | 14880952
+| | Given IPv4 forwarding initialized in a 3-node circular topology
+| | Then Find NDR using linear search and pps | ${framesize} | ${start_rate} | ${step_rate}
+| | ...                                       | 3-node-IPv4 | ${min_rate} | ${max_rate}
+
+| Find NDR by using linear search and 1518B frames through IPv4 forwarding in 3-node topology
+| | ${framesize}= | Set Variable | 1518
+| | ${start_rate}= | Set Variable | 812743
+| | ${step_rate}= | Set Variable | 10000
+| | ${min_rate}= | Set Variable | 10000
+| | ${max_rate}= | Set Variable | 812743
+| | Given IPv4 forwarding initialized in a 3-node circular topology
+| | Then Find NDR using linear search and pps | ${framesize} | ${start_rate} | ${step_rate}
+| | ...                                       | 3-node-IPv4 | ${min_rate} | ${max_rate}
+
+| Find NDR by using linear search and 9000B frames through IPv4 forwarding in 3-node topology
+| | ${framesize}= | Set Variable | 9000
+| | ${start_rate}= | Set Variable | 138580
+| | ${step_rate}= | Set Variable | 5000
+| | ${min_rate}= | Set Variable | 5000
+| | ${max_rate}= | Set Variable | 138580
+| | Given IPv4 forwarding initialized in a 3-node circular topology
+| | Then Find NDR using linear search and pps | ${framesize} | ${start_rate} | ${step_rate}
+| | ...                                       | 3-node-IPv4 | ${min_rate} | ${max_rate}

diff --git a/tests/suites/performance/long_xconnect.robot b/tests/suites/performance/long_xconnect.robot
new file mode 100644 (file)
index 0000000..773bd7c
--- /dev/null
+++ b/tests/suites/performance/long_xconnect.robot
@@ -0,0 +1,53 @@
+# Copyright (c) 2016 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+*** Settings ***
+| Resource | resources/libraries/robot/performance.robot
+| Library | resources.libraries.python.InterfaceUtil
+| Library | resources.libraries.python.NodePath
+| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | PERFTEST_LONG
+| Suite Setup | 3-node Performance Suite Setup | L2
+| Suite Teardown | 3-node Performance Suite Teardown
+| Test Setup | Setup all DUTs before test
+| Test Teardown  | Run Keyword If Test Failed | Show statistics on all DUTs
+| Documentation | Throughput search suite (long running test suite based on RFC2544).
+
+*** Test Cases ***
+| Find NDR by using linear search and 64B frames through L2 cross connect in 3-node topology
+| | ${framesize}= | Set Variable | 64
+| | ${start_rate}= | Set Variable | 5000000
+| | ${step_rate}= | Set Variable | 100000
+| | ${min_rate}= | Set Variable | 100000
+| | ${max_rate}= | Set Variable | 14880952
+| | Given L2 xconnect initialized in a 3-node circular topology
+| | Then Find NDR using linear search and pps | ${framesize} | ${start_rate} | ${step_rate}
+| | ...                                       | 3-node-xconnect | ${min_rate} | ${max_rate}
+
+| Find NDR by using linear search and 1518B frames through L2 cross connect in 3-node topology
+| | ${framesize}= | Set Variable | 1518
+| | ${start_rate}= | Set Variable | 812743
+| | ${step_rate}= | Set Variable | 10000
+| | ${min_rate}= | Set Variable | 10000
+| | ${max_rate}= | Set Variable | 812743
+| | Given L2 xconnect initialized in a 3-node circular topology
+| | Then Find NDR using linear search and pps | ${framesize} | ${start_rate} | ${step_rate}
+| | ...                                       | 3-node-xconnect | ${min_rate} | ${max_rate}
+
+| Find NDR by using linear search and 9000B frames through L2 cross connect in 3-node topology
+| | ${framesize}= | Set Variable | 9000
+| | ${start_rate}= | Set Variable | 138580
+| | ${step_rate}= | Set Variable | 5000
+| | ${min_rate}= | Set Variable | 5000
+| | ${max_rate}= | Set Variable | 138580
+| | Given L2 xconnect initialized in a 3-node circular topology
+| | Then Find NDR using linear search and pps | ${framesize} | ${start_rate} | ${step_rate}
+| | ...                                       | 3-node-xconnect | ${min_rate} | ${max_rate}

diff --git a/tests/suites/performance/short_bridge_domain.robot b/tests/suites/performance/short_bridge_domain.robot
index 96a8be3..210664c 100644 (file)
--- a/tests/suites/performance/short_bridge_domain.robot
+++ b/tests/suites/performance/short_bridge_domain.robot
@@ -11,56 +11,32 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 *** Settings ***
 # See the License for the specific language governing permissions and
 # limitations under the License.
 *** Settings ***

-| Resource | resources/libraries/robot/default.robot
-| Resource | resources/libraries/robot/interfaces.robot
-| Resource | resources/libraries/robot/bridge_domain.robot

 | Resource | resources/libraries/robot/performance.robot
 | Resource | resources/libraries/robot/performance.robot

-| Resource | resources/libraries/robot/counters.robot
-| Library | resources.libraries.python.TrafficGenerator

 | Library | resources.libraries.python.NodePath
 | Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | PERFTEST_SHORT
 | Library | resources.libraries.python.NodePath
 | Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | PERFTEST_SHORT

-| Suite Setup | 3-node Performance Suite Setup
+| Suite Setup | 3-node Performance Suite Setup | L2

 | Suite Teardown | 3-node Performance Suite Teardown
 | Test Setup | Setup all DUTs before test
 | Test Teardown  | Run Keyword If Test Failed | Show statistics on all DUTs
 
 *** Test Cases ***
 | 1core VPP passes 64B frames through bridge domain at 3.5mpps in 3-node topology
 | Suite Teardown | 3-node Performance Suite Teardown
 | Test Setup | Setup all DUTs before test
 | Test Teardown  | Run Keyword If Test Failed | Show statistics on all DUTs
 
 *** Test Cases ***
 | 1core VPP passes 64B frames through bridge domain at 3.5mpps in 3-node topology

+| | ${framesize}= | Set Variable | 64
+| | ${duration}= | Set Variable | 10
+| | ${rate}= | Set Variable | 3.5mpps

 | | Given L2 bridge domain initialized in a 3-node circular topology
 | | Given L2 bridge domain initialized in a 3-node circular topology

-| | Then Traffic should pass with no loss | 10 | 3.5mpps | 64 | 3-node-bridge
+| | Then Traffic should pass with no loss | ${duration} | ${rate} | ${framesize} | 3-node-bridge

 
 | 1core VPP passes 1518B frames through bridge domain at 10gbps in 3-node topology
 
 | 1core VPP passes 1518B frames through bridge domain at 10gbps in 3-node topology

+| | ${framesize}= | Set Variable | 1518
+| | ${duration}= | Set Variable | 10
+| | ${rate}= | Set Variable | 10gbps

 | | Given L2 bridge domain initialized in a 3-node circular topology
 | | Given L2 bridge domain initialized in a 3-node circular topology

-| | Then Traffic should pass with no loss | 10 | 10gbps | 1518 | 3-node-bridge
+| | Then Traffic should pass with no loss | ${duration} | ${rate} | ${framesize} | 3-node-bridge

 
 | 1core VPP passes 9000B frames through bridge domain at 10gbps in 3-node topology
 
 | 1core VPP passes 9000B frames through bridge domain at 10gbps in 3-node topology

+| | ${framesize}= | Set Variable | 9000
+| | ${duration}= | Set Variable | 10
+| | ${rate}= | Set Variable | 10gbps

 | | Given L2 bridge domain initialized in a 3-node circular topology
 | | Given L2 bridge domain initialized in a 3-node circular topology

-| | Then Traffic should pass with no loss | 10 | 10gbps | 9000 | 3-node-bridge
-
-*** Keywords ***
-
-| 3-node Performance Suite Setup
-| | 3-node circular Topology Variables Setup
-| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
-| | ...                          | ${dut1} | ${dut1_if1} | ${dut1_if2}
-| | ...                          | ${dut2} | ${dut2_if1} | ${dut2_if2}
-| | ...                          | L2
-
-| 3-node Performance Suite Teardown
-| | Teardown traffic generator | ${tg}
-
-| L2 bridge domain initialized in a 3-node circular topology
-| | Vpp l2bd forwarding setup | ${dut1} | ${dut1_if1} | ${dut1_if2}
-| | Vpp l2bd forwarding setup | ${dut2} | ${dut2_if1} | ${dut2_if2}
-| | All Vpp Interfaces Ready Wait | ${nodes}
-
-| Traffic should pass with no loss
-| | [Arguments] | ${duration} | ${rate} | ${framesize} | ${topology_type}
-| | Send traffic on | ${tg} | ${duration}
-| | ...             | ${rate} | ${framesize} | ${topology_type}
-| | No traffic loss occured
-
-| Show statistics on all DUTs
-| | Sleep | 10 | Waiting for statistics to be collected
-| | Vpp show stats | ${dut1}
-| | Vpp show stats | ${dut2}
+| | Then Traffic should pass with no loss | ${duration} | ${rate} | ${framesize} | 3-node-bridge

diff --git a/tests/suites/performance/short_ipv4.robot b/tests/suites/performance/short_ipv4.robot
index 067646b..95e1351 100644 (file)
--- a/tests/suites/performance/short_ipv4.robot
+++ b/tests/suites/performance/short_ipv4.robot
@@ -11,78 +11,36 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 *** Settings ***
 # See the License for the specific language governing permissions and
 # limitations under the License.
 *** Settings ***

-| Resource | resources/libraries/robot/default.robot
-| Resource | resources/libraries/robot/interfaces.robot
-| Resource | resources/libraries/robot/ipv4.robot

 | Resource | resources/libraries/robot/performance.robot
 | Resource | resources/libraries/robot/performance.robot

-| Resource | resources/libraries/robot/counters.robot

 | Library | resources.libraries.python.topology.Topology
 | Library | resources.libraries.python.topology.Topology

-| Library | resources.libraries.python.TrafficGenerator

 | Library | resources.libraries.python.NodePath
 | Library | resources.libraries.python.InterfaceUtil
 | Library | resources.libraries.python.IPv4Setup.Dut | ${nodes['DUT1']} | WITH NAME | dut1_v4
 | Library | resources.libraries.python.IPv4Setup.Dut | ${nodes['DUT2']} | WITH NAME | dut2_v4
 | Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | PERFTEST_SHORT
 | Library | resources.libraries.python.NodePath
 | Library | resources.libraries.python.InterfaceUtil
 | Library | resources.libraries.python.IPv4Setup.Dut | ${nodes['DUT1']} | WITH NAME | dut1_v4
 | Library | resources.libraries.python.IPv4Setup.Dut | ${nodes['DUT2']} | WITH NAME | dut2_v4
 | Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | PERFTEST_SHORT

-| Suite Setup | 3-node Performance Suite Setup
+| Suite Setup | 3-node Performance Suite Setup | L3

 | Suite Teardown | 3-node Performance Suite Teardown
 | Test Setup | Setup all DUTs before test
 | Test Teardown  | Run Keyword If Test Failed | Show statistics on all DUTs
 
 *** Test Cases ***
 | 1core VPP passes 64B frames through IPv4 forwarding at 3.5mpps in 3-node topology
 | Suite Teardown | 3-node Performance Suite Teardown
 | Test Setup | Setup all DUTs before test
 | Test Teardown  | Run Keyword If Test Failed | Show statistics on all DUTs
 
 *** Test Cases ***
 | 1core VPP passes 64B frames through IPv4 forwarding at 3.5mpps in 3-node topology

+| | ${framesize}= | Set Variable | 64
+| | ${duration}= | Set Variable | 10
+| | ${rate}= | Set Variable | 3.5mpps

 | | Given IPv4 forwarding initialized in a 3-node circular topology
 | | Given IPv4 forwarding initialized in a 3-node circular topology

-| | Then Traffic should pass with no loss | 10 | 3.5mpps | 64 | 3-node-IPv4
+| | Then Traffic should pass with no loss | ${duration} | ${rate} | ${framesize} | 3-node-IPv4

 
 | 1core VPP passes 1518B frames through IPv4 forwarding at 10gbps in 3-node topology
 
 | 1core VPP passes 1518B frames through IPv4 forwarding at 10gbps in 3-node topology

+| | ${framesize}= | Set Variable | 1518
+| | ${duration}= | Set Variable | 10
+| | ${rate}= | Set Variable | 10gbps

 | | Given IPv4 forwarding initialized in a 3-node circular topology
 | | Given IPv4 forwarding initialized in a 3-node circular topology

-| | Then Traffic should pass with no loss | 10 | 10gbps | 1518 | 3-node-IPv4
+| | Then Traffic should pass with no loss | ${duration} | ${rate} | ${framesize} | 3-node-IPv4

 
 | 1core VPP passes 9000B frames through IPv4 forwarding at 10gbps in 3-node topology
 
 | 1core VPP passes 9000B frames through IPv4 forwarding at 10gbps in 3-node topology

+| | ${framesize}= | Set Variable | 9000
+| | ${duration}= | Set Variable | 10
+| | ${rate}= | Set Variable | 10gbps

 | | Given IPv4 forwarding initialized in a 3-node circular topology
 | | Given IPv4 forwarding initialized in a 3-node circular topology

-| | Then Traffic should pass with no loss | 10 | 10gbps | 9000 | 3-node-IPv4
-
-*** Keywords ***
-
-| 3-node Performance Suite Setup
-| | 3-node circular Topology Variables Setup
-| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
-| | ...                          | ${dut1} | ${dut1_if1} | ${dut1_if2}
-| | ...                          | ${dut2} | ${dut2_if1} | ${dut2_if2}
-| | ...                          | L3
-
-| 3-node Performance Suite Teardown
-| | Teardown traffic generator | ${tg}
-
-| IPv4 forwarding initialized in a 3-node circular topology
-| | Set Interface State | ${dut1} | ${dut1_if1} | up
-| | Set Interface State | ${dut1} | ${dut1_if2} | up
-| | Set Interface State | ${dut2} | ${dut2_if1} | up
-| | Set Interface State | ${dut2} | ${dut2_if2} | up
-| | ${tg1_if1_mac}= | Get Interface MAC | ${tg} | ${tg_if1}
-| | ${tg1_if2_mac}= | Get Interface MAC | ${tg} | ${tg_if2}
-| | ${dut1_if1_mac}= | Get Interface MAC | ${dut1} | ${dut1_if1}
-| | ${dut1_if2_mac}= | Get Interface MAC | ${dut1} | ${dut1_if2}
-| | ${dut2_if1_mac}= | Get Interface MAC | ${dut2} | ${dut1_if1}
-| | ${dut2_if2_mac}= | Get Interface MAC | ${dut2} | ${dut1_if2}
-| | dut1_v4.set_arp | ${dut1_if1} | 10.10.10.2 | ${tg1_if1_mac}
-| | dut1_v4.set_arp | ${dut1_if2} | 1.1.1.2 | ${dut2_if1_mac}
-| | dut2_v4.set_arp | ${dut2_if1} | 1.1.1.1 | ${dut1_if2_mac}
-| | dut2_v4.set_arp | ${dut2_if2} | 20.20.20.2 | ${tg1_if2_mac}
-| | dut1_v4.set_ip | ${dut1_if1} | 10.10.10.1 | 24
-| | dut1_v4.set_ip | ${dut1_if2} | 1.1.1.1 | 30
-| | dut2_v4.set_ip | ${dut2_if1} | 1.1.1.2 | 30
-| | dut2_v4.set_ip | ${dut2_if2} | 20.20.20.1 | 24
-| | dut1_v4.set_route | 20.20.20.0 | 24 | 1.1.1.2 | ${dut1_if2}
-| | dut2_v4.set_route | 10.10.10.0 | 24 | 1.1.1.1 | ${dut2_if1}
-| | All Vpp Interfaces Ready Wait | ${nodes}
-
-| Traffic should pass with no loss
-| | [Arguments] | ${duration} | ${rate} | ${framesize} | ${topology_type}
-| | Send traffic on | ${tg} | ${duration}
-| | ...             | ${rate} | ${framesize} | ${topology_type}
-| | No traffic loss occured
-
-| Show statistics on all DUTs
-| | Sleep | 10 | Waiting for statistics to be collected
-| | Vpp show stats | ${dut1}
-| | Vpp show stats | ${dut2}
+| | Then Traffic should pass with no loss | ${duration} | ${rate} | ${framesize} | 3-node-IPv4

diff --git a/tests/suites/performance/short_xconnect.robot b/tests/suites/performance/short_xconnect.robot
index 7eff81c..55e0523 100644 (file)
--- a/tests/suites/performance/short_xconnect.robot
+++ b/tests/suites/performance/short_xconnect.robot
@@ -11,57 +11,33 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 *** Settings ***
 # See the License for the specific language governing permissions and
 # limitations under the License.
 *** Settings ***

-| Resource | resources/libraries/robot/default.robot
-| Resource | resources/libraries/robot/interfaces.robot
-| Resource | resources/libraries/robot/l2_xconnect.robot

 | Resource | resources/libraries/robot/performance.robot
 | Resource | resources/libraries/robot/performance.robot

-| Resource | resources/libraries/robot/counters.robot
-| Library | resources.libraries.python.TrafficGenerator

 | Library | resources.libraries.python.InterfaceUtil
 | Library | resources.libraries.python.NodePath
 | Library | resources.libraries.python.InterfaceUtil
 | Library | resources.libraries.python.NodePath

-| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV| PERFTEST_SHORT
-| Suite Setup | 3-node Performance Suite Setup
+| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | PERFTEST_SHORT
+| Suite Setup | 3-node Performance Suite Setup | L2

 | Suite Teardown | 3-node Performance Suite Teardown
 | Test Setup | Setup all DUTs before test
 | Test Teardown  | Run Keyword If Test Failed | Show statistics on all DUTs
 
 *** Test Cases ***
 | 1core VPP passes 64B frames through L2 cross connect at 3.5mpps in 3-node topology
 | Suite Teardown | 3-node Performance Suite Teardown
 | Test Setup | Setup all DUTs before test
 | Test Teardown  | Run Keyword If Test Failed | Show statistics on all DUTs
 
 *** Test Cases ***
 | 1core VPP passes 64B frames through L2 cross connect at 3.5mpps in 3-node topology

+| | ${framesize}= | Set Variable | 64
+| | ${duration}= | Set Variable | 10
+| | ${rate}= | Set Variable | 3.5mpps

 | | Given L2 xconnect initialized in a 3-node circular topology
 | | Given L2 xconnect initialized in a 3-node circular topology

-| | Then Traffic should pass with no loss | 10 | 3.5mpps | 64 | 3-node-xconnect
+| | Then Traffic should pass with no loss | ${duration} | ${rate} | ${framesize} | 3-node-xconnect

 
 | 1core VPP passes 1518B frames through L2 cross connect at 10gbps in 3-node topology
 
 | 1core VPP passes 1518B frames through L2 cross connect at 10gbps in 3-node topology

+| | ${framesize}= | Set Variable | 1518
+| | ${duration}= | Set Variable | 10
+| | ${rate}= | Set Variable | 10gbps

 | | Given L2 xconnect initialized in a 3-node circular topology
 | | Given L2 xconnect initialized in a 3-node circular topology

-| | Then Traffic should pass with no loss | 10 | 10gbps | 1518 | 3-node-xconnect
+| | Then Traffic should pass with no loss | ${duration} | ${rate} | ${framesize} | 3-node-xconnect

 
 | 1core VPP passes 9000B frames through L2 cross connect at 10gbps in 3-node topology
 
 | 1core VPP passes 9000B frames through L2 cross connect at 10gbps in 3-node topology

+| | ${framesize}= | Set Variable | 9000
+| | ${duration}= | Set Variable | 10
+| | ${rate}= | Set Variable | 10gbps

 | | Given L2 xconnect initialized in a 3-node circular topology
 | | Given L2 xconnect initialized in a 3-node circular topology

-| | Then Traffic should pass with no loss | 10 | 10gbps | 9000 | 3-node-xconnect
-
-*** Keywords ***
-
-| 3-node Performance Suite Setup
-| | 3-node circular Topology Variables Setup
-| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
-| | ...                          | ${dut1} | ${dut1_if1} | ${dut1_if2}
-| | ...                          | ${dut2} | ${dut2_if1} | ${dut2_if2}
-| | ...                          | L2
-
-| 3-node Performance Suite Teardown
-| | Teardown traffic generator | ${tg}
-
-| L2 xconnect initialized in a 3-node circular topology
-| | L2 setup xconnect on DUT | ${dut1} | ${dut1_if1} | ${dut1_if2}
-| | L2 setup xconnect on DUT | ${dut2} | ${dut2_if1} | ${dut2_if2}
-| | All Vpp Interfaces Ready Wait | ${nodes}
-
-| Traffic should pass with no loss
-| | [Arguments] | ${duration} | ${rate} | ${framesize} | ${topology_type}
-| | Send traffic on | ${tg} | ${duration}
-| | ...             | ${rate} | ${framesize} | ${topology_type}
-| | No traffic loss occured
-
-| Show statistics on all DUTs
-| | Sleep | 10 | Waiting for statistics to be collected
-| | Vpp show stats | ${dut1}
-| | Vpp show stats | ${dut2}
+| | Then Traffic should pass with no loss | ${duration} | ${rate} | ${framesize} | 3-node-xconnect