Device: Add stats test with AVF

+ Extract the stats keyword into a library.

Change-Id: Icfaaa32c2a97d479ba0339acf34b865006949a89
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

diff --git a/resources/libraries/python/VppCounters.py b/resources/libraries/python/VppCounters.py
index e9b607b..6bd0aea 100644 (file)
--- a/resources/libraries/python/VppCounters.py
+++ b/resources/libraries/python/VppCounters.py
@@ -1,4 +1,4 @@
-# Copyright (c) 2020 Cisco and/or its affiliates.
+# Copyright (c) 2021 Cisco and/or its affiliates.

 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at:
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at:

diff --git a/resources/libraries/robot/shared/default.robot b/resources/libraries/robot/shared/default.robot
index 891ef21..dcca1f2 100644 (file)
--- a/resources/libraries/robot/shared/default.robot
+++ b/resources/libraries/robot/shared/default.robot
@@ -94,6 +94,34 @@
 | | # Parens are there to perform the call.
 | | Run Keyword If | $resetter | Evaluate | $resetter()
 
 | | # Parens are there to perform the call.
 | | Run Keyword If | $resetter | Evaluate | $resetter()
 

+| Verify statistic commands
+| | [Documentation]
+| | ... | Execute several commands related to stats.
+| | ... | To be used by "stats" device tests,
+| | ... | as other device tests do not interact with stats.
+| |
+| | ... | The commands should also cover frequently used
+| | ... | PAPI (non-stats) commands and CLI commands.
+| |
+| | ${results}= | Create List
+| | ${status} | ${value}= | Run Keyword And Ignore Error
+| | ... | VPP Show Runtime On All DUTs | ${nodes}
+| | Append To List | ${results} | ${status}
+| | ${status} | ${value}= | Run Keyword And Ignore Error
+| | ... | Show Statistics On All DUTs | ${nodes}
+| | Append To List | ${results} | ${status}
+| | ${status} | ${value}= | Run Keyword And Ignore Error
+| | ... | Show Event Logger On All DUTs | ${nodes}
+| | Append To List | ${results} | ${status}
+| | ${status} | ${value}= | Run Keyword And Ignore Error
+| | ... | VPP Clear Runtime On All DUTs | ${nodes}
+| | Append To List | ${results} | ${status}
+| | ${status} | ${value}= | Run Keyword And Ignore Error
+| | ... | Clear Statistics On All DUTs | ${nodes}
+| | Append To List | ${results} | ${status}
+| | Should Not Contain Match | ${results} | FAIL
+| | ... | msg=At least one of statistic commands failed!
+

 | Workers From Physical Cores
 | | [Documentation]
 | | ... | Convert from core count to worker count.
 | Workers From Physical Cores
 | | [Documentation]
 | | ... | Convert from core count to worker count.

diff --git a/tests/vpp/device/stats/eth2p-avf-dot1qip4-l2bdbasemaclrn-stats-dev.robot b/tests/vpp/device/stats/eth2p-avf-dot1qip4-l2bdbasemaclrn-stats-dev.robot
new file mode 100644 (file)
index 0000000..2436e9f
--- /dev/null
+++ b/tests/vpp/device/stats/eth2p-avf-dot1qip4-l2bdbasemaclrn-stats-dev.robot
@@ -0,0 +1,87 @@
+# Copyright (c) 2021 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+| Resource | resources/libraries/robot/shared/default.robot
+|
+| Force Tags | 2_NODE_SINGLE_LINK_TOPO | DEVICETEST | HW_ENV | DCR_ENV | SCAPY
+| ... | NIC_Virtual | ETH | IP4FWD | BASE | DOT1Q | IP4BASE | DRV_AVF
+| ... | RXQ_SIZE_0 | TXQ_SIZE_0
+| ... | avf-dot1qip4-l2bdbasemaclrn
+|
+| Suite Setup | Setup suite topology interfaces | scapy
+| Suite Teardown | Tear down suite
+| Test Setup | Setup test
+| Test Teardown | Tear down test | packet_trace
+|
+| Test Template | Local Template
+|
+| Documentation | *IPv4 routing with IEEE 802.1Q test cases*
+|
+| ... | *[Top] Network Topologies:* TG-DUT1-TG 2-node circular topology with\
+| ... | single links between nodes.
+| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. IEEE 802.1Q\
+| ... | tagging is applied on links between TG-DUT1.
+| ... | *[Cfg] DUT configuration:* DUT1 is configured with IPv4 routing and\
+| ... | two static IPv4 /30 route entries. DUT1 is tested with ${nic_name}.
+| ... | *[Ver] TG verification:* Test IPv4 packets are sent in one direction \
+| ... | by TG on link to DUT1; on receive TG verifies packets for correctness \
+| ... | and drops as applicable.
+| ... | *[Ref] Applicable standard specifications:* IEEE 802.1q.
+
+*** Variables ***
+| @{plugins_to_enable}= | avf_plugin.so
+| ${crypto_type}= | ${None}
+| ${nic_name}= | virtual
+| ${nic_driver}= | avf
+| ${nic_rxq_size}= | 0
+| ${nic_txq_size}= | 0
+| ${nic_pfs}= | 2
+| ${nic_vfs}= | 1
+| ${overhead}= | ${4}
+
+*** Keywords ***
+| Local Template
+| | [Documentation]
+| | ... | [Ver] Make TG send IPv4 packets in both directions between two\
+| | ... | of its interfaces to be switched by DUT to and from docker; verify\
+| | ... | all packets are received.
+| |
+| | ... | *Arguments:*
+| | ... | - frame_size - Framesize in Bytes in integer. Type: integer
+| | ... | - phy_cores - Number of physical cores. Type: integer
+| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer
+| |
+| | [Arguments] | ${frame_size} | ${phy_cores} | ${rxq}=${None}
+| |
+| | Set Test Variable | \${frame_size}
+| |
+| | Given Set Max Rate And Jumbo
+| | And Add worker threads to all DUTs | ${phy_cores} | ${rxq}
+| | And Pre-initialize layer driver | ${nic_driver}
+| | And Apply startup configuration on all VPP DUTs | with_trace=${True}
+| | When Initialize layer driver | ${nic_driver}
+| | And Initialize layer interface
+| | And Initialize layer dot1q
+| | And Initialize L2 bridge domain
+| | # Do not fail here, we have a non-stats copy of this test elsewhere,
+| | # here we want to see stats after traffic, even if that test fails.
+| | Then Run Keyword And Ignore Error
+| | ... | Send IPv4 bidirectionally and verify received packets
+| | ... | ${tg} | ${TG_pf1}[0] | ${TG_pf2}[0]
+| | And Verify statistic commands
+
+*** Test Cases ***
+| 68B-avf-dot1qip4-l2bdbasemaclrn-stats-dev
+| | [Tags] | 68B
+| | frame_size=${68} | phy_cores=${0}

diff --git a/tests/vpp/device/stats/eth2p-ethipv4-l2xcbase-stats-dev.robot b/tests/vpp/device/stats/eth2p-ethipv4-l2xcbase-stats-dev.robot
index b37836e..12ce4f5 100644 (file)
--- a/tests/vpp/device/stats/eth2p-ethipv4-l2xcbase-stats-dev.robot
+++ b/tests/vpp/device/stats/eth2p-ethipv4-l2xcbase-stats-dev.robot
@@ -1,4 +1,4 @@
-# Copyright (c) 2020 Cisco and/or its affiliates.
+# Copyright (c) 2021 Cisco and/or its affiliates.

 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at:
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at:


@@ -78,26 +78,6 @@
 | | ... | ${tg} | ${TG_pf1}[0] | ${TG_pf2}[0]
 | | And Verify statistic commands
 
 | | ... | ${tg} | ${TG_pf1}[0] | ${TG_pf2}[0]
 | | And Verify statistic commands
 

-| Verify statistic commands
-| | ${results}= | Create List
-| | ${status} | ${value}= | Run Keyword And Ignore Error
-| | ... | VPP Show Runtime On All DUTs | ${nodes}
-| | Append To List | ${results} | ${status}
-| | ${status} | ${value}= | Run Keyword And Ignore Error
-| | ... | Show Statistics On All DUTs | ${nodes}
-| | Append To List | ${results} | ${status}
-| | ${status} | ${value}= | Run Keyword And Ignore Error
-| | ... | Show Event Logger On All DUTs | ${nodes}
-| | Append To List | ${results} | ${status}
-| | ${status} | ${value}= | Run Keyword And Ignore Error
-| | ... | VPP Clear Runtime On All DUTs | ${nodes}
-| | Append To List | ${results} | ${status}
-| | ${status} | ${value}= | Run Keyword And Ignore Error
-| | ... | Clear Statistics On All DUTs | ${nodes}
-| | Append To List | ${results} | ${status}
-| | Should Not Contain Match | ${results} | FAIL
-| | ... | msg=At least one of statistic commands failed!
-

 *** Test Cases ***
 | 64B-ethipv4-l2xcbase-stats-dev
 | | [Tags] | 64B
 *** Test Cases ***
 | 64B-ethipv4-l2xcbase-stats-dev
 | | [Tags] | 64B