CSIT-457: HC Test: Document expected test failures

[csit.git] / resources / libraries / robot / honeycomb / interfaces.robot

diff --git a/resources/libraries/robot/honeycomb/interfaces.robot b/resources/libraries/robot/honeycomb/interfaces.robot
index 1ffc2e4..027b7d7 100644 (file)
--- a/resources/libraries/robot/honeycomb/interfaces.robot
+++ b/resources/libraries/robot/honeycomb/interfaces.robot
@@ -18,23 +18,6 @@
 | ...     | WITH NAME | InterfaceAPI
 
 *** Keywords ***
-| Interface state is
-| | [Documentation] | Uses VPP binary API to ensure that the interface under\
-| | ... | test is in the specified admin state.
-| | ...
-| | ... | *Arguments:*
-| | ... | - node - information about a DUT node. Type: dictionary
-| | ... | - interface - name of an interface on the specified node. Type: string
-| | ... | - state - state to set on interface. Type:string
-| | ...
-| | ... | *Example:*
-| | ...
-| | ... | \| Interface state is \| ${nodes['DUT1']} \| GigabitEthernet0/8/0 \
-| | ... | \| up \|
-| | [Arguments] | ${node} | ${interface} | ${state}
-| | interfaceCLI.Set interface state | ${node} | ${interface} | ${state}
-| | ... | if_type=name
-
 | Honeycomb sets interface state
 | | [Documentation] | Uses Honeycomb API to change the admin state\
 | | ... | of the specified interface.
@@ -92,7 +75,7 @@
 | | Should be equal | ${vat_state} | ${state}
 
 | Honeycomb sets interface ipv4 address
-| | [Documentation] | Uses Honeycomb API to change ipv4 configuration\
+| | [Documentation] | Uses Honeycomb API to change ipv4 address\
 | | ... | of the specified interface.
 | | ...
 | | ... | *Arguments:*
@@ -100,20 +83,14 @@
 | | ... | - interface - name of an interface on the specified node. Type: string
 | | ... | - address - IP address to set. Type: string
 | | ... | - netmask - subnet mask to set. Type: string
-| | ... | - settings - ipv4 interface settings. Type: dictionary
 | | ...
 | | ... | *Example:*
 | | ...
-| | ... | \| Honeycomb sets interface ipv4 configuration \| ${nodes['DUT1']} \
-| | ... | \| GigabitEthernet0/8/0 \| 192.168.0.2 \| 255.255.255.0 \
-| | ... | \| ${{'enabled': True, 'mtu': 1500}} \|
+| | ... | \| Honeycomb sets interface ipv4 address \| ${nodes['DUT1']} \
+| | ... | \| GigabitEthernet0/8/0 \| 192.168.0.2 \| 255.255.255.0 \|
 | | [Arguments] | ${node} | ${interface} | ${address} | ${netmask}
-| | ... | ${settings}
 | | interfaceAPI.Add first ipv4 address
 | | ... | ${node} | ${interface} | ${address} | ${netmask}
-| | :FOR | ${key} | IN | @{settings.keys()}
-| | | interfaceAPI.Configure interface ipv4
-| | | ... | ${node} | ${interface} | ${key} | ${settings['${key}']}
 
 | Honeycomb sets interface ipv4 address with prefix
 | | [Documentation] | Uses Honeycomb API to assign an ipv4 address to the\
@@ -124,41 +101,17 @@
 | | ... | - interface - name of an interface on the specified node. Type: string
 | | ... | - address - IP address to set. Type: string
 | | ... | - prefix - length of address network prefix. Type: int
-| | ... | - settings - ipv4 interface settings. Type: dictionary
 | | ...
 | | ... | *Example:*
 | | ...
 | | ... | \| Honeycomb sets interface ipv4 address with prefix \
 | | ... | \| ${nodes['DUT1']} \| GigabitEthernet0/8/0 \| 192.168.0.2 \| 24 \|
 | | [Arguments] | ${node} | ${interface} | ${address} | ${prefix}
-| | ... | ${settings}
 | | interfaceAPI.Add first ipv4 address
 | | ... | ${node} | ${interface} | ${address} | ${prefix}
-| | :FOR | ${key} | IN | @{settings.keys()}
-| | | interfaceAPI.Configure interface ipv4
-| | | ... | ${node} | ${interface} | ${key} | ${settings['${key}']}
 
-| Honeycomb adds interface ipv4 neighbor
-| | [Documentation] | Uses Honeycomb API to assign an ipv4 neighbor to the\
-| | ... | specified interface.
-| | ...
-| | ... | *Arguments:*
-| | ... | - node - information about a DUT node. Type: dictionary
-| | ... | - interface - name of an interface on the specified node. Type: string
-| | ... | - fib_address - IP address to add to fib table. Type: string
-| | ... | - fib_mac - MAC address to add to fib table. Type: string
-| | ...
-| | ... | *Example:*
-| | ...
-| | ... | \| Honeycomb adds interface ipv4 neighbor \| ${nodes['DUT1']} \
-| | ... | \| GigabitEthernet0/8/0 \| 192.168.0.3 \| 08:00:27:c0:5d:37 \
-| | ... | \| ${{'enabled': True, 'mtu': 1500}} \|
-| | [Arguments] | ${node} | ${interface} | ${fib_address} | ${fib_mac}
-| | interfaceAPI.Add ipv4 neighbor
-| | ... | ${node} | ${interface} | ${fib_address} | ${fib_mac}
-
-| IPv4 config from Honeycomb should be
-| | [Documentation] | Retrieves interface ipv4 configuration through Honeycomb\
+| IPv4 address from Honeycomb should be
+| | [Documentation] | Retrieves interface ipv4 address through Honeycomb\
 | | ... | and compares with state supplied in argument.
 | | ...
 | | ... | *Arguments:*
@@ -166,33 +119,20 @@
 | | ... | - interface - name of an interface on the specified node. Type: string
 | | ... | - address - IP address to expect. Type: string
 | | ... | - prefix - prefix length to expect. Type: string
-| | ... | - fib_address - IP address to expect in fib table. Type: string
-| | ... | - fib_mac - MAC address to expect in fib table. Type: string
-| | ... | - settings - ipv4 interface settings to expect. Type: dictionary
 | | ...
 | | ... | *Example:*
 | | ...
-| | ... | \| IPv4 config from Honeycomb should be \| ${nodes['DUT1']} \
+| | ... | \| IPv4 address from Honeycomb should be \| ${nodes['DUT1']} \
 | | ... | \| GigabitEthernet0/8/0 \| 192.168.0.2 \| 255.255.255.0 \
-| | ... | \| 192.168.0.3 \| 08:00:27:c0:5d:37 \
-| | ... | \| ${{'enabled': True, 'mtu': 1500}} \|
 | | [Arguments] | ${node} | ${interface} | ${address} | ${prefix}
-| | ... | ${fib_address} | ${fib_mac} | ${settings}
 | | ${api_data}= | interfaceAPI.Get interface oper data | ${node} | ${interface}
 | | Should be equal | ${address}
 | | ... | ${api_data['ietf-ip:ipv4']['address'][0]['ip']}
 | | Should be equal | ${prefix}
 | | ... | ${api_data['ietf-ip:ipv4']['address'][0]['prefix-length']}
-| | Should be equal | ${fib_address}
-| | ... | ${api_data['ietf-ip:ipv4']['neighbor'][0]['ip']}
-| | Should be equal | ${fib_mac}
-| | ... | ${api_data['ietf-ip:ipv4']['neighbor'][0]['link-layer-address']}
-| | :FOR | ${key} | IN | @{settings.keys()}
-| | | Should be equal
-| | | ... | ${settings['${key}']} | ${api_data['ietf-ip:ipv4']['${key}']}
 
-| IPv4 config from VAT should be
-| | [Documentation] | Retrieves interface ipv4 configuration through VAT and\
+| IPv4 address from VAT should be
+| | [Documentation] | Retrieves interface ipv4 address through VAT and\
 | | ... | compares with state supplied in argument.
 | | ...
 | | ... | *Arguments:*
@@ -203,7 +143,7 @@
 | | ...
 | | ... | *Example:*
 | | ...
-| | ... | \| IPv4 config from VAT should be \| ${nodes['DUT1']} \
+| | ... | \| IPv4 address from VAT should be \| ${nodes['DUT1']} \
 | | ... | \| GigabitEthernet0/8/0 \| 192.168.0.2 \| 255.255.255.0 \|
 | | [Arguments] | ${node} | ${interface} | ${address} | ${netmask}
 | | ${vpp_data}= | interfaceCLI.VPP get interface ip addresses
@@ -240,7 +180,8 @@
 | | ... | \| GigabitEthernet0/8/0 \|
 | | [Arguments] | ${node} | ${interface}
 | | ${api_data}= | interfaceAPI.Get interface oper data | ${node} | ${interface}
-| | Should be empty | ${api_data['ietf-ip:ipv4']['address']
+| | Run keyword and expect error | *KeyError: 'ietf-ip:ipv4'
+| | ... | Set Variable | ${api_data['ietf-ip:ipv4']['address']}
 
 | IPv4 address from VAT should be empty
 | | [Documentation] | Retrieves interface ipv4 configuration through VAT and\
@@ -255,12 +196,68 @@
 | | ... | \| IPv4 config from VAT should be empty \| ${nodes['DUT1']} \
 | | ... | \| GigabitEthernet0/8/0 \|
 | | [Arguments] | ${node} | ${interface}
-| | Run keyword and expect error | *No JSON object could be decoded.*
+| | Run keyword and expect error | *No JSON object could be decoded*
 | | ... | InterfaceCLI.VPP get interface ip addresses
 | | ... | ${node} | ${interface} | ipv4
 
-| Honeycomb sets interface ipv6 configuration
-| | [Documentation] | Uses Honeycomb API to change ipv6 configuration\
+| Honeycomb adds interface ipv4 neighbor
+| | [Documentation] | Uses Honeycomb API to assign an ipv4 neighbor to the\
+| | ... | specified interface.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - interface - name of an interface on the specified node. Type: string
+| | ... | - fib_address - IP address to add to fib table. Type: string
+| | ... | - fib_mac - MAC address to add to fib table. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Honeycomb adds interface ipv4 neighbor \| ${nodes['DUT1']} \
+| | ... | \| GigabitEthernet0/8/0 \| 192.168.0.3 \| 08:00:27:c0:5d:37 \
+| | [Arguments] | ${node} | ${interface} | ${fib_address} | ${fib_mac}
+| | interfaceAPI.Add ipv4 neighbor
+| | ... | ${node} | ${interface} | ${fib_address} | ${fib_mac}
+
+| IPv4 neighbor from Honeycomb should be
+| | [Documentation] | Retrieves ipv4 neighbor list through Honeycomb\
+| | ... | and compares with neighbor list supplied in argument.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - interface - name of an interface on the specified node. Type: string
+| | ... | - neighbors - list of ipv4 neighbor dictionaries. Type: list
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| IPv4 neighbor from Honeycomb should be \| ${nodes['DUT1']} \
+| | ... | \| GigabitEthernet0/8/0 \| 192.168.0.4 \| 08:00:27:60:26:ab \|
+| | [Arguments] | ${node} | ${interface} | @{neighbors}
+| | ${api_data}= | interfaceAPI.Get interface oper data | ${node} | ${interface}
+| | ${data_neighbors}= | Set Variable | ${api_data['ietf-ip:ipv4']['neighbor']}
+| | Compare data structures
+| | ... | ${data_neighbors} | ${neighbors} | list_order=${False}
+| | Should be equal | ${neighbor['fib_address']}
+| | ... | ${api_data['ietf-ip:ipv4']['neighbor'][0]['ip']}
+| | Should be equal | ${neighbor['fib_mac']}
+| | ... | ${api_data['ietf-ip:ipv4']['neighbor'][0]['link-layer-address']}
+
+| Honeycomb clears all interface ipv4 neighbors
+| | [Documentation] | Uses Honeycomb API to assign an ipv4 neighbor to the\
+| | ... | specified interface.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - interface - name of an interface on the specified node. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Honeycomb clears all interface ipv4 neighbors \| ${nodes['DUT1']} \
+| | ... | \| GigabitEthernet0/8/0 \|
+| | [Arguments] | ${node} | ${interface}
+| | interfaceAPI.Remove all ipv4 neighbors | ${node} | ${interface}
+
+| Honeycomb sets interface ipv6 address
+| | [Documentation] | Uses Honeycomb API to change ipv6 address\
 | | ... | of the specified interface.
 | | ...
 | | ... | *Arguments:*
@@ -268,27 +265,17 @@
 | | ... | - interface - name of an interface on the specified node. Type: string
 | | ... | - address - IP address to set. Type: string
 | | ... | - prefix - length of subnet prefix to set. Type: string
-| | ... | - fib_address - IP address to add to fib table. Type: string
-| | ... | - fib_mac - MAC address to add to fib table. Type: string
-| | ... | - settings - ipv6 interface settings. Type: dictionary
 | | ...
 | | ... | *Example:*
 | | ...
-| | ... | \| Honeycomb sets interface ipv6 configuration \| ${nodes['DUT1']} \
-| | ... | \| GigabitEthernet0/8/0 \| 10::10 \| 64 \
-| | ... | \| 10::11 \| 08:00:27:c0:5d:37 \| ${{'enabled': True, 'mtu': 1500}} \|
+| | ... | \| Honeycomb sets interface ipv6 address \| ${nodes['DUT1']} \
+| | ... | \| GigabitEthernet0/8/0 \| 10::10 \| 64 \|
 | | [Arguments] | ${node} | ${interface} | ${address} | ${prefix}
-| | ... | ${fib_address} | ${fib_mac} | ${settings}
 | | interfaceAPI.Add first ipv6 address
 | | ... | ${node} | ${interface} | ${address} | ${prefix}
-| | interfaceAPI.Add first ipv6 neighbor
-| | ... | ${node} | ${interface} | ${fib_address} | ${fib_mac}
-| | :FOR | ${key} | IN | @{settings.keys()}
-| | | interfaceAPI.Configure interface ipv6
-| | | ... | ${node} | ${interface} | ${key} | ${settings['${key}']}
 
-| IPv6 config from Honeycomb should be
-| | [Documentation] | Retrieves interface ipv6 configuration through Honeycomb\
+| IPv6 address from Honeycomb should be
+| | [Documentation] | Retrieves interface ipv6 address through Honeycomb\
 | | ... | and compares with state supplied in argument.
 | | ...
 | | ... | *Arguments:*
@@ -296,32 +283,21 @@
 | | ... | - interface - name of an interface on the specified node. Type: string
 | | ... | - address - IP address to expect. Type: string
 | | ... | - prefix - length of subnet prefix to expect. Type: string
-| | ... | - fib_address - IP address to expect in fib table. Type: string
-| | ... | - fib_mac - MAC address to expect in fib table. Type: string
-| | ... | - settings - ipv6 interface settings to expect. Type: dictionary
 | | ...
 | | ... | *Example:*
 | | ...
-| | ... | \| IPv6 config from Honeycomb should be \| ${nodes['DUT1']} \
-| | ... | \| GigabitEthernet0/8/0 \| 10::10 \| 64 \
-| | ... | \| 10::11 \| 08:00:27:c0:5d:37 \| ${{'enabled': True, 'mtu': 1500}} \|
+| | ... | \| IPv6 address from Honeycomb should be \| ${nodes['DUT1']} \
+| | ... | \| GigabitEthernet0/8/0 \| 10::10 \| 64 \|
 | | [Arguments] | ${node} | ${interface} | ${address} | ${prefix}
-| | ... | ${fib_address} | ${fib_mac} | ${settings}
 | | ${api_data}= | interfaceAPI.Get interface oper data | ${node} | ${interface}
 | | Should be equal | ${address}
 | | ... | ${api_data['ietf-ip:ipv6']['address'][0]['ip']}
 | | Should be equal | ${prefix}
 | | ... | ${api_data['ietf-ip:ipv6']['address'][0]['prefix-length']}
 | | Should be equal | ${fib_address}
-| | ... | ${api_data['ietf-ip:ipv6']['neighbor'][0]['ip']
-| | Should be equal | ${fib_mac}
-| | ... | ${api_data['ietf-ip:ipv6']['neighbor'][0]['link-layer-address']
-| | :FOR | ${key} | IN | @{settings.keys()}
-| | | Should be equal
-| | ... | ${settings['{key']} | ${api_data['ietf-ip:ipv6']['{$key}']}
 
-| IPv6 config from VAT should be
-| | [Documentation] | Retrieves interface ipv6 configuration through VAT and\
+| IPv6 address from VAT should be
+| | [Documentation] | Retrieves interface ipv6 address through VAT and\
 | | ... | compares with state supplied in argument.
 | | ...
 | | ... | *Arguments:*
@@ -332,7 +308,7 @@
 | | ...
 | | ... | *Example:*
 | | ...
-| | ... | \| IPv6 config from VAT should be \| ${nodes['DUT1']} \
+| | ... | \| IPv6 address from VAT should be \| ${nodes['DUT1']} \
 | | ... | \| GigabitEthernet0/8/0 \| 10::10 \| 64 \|
 | | [Arguments] | ${node} | ${interface} | ${address} | ${prefix}
 | | ${vpp_data}= | interfaceCLI.VPP get interface ip addresses
@@ -340,67 +316,58 @@
 | | Should be equal | ${vpp_data[0]['ip']} | ${address}
 | | Should be equal | ${vpp_data[0]['prefix-length']} | ${prefix}
 
-| Honeycomb sets interface ethernet and routing configuration
-| | [Documentation] | Uses Honeycomb API to change interface configuration.
+| Honeycomb sets interface ethernet configuration
+| | [Documentation] | Uses Honeycomb API to change interface ethernet\
+| | ... | configuration.
 | | ...
 | | ... | *Arguments:*
 | | ... | - node - information about a DUT node. Type: dictionary
 | | ... | - interface - name of an interface on the specified node. Type: string
 | | ... | - ethernet - interface ethernet settings. Type: dictionary
-| | ... | - routing - interface routing settings. Type: dictionary
 | | ...
 | | ... | *Example:*
 | | ...
 | | ... | \| Honeycomb sets interface ethernet and routing configuration \
 | | ... | \| ${nodes['DUT1']} \| GigabitEthernet0/8/0 \| ${{'mtu': 1500}} \
-| | ... | \| ${{'vrf-if': 2}} \|
-| | [Arguments] | ${node} | ${interface} | ${ethernet} | ${routing}
+| | [Arguments] | ${node} | ${interface} | ${ethernet}
 | | :FOR | ${key} | IN | @{ethernet.keys()}
 | | | interfaceAPI.Configure interface ethernet
 | | | ... | ${node} | ${interface} | ${key} | ${ethernet['${key}']}
-| | :FOR | ${key} | IN | @{routing.keys()}
-| | | interfaceAPI.Configure interface routing
-| | | ... | ${node} | ${interface} | ${key} | ${routing['${key}']}
 
-| Interface ethernet and routing configuration from Honeycomb should be
-| | [Documentation] | Retrieves interface routing and ethernet configuration\
+| Interface ethernet configuration from Honeycomb should be
+| | [Documentation] | Retrieves interface ethernet configuration\
 | | ... | through Honeycomb and compares with settings supplied in arguments.
 | | ...
 | | ... | *Arguments:*
 | | ... | - node - information about a DUT node. Type: dictionary
 | | ... | - interface - name of an interface on the specified node. Type: string
 | | ... | - ethernet - interface ethernet settings. Type: dictionary
-| | ... | - routing - interface routing settings. Type: dictionary
 | | ...
 | | ... | *Example:*
 | | ...
 | | ... | \| Interface ethernet and routing configuration from Honeycomb \
 | | ... | should be \| ${nodes['DUT1']} \| GigabitEthernet0/8/0 \
-| | ... | \| ${{'mtu': 1500}} \| ${{'vrf-id': 2}} \|
-| | [Arguments] | ${node} | ${interface} | ${ethernet} | ${routing}
+| | ... | \| ${{'mtu': 1500}} \|
+| | [Arguments] | ${node} | ${interface} | ${ethernet}
 | | ${api_data}= | interfaceAPI.Get interface oper data | ${node} | ${interface}
 | | :FOR | ${key} | IN | @{ethernet.keys()}
 | | | Should be equal
 | | | ... | ${api_data['v3po:ethernet']['${key}']} | ${ethernet['${key}']}
-| | :FOR | ${key} | IN | @{routing.keys()}
-| | | Should be equal | ${api_data['${key}']} | ${routing['${key}']}
 
-| Interface ethernet and routing configuration from VAT should be
-| | [Documentation] | Retrieves interface routing and ethernet configuration\
+| Interface ethernet configuration from VAT should be
+| | [Documentation] | Retrieves interface ethernet configuration\
 | | ... | through VAT and compares with settings supplied in arguments.
 | | ...
 | | ... | *Arguments:*
 | | ... | - node - information about a DUT node. Type: dictionary
 | | ... | - interface - name of an interface on the specified node. Type: string
 | | ... | - mtu - value of maximum transmission unit expected. Type: integer
-| | ... | - vrf-id - ID number of a VPN expected on interface. Type: integer
 | | ...
 | | ... | *Example:*
 | | ...
 | | ... | \| Interface ethernet and routing configuration from VAT \
-| | ... | should be \| ${nodes['DUT1']} \| GigabitEthernet0/8/0 \| ${1500} \
-| | ... | \| ${2} \|
-| | [Arguments] | ${node} | ${interface} | ${mtu} | ${vrf-id}
+| | ... | should be \| ${nodes['DUT1']} \| GigabitEthernet0/8/0 \| ${1500} \|
+| | [Arguments] | ${node} | ${interface} | ${mtu}
 | | ${vat_data}= | InterfaceCLI.VPP get interface data | ${node} | ${interface}
 | | Should be equal | ${vat_data['mtu']} | ${mtu}
 
@@ -459,3 +426,50 @@
 | | ${sw_if_index}= | EVALUATE | ${vat_data['sw_if_index']} + 1
 | | Should be equal as strings
 | | ... | ${api_data['if-index']} | ${sw_if_index}
+
+| Get Interface index from oper data
+| | [Documentation] | Retrieves interface operational data and returns\
+| | ... | if-index of the specified interface.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - interface - name of the interface. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Get Interface index from oper data \| ${nodes['DUT1']} \| local0 \|
+| | [Arguments] | ${node} | ${interface}
+| | ${data}= | interfaceAPI.Get interface oper data | ${node} | ${interface}
+| | Return from keyword | ${data['if-index']}
+
+| Honeycomb should show disabled interface in oper data
+| | [Documentation] | Retrieves list of disabled interfaces\
+| | ... | and verifies that there is at least one.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - index - index of the interface to be checked. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Honeycomb should show disabled interface in oper data \
+| | ... | \|${nodes['DUT1']} \| ${vx_interface} \|
+| | [Arguments] | ${node} | ${index}
+| | interfaceAPI.check disabled interface | ${node} | ${index}
+
+| Honeycomb should not show disabled interface in oper data
+| | [Documentation] | Retrieves list of disabled interfaces\
+| | ... | and expects to fail with a 404 - not found.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - index - index of the interface to be checked. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Honeycomb should not show disabled interface in oper data \
+| | ... | \|${nodes['DUT1']} \| ${vx_interface} \|
+| | [Arguments] | ${node} | ${index}
+| | Run keyword and expect error | *
+| | ... | Honeycomb should show disabled interface in oper data
+| | ... | ${node} | ${index}