resources/libraries/robot/performance.robot

   1 # Copyright (c) 2016 Cisco and/or its affiliates.
   2 # Licensed under the Apache License, Version 2.0 (the "License");
   3 # you may not use this file except in compliance with the License.
   4 # You may obtain a copy of the License at:
   5 #
   6 #     http://www.apache.org/licenses/LICENSE-2.0
   7 #
   8 # Unless required by applicable law or agreed to in writing, software
   9 # distributed under the License is distributed on an "AS IS" BASIS,
  10 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  11 # See the License for the specific language governing permissions and
  12 # limitations under the License.
  13
  14 *** Settings ***
  15 | Library | resources.libraries.python.topology.Topology
  16 | Library | resources.libraries.python.NodePath
  17 | Library | resources.libraries.python.InterfaceUtil
  18 | Library | resources.libraries.python.TrafficGenerator
  19 | Library | resources.libraries.python.TrafficGenerator.TGDropRateSearchImpl
  20 | Resource | resources/libraries/robot/default.robot
  21 | Resource | resources/libraries/robot/interfaces.robot
  22 | Resource | resources/libraries/robot/counters.robot
  23 | Resource | resources/libraries/robot/bridge_domain.robot
  24 | Resource | resources/libraries/robot/l2_xconnect.robot
  25 | Resource | resources/libraries/robot/ipv4.robot
  26 | Resource | resources/libraries/robot/ipv6.robot
  27 | Documentation | Performance suite keywords
  28
  29 *** Keywords ***
  30 | 3-node circular Topology Variables Setup
  31 | | Append Nodes | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes['DUT2']}
  32 | | ...          | ${nodes['TG']}
  33 | | Compute Path
  34 | | ${tg_if1} | ${tg}= | Next Interface
  35 | | ${dut1_if1} | ${dut1}= | Next Interface
  36 | | ${dut1_if2} | ${dut1}= | Next Interface
  37 | | ${dut2_if1} | ${dut2}= | Next Interface
  38 | | ${dut2_if2} | ${dut2}= | Next Interface
  39 | | ${tg_if2} | ${tg}= | Next Interface
  40 | | Set Suite Variable | ${tg}
  41 | | Set Suite Variable | ${tg_if1}
  42 | | Set Suite Variable | ${tg_if2}
  43 | | Set Suite Variable | ${dut1}
  44 | | Set Suite Variable | ${dut1_if1}
  45 | | Set Suite Variable | ${dut1_if2}
  46 | | Set Suite Variable | ${dut2}
  47 | | Set Suite Variable | ${dut2_if1}
  48 | | Set Suite Variable | ${dut2_if2}
  49
  50 | 3-node circular Topology Variables Setup with DUT interface model
  51 | | [Documentation] | Find a path between TG-DUT1-DUT2-TG based on interface
  52 | | ...             | model provided as an argument. Set suite variables
  53 | | ...             | tg, tg_if1, tg_if2, dut1, dut1_if1, dut1_if2,
  54 | | ...             | dut2, dut2_if1, dut2_if2
  55 | | [Arguments] | ${iface_model}
  56 | | ${iface_model_list}= | Create list | ${iface_model}
  57 | | Append Node | ${nodes['TG']}
  58 | | Append Node | ${nodes['DUT1']} | filter_list=${iface_model_list}
  59 | | Append Node | ${nodes['DUT2']} | filter_list=${iface_model_list}
  60 | | Append Node | ${nodes['TG']}
  61 | | Compute Path
  62 | | ${tg_if1} | ${tg}= | Next Interface
  63 | | ${dut1_if1} | ${dut1}= | Next Interface
  64 | | ${dut1_if2} | ${dut1}= | Next Interface
  65 | | ${dut2_if1} | ${dut2}= | Next Interface
  66 | | ${dut2_if2} | ${dut2}= | Next Interface
  67 | | ${tg_if2} | ${tg}= | Next Interface
  68 | | Set Suite Variable | ${tg}
  69 | | Set Suite Variable | ${tg_if1}
  70 | | Set Suite Variable | ${tg_if2}
  71 | | Set Suite Variable | ${dut1}
  72 | | Set Suite Variable | ${dut1_if1}
  73 | | Set Suite Variable | ${dut1_if2}
  74 | | Set Suite Variable | ${dut2}
  75 | | Set Suite Variable | ${dut2_if1}
  76 | | Set Suite Variable | ${dut2_if2}
  77
  78 | VPP interfaces in path are up
  79 | | [Documentation] | *Set UP state on VPP interfaces in path on nodes.*
  80 | | ...
  81 | | Set Interface State | ${dut1} | ${dut1_if1} | up
  82 | | Set Interface State | ${dut1} | ${dut1_if2} | up
  83 | | Set Interface State | ${dut2} | ${dut2_if1} | up
  84 | | Set Interface State | ${dut2} | ${dut2_if2} | up
  85 | | Vpp Node Interfaces Ready Wait | ${dut1}
  86 | | Vpp Node Interfaces Ready Wait | ${dut2}
  87
  88 | IPv4 forwarding initialized in a 3-node circular topology
  89 | | [Documentation] | Custom setup of IPv4 addresses on all DUT nodes and TG
  90 | | Set Interface State | ${dut1} | ${dut1_if1} | up
  91 | | Set Interface State | ${dut1} | ${dut1_if2} | up
  92 | | Set Interface State | ${dut2} | ${dut2_if1} | up
  93 | | Set Interface State | ${dut2} | ${dut2_if2} | up
  94 | | ${tg1_if1_mac}= | Get Interface MAC | ${tg} | ${tg_if1}
  95 | | ${tg1_if2_mac}= | Get Interface MAC | ${tg} | ${tg_if2}
  96 | | ${dut1_if1_mac}= | Get Interface MAC | ${dut1} | ${dut1_if1}
  97 | | ${dut1_if2_mac}= | Get Interface MAC | ${dut1} | ${dut1_if2}
  98 | | ${dut2_if1_mac}= | Get Interface MAC | ${dut2} | ${dut1_if1}
  99 | | ${dut2_if2_mac}= | Get Interface MAC | ${dut2} | ${dut1_if2}
 100 | | dut1_v4.set_arp | ${dut1_if1} | 10.10.10.2 | ${tg1_if1_mac}
 101 | | dut1_v4.set_arp | ${dut1_if2} | 1.1.1.2 | ${dut2_if1_mac}
 102 | | dut2_v4.set_arp | ${dut2_if1} | 1.1.1.1 | ${dut1_if2_mac}
 103 | | dut2_v4.set_arp | ${dut2_if2} | 20.20.20.2 | ${tg1_if2_mac}
 104 | | dut1_v4.set_ip | ${dut1_if1} | 10.10.10.1 | 24
 105 | | dut1_v4.set_ip | ${dut1_if2} | 1.1.1.1 | 30
 106 | | dut2_v4.set_ip | ${dut2_if1} | 1.1.1.2 | 30
 107 | | dut2_v4.set_ip | ${dut2_if2} | 20.20.20.1 | 24
 108 | | dut1_v4.set_route | 20.20.20.0 | 24 | 1.1.1.2 | ${dut1_if2}
 109 | | dut2_v4.set_route | 10.10.10.0 | 24 | 1.1.1.1 | ${dut2_if1}
 110 | | All Vpp Interfaces Ready Wait | ${nodes}
 111
 112 | IPv6 forwarding initialized in a 3-node circular topology
 113 | | [Documentation] | Custom setup of IPv6 topology on all DUT nodes
 114 | | ${prefix}= | Set Variable | 64
 115 | | ${tg1_if1_mac}= | Get Interface MAC | ${tg} | ${tg_if1}
 116 | | ${tg1_if2_mac}= | Get Interface MAC | ${tg} | ${tg_if2}
 117 | | ${dut1_if2_mac}= | Get Interface MAC | ${dut1} | ${dut1_if2}
 118 | | ${dut2_if1_mac}= | Get Interface MAC | ${dut2} | ${dut1_if1}
 119 | | VPP Set If IPv6 Addr | ${dut1} | ${dut1_if1} | 2001:1::1 | ${prefix}
 120 | | VPP Set If IPv6 Addr | ${dut1} | ${dut1_if2} | 2001:3::1 | ${prefix}
 121 | | VPP Set If IPv6 Addr | ${dut2} | ${dut2_if1} | 2001:3::2 | ${prefix}
 122 | | VPP Set If IPv6 Addr | ${dut2} | ${dut2_if2} | 2001:2::1 | ${prefix}
 123 | | Vpp nodes ra suppress link layer | ${nodes}
 124 | | Vpp set IPv6 neighbor | ${dut1} | ${dut1_if1} | 2001:1::2
 125 | | ...                    | ${tg1_if1_mac}
 126 | | Vpp set IPv6 neighbor | ${dut2} | ${dut2_if2} | 2001:2::2
 127 | | ...                    | ${tg1_if2_mac}
 128 | | Vpp set IPv6 neighbor | ${dut1} | ${dut1_if2} | 2001:3::2
 129 | | ...                    | ${dut2_if1_mac}
 130 | | Vpp set IPv6 neighbor | ${dut2} | ${dut2_if1} | 2001:3::1
 131 | | ...                    | ${dut1_if2_mac}
 132 | | Vpp Route Add | ${dut1} | 2001:2::0 | ${prefix} | 2001:3::2 | ${dut1_if2}
 133 | | Vpp Route Add | ${dut2} | 2001:1::0 | ${prefix} | 2001:3::1 | ${dut2_if1}
 134
 135 | L2 xconnect initialized in a 3-node circular topology
 136 | | [Documentation] | Custom setup of L2 xconnect topology
 137 | | L2 setup xconnect on DUT | ${dut1} | ${dut1_if1} | ${dut1_if2}
 138 | | L2 setup xconnect on DUT | ${dut2} | ${dut2_if1} | ${dut2_if2}
 139 | | All Vpp Interfaces Ready Wait | ${nodes}
 140
 141 | L2 bridge domain initialized in a 3-node circular topology
 142 | | [Documentation] | Custom setup of L2 bridge topology
 143 | | Vpp l2bd forwarding setup | ${dut1} | ${dut1_if1} | ${dut1_if2}
 144 | | Vpp l2bd forwarding setup | ${dut2} | ${dut2_if1} | ${dut2_if2}
 145 | | All Vpp Interfaces Ready Wait | ${nodes}
 146
 147 | 3-node Performance Suite Setup
 148 | | [Arguments] | ${topology_type}
 149 | | Setup default startup configuration of VPP on all DUTs
 150 | | Update All Interface Data On All Nodes | ${nodes}
 151 | | 3-node circular Topology Variables Setup
 152 | | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
 153 | | ...                          | ${dut1} | ${dut1_if1} | ${dut1_if2}
 154 | | ...                          | ${dut2} | ${dut2_if1} | ${dut2_if2}
 155 | | ...                          | ${topology_type}
 156
 157 3-node Performance Suite Setup with DUT's NIC model
 158 | | [Arguments] | ${topology_type} | ${nic_model}
 159 | | Setup default startup configuration of VPP on all DUTs
 160 | | Update All Interface Data On All Nodes | ${nodes}
 161 | | 3-node circular Topology Variables Setup with DUT interface model
 162 | | ... | ${nic_model}
 163 | | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
 164 | | ...                          | ${dut1} | ${dut1_if1} | ${dut1_if2}
 165 | | ...                          | ${dut2} | ${dut2_if1} | ${dut2_if2}
 166 | | ...                          | ${topology_type}
 167
 168 | 3-node Performance Suite Teardown
 169 | | Teardown traffic generator | ${tg}
 170
 171 | Find NDR using linear search and pps
 172 | | [Documentation] | Find throughput by using RFC2544 linear search
 173 | | [Arguments] | ${framesize} | ${start_rate} | ${step_rate}
 174 | | ...         | ${topology_type} | ${min_rate} | ${max_rate}
 175 | | ${duration}= | Set Variable | 60
 176 | | Set Duration | ${duration}
 177 | | Set Search Rate Boundaries | ${max_rate} | ${min_rate}
 178 | | Set Search Linear Step | ${step_rate}
 179 | | Set Search Frame Size | ${framesize}
 180 | | Set Search Rate Type pps
 181 | | Linear Search | ${start_rate} | ${topology_type}
 182 | | ${rate_per_stream}= | Verify Search Result
 183 | | Display result of NDR search | ${rate_per_stream} | ${framesize} | 2
 184 | | Traffic should pass with no loss | ${duration} | ${rate_per_stream}pps
 185 | | ...                              | ${framesize} | ${topology_type}
 186 | | ...                              | fail_on_loss=${False}
 187
 188 | Find NDR using binary search and pps
 189 | | [Documentation] | Find throughput by using RFC2544 binary search
 190 | | [Arguments] | ${framesize} | ${binary_min} | ${binary_max}
 191 | | ...         | ${topology_type} | ${min_rate} | ${max_rate} | ${threshold}
 192 | | ${duration}= | Set Variable | 60
 193 | | Set Duration | ${duration}
 194 | | Set Search Rate Boundaries | ${max_rate} | ${min_rate}
 195 | | Set Search Frame Size | ${framesize}
 196 | | Set Search Rate Type pps
 197 | | Set Binary Convergence Threshold | ${threshold}
 198 | | Binary Search | ${binary_min} | ${binary_max} | ${topology_type}
 199 | | ${rate_per_stream}= | Verify Search Result
 200 | | Display result of NDR search | ${rate_per_stream} | ${framesize} | 2
 201 | | Traffic should pass with no loss | ${duration} | ${rate_per_stream}pps
 202 | | ...                              | ${framesize} | ${topology_type}
 203 | | ...                              | fail_on_loss=${False}
 204
 205 | Find NDR using combined search and pps
 206 | | [Documentation] | Find throughput by using RFC2544 combined search
 207 | | ...             | (linear + binary)
 208 | | [Arguments] | ${framesize} | ${start_rate} | ${step_rate}
 209 | | ...         | ${topology_type} | ${min_rate} | ${max_rate} | ${threshold}
 210 | | ${duration}= | Set Variable | 60
 211 | | Set Duration | ${duration}
 212 | | Set Search Rate Boundaries | ${max_rate} | ${min_rate}
 213 | | Set Search Linear Step | ${step_rate}
 214 | | Set Search Frame Size | ${framesize}
 215 | | Set Search Rate Type pps
 216 | | Set Binary Convergence Threshold | ${threshold}
 217 | | Combined Search | ${start_rate} | ${topology_type}
 218 | | ${rate_per_stream}= | Verify Search Result
 219 | | Display result of NDR search | ${rate_per_stream} | ${framesize} | 2
 220 | | Traffic should pass with no loss | ${duration} | ${rate_per_stream}pps
 221 | | ...                              | ${framesize} | ${topology_type}
 222 | | ...                              | fail_on_loss=${False}
 223
 224 | Display result of NDR search
 225 | | [Documentation] | Display result of NDR search in packet per seconds (total
 226 | | ...             | and per stream) and Gbps
 227 | | [Arguments] | ${rate_per_stream} | ${framesize} | ${nr_streams}
 228 | | ${rate_total}= | Evaluate | ${rate_per_stream}*${nr_streams}
 229 | | ${bandwidth_total}= | Evaluate | ${rate_total}*(${framesize}+20)*8/(10**9)
 230 | | Set Test Message | FINAL_RATE: ${rate_total} pps
 231 | | Set Test Message | (${nr_streams}x ${rate_per_stream} pps) | append=yes
 232 | | Set Test Message | FINAL_BANDWIDTH: ${bandwidth_total} Gbps | append=yes
 233
 234 | Traffic should pass with no loss
 235 | | [Arguments] | ${duration} | ${rate} | ${framesize} | ${topology_type}
 236 | | ...         | ${fail_on_loss}=${True}
 237 | | Clear and show runtime counters with running traffic | ${duration}
 238 | | ...  | ${rate} | ${framesize} | ${topology_type}
 239 | | Clear all counters on all DUTs
 240 | | Send traffic on tg | ${duration} | ${rate} | ${framesize}
 241 | | ...                | ${topology_type} | warmup_time=0
 242 | | Show statistics on all DUTs
 243 | | Run Keyword If | ${fail_on_loss} | No traffic loss occurred
 244
 245 | Clear and show runtime counters with running traffic
 246 | | [Arguments] | ${duration} | ${rate} | ${framesize} | ${topology_type}
 247 | | Send traffic on tg | -1 | ${rate} | ${framesize}
 248 | | ...                | ${topology_type} | warmup_time=0 | async_call=True
 249 | | Clear runtime counters on all DUTs
 250 | | Sleep | ${duration}
 251 | | Show runtime counters on all DUTs
 252 | | Stop traffic on tg