| Library | resources.libraries.python.TrafficGenerator.OptimizedSearch
| Library | resources.libraries.python.TrafficGenerator.TGDropRateSearchImpl
| Library | resources.libraries.python.Trace
-| Resource | resources/libraries/robot/shared/default.robot
-| Resource | resources/libraries/robot/shared/interfaces.robot
-| Resource | resources/libraries/robot/shared/counters.robot
-| Resource | resources/libraries/robot/shared/container.robot
-| Resource | resources/libraries/robot/shared/memif.robot
-| Resource | resources/libraries/robot/l2/l2_bridge_domain.robot
-| Resource | resources/libraries/robot/l2/l2_xconnect.robot
-| Resource | resources/libraries/robot/ip/ip4.robot
-| Resource | resources/libraries/robot/ip/ip6.robot
-| Resource | resources/libraries/robot/l2/tagging.robot
-| Documentation | Performance suite keywords - utilities to find and verify NDR
-| ... | and PDR.
+| ...
+| Documentation
+| ... | Performance suite keywords - utilities to find and verify NDR and PDR.
*** Keywords ***
| Find NDR and PDR intervals using optimized search
| | [Documentation]
| | ... | Find boundaries for RFC2544 compatible NDR and PDR values
| | ... | using an optimized search algorithm.
-| | ... | Display results as formatted test message.
+| | ... | Display findings as a formatted test message.
| | ... | Fail if a resulting lower bound has too high loss fraction.
| | ... | Input rates are understood as uni-directional,
| | ... | reported result contains bi-directional rates.
| | ... | Some inputs are read from variables to streamline suites.
| | ...
| | ... | *Test (or broader scope) variables read:*
-| | ... | - traffic_profile - Topology type. Type: string
-| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str
-| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps].
-| | ... | Type: float
+| | ... | - traffic_profile - Name of module defining traffc for measurements.
+| | ... | Type: string
+| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str
+| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps].
+| | ... | Type: float
| | ...
| | ... | *Arguments:*
| | ... | - packet_loss_ratio - Accepted loss during search. Type: float
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Find NDR and PDR intervals using optimized search \| \${0.005}
+| | ... | \| Find NDR and PDR intervals using optimized search \| \${0.005} \
| | ... | \| \${0.005} \| \${30.0} \| \${1.0} \| \${2} \| ${600.0} \| ${2} \|
| | ...
| | [Arguments] | ${packet_loss_ratio}=${0.005}
| | ... | ${final_trial_duration} | ${initial_trial_duration}
| | ... | ${number_of_intermediate_phases} | timeout=${timeout}
| | ... | doublings=${doublings}
-| | Display result of NDRPDR search | ${result} | ${frame_size}
+| | Display result of NDRPDR search | ${result}
| | Check NDRPDR interval validity | ${result.pdr_interval}
| | ... | ${packet_loss_ratio}
| | Check NDRPDR interval validity | ${result.ndr_interval}
+| | Perform additional measurements based on NDRPDR result
+| | ... | ${result} | ${frame_size} | ${traffic_profile}
+
+| Display Reconfig Test Message
+| | [Documentation]
+| | ... | Display the number of packets lost (bidirectionally)
+| | ... | due to reconfiguration under traffic.
+| | ...
+| | ... | *Arguments:*
+| | ... | - result - Result of bidirectional measurtement.
+| | ... | Type: ReceiveRateMeasurement
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Display Reconfig Test Message \| \${result} \|
+| | ...
+| | [Arguments] | ${result}
+| | ...
+| | Set Test Message | Packets lost due to reconfig: ${result.loss_count}
+| | ${time_lost} = | Evaluate | ${result.loss_count} / ${result.target_tr}
+| | Set Test Message | ${\n}Implied time lost: ${time_lost} | append=yes
+
+| Find Throughput Using MLRsearch
+| | [Documentation]
+| | ... | Find and return lower bound PDR (zero PLR by default)
+| | ... | bi-directional throughput using MLRsearch algorithm.
+| | ... | Input rates are understood as uni-directional,
+| | ... | reported result contains bi-directional rates.
+| | ... | Currently, the min_rate value is hardcoded to match test teardowns.
+| | ...
+| | ... | TODO: Should the trial duration of the additional
+| | ... | measurements be configurable?
+| | ...
+| | ... | Some inputs are read from variables to streamline suites.
+| | ...
+| | ... | *Test (or broader scope) variables read:*
+| | ... | - traffic_profile - Name of module defining traffc for measurements.
+| | ... | Type: string
+| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str
+| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps].
+| | ... | Type: float
+| | ...
+| | ... | *Arguments:*
+| | ... | - packet_loss_ratio - Accepted loss during search. Type: float
+| | ... | - final_relative_width - Maximal width multiple of upper. Type: float
+| | ... | - final_trial_duration - Duration of final trials [s]. Type: float
+| | ... | - initial_trial_duration - Duration of initial trials [s]. Type: float
+| | ... | - intermediate phases - Number of intermediate phases [1]. Type: int
+| | ... | - timeout - Fail if search duration is longer [s]. Type: float
+| | ... | - doublings - How many doublings to do when expanding [1]. Type: int
+| | ...
+| | ... | *Returns:*
+| | ... | - Lower bound for bi-directional throughput at given PLR. Type: float
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| \${throughpt}= \| Find Throughput Using MLRsearch \| \${0} \
+| | ... | \| \${0.001} \| \${10.0}\| \${1.0} \| \${1} \| ${720.0} \| ${2} \|
+| | ...
+| | [Arguments] | ${packet_loss_ratio}=${0.0}
+| | ... | ${final_relative_width}=${0.001} | ${final_trial_duration}=${10.0}
+| | ... | ${initial_trial_duration}=${1.0}
+| | ... | ${number_of_intermediate_phases}=${1} | ${timeout}=${720.0}
+| | ... | ${doublings}=${2}
+| | ...
+| | ${result} = | Perform optimized ndrpdr search | ${frame_size}
+| | ... | ${traffic_profile} | ${20000} | ${max_rate*2}
+| | ... | ${packet_loss_ratio} | ${final_relative_width}
+| | ... | ${final_trial_duration} | ${initial_trial_duration}
+| | ... | ${number_of_intermediate_phases} | timeout=${timeout}
+| | ... | doublings=${doublings}
+| | Return From Keyword | ${result.pdr_interval.measured_low.transmit_rate}
| Find critical load using PLRsearch
| | [Documentation]
| | ... | Some inputs are read from variables to streamline suites.
| | ...
| | ... | *Test (or broader scope) variables read:*
-| | ... | - traffic_profile - Topology type. Type: string
-| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str
-| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps].
-| | ... | Type: float
+| | ... | - traffic_profile - Name of module defining traffc for measurements.
+| | ... | Type: string
+| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str
+| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps].
+| | ... | Type: float
| | ...
| | ... | *Arguments:*
| | ... | - packet_loss_ratio - Accepted loss during search. Type: float
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Find critical load usingPLR search \| \${1e-7} \| \${1800} \|
+| | ... | \| Find critical load using PLR search \| \${1e-7} \| \${120} \|
| | ...
| | [Arguments] | ${packet_loss_ratio}=${1e-7} | ${timeout}=${1800.0}
| | ...
| | ... | ${traffic_profile} | ${min_rate} | ${max_rate*2}
| | ... | ${packet_loss_ratio} | timeout=${timeout}
| | ${lower} | ${upper} = | Display result of soak search
-| | ... | ${average} | ${stdev} | ${frame_size}
+| | ... | ${average} | ${stdev}
| | Should Not Be True | ${lower} < ${min_rate}
| | ... | Lower bound ${lower} is below bidirectional minimum ${min_rate}.
| | ... | (Throughput * (L2 Frame Size + IPG) * 8) / Max bitrate of NIC
| | ... | The given result should contain latency data as well.
| | ...
+| | ... | *Test (or broader scope) variables read:*
+| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str
| | ... | *Arguments:*
| | ... | - result - Measured result data per stream [pps]. Type: NdrPdrResult
-| | ... | - frame_size - L2 Frame Size [B]. Type: integer
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Display result of NDRPDR search \| \${result} \| \${64} \|
+| | ... | \| Display result of NDRPDR search \| \${result} \|
| | ...
-| | [Arguments] | ${result} | ${frame_size}
+| | [Arguments] | ${result}
| | ...
| | ${frame_size} = | Get Average Frame Size | ${frame_size}
| | Display single bound | NDR_LOWER
| | ... | TODO: Do we want to report some latency data,
| | ... | even if not measured at the reported bounds?.
| | ...
+| | ... | *Test (or broader scope) variables read:*
+| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str
| | ... | *Arguments:*
| | ... | - avg - Estimated average critical load [pps]. Type: float
| | ... | - stdev - Standard deviation of critical load [pps]. Type: float
-| | ... | - frame_size - L2 Frame Size [B]. Type: integer
| | ...
| | ... | *Returns:*
| | ... | - Lower and upper bound of critical load [pps]. Type: 2-tuple of float
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Display result of soak search \| \${100000} \| \${100} \| \${64} \|
+| | ... | \| Display result of soak search \| \${100000} \| \${100} \|
| | ...
-| | [Arguments] | ${avg} | ${stdev} | ${frame_size}
+| | [Arguments] | ${avg} | ${stdev}
| | ...
| | ${frame_size} = | Get Average Frame Size | ${frame_size}
| | ${avg} = | Convert To Number | ${avg}
| | ... | ${message}${\n}${message_zero} | ${message}${\n}${message_other}
| | Fail | ${message}
+| Perform additional measurements based on NDRPDR result
+| | [Documentation]
+| | ... | Perform any additional measurements which are not directly needed
+| | ... | for determining NDR nor PDR, but which are needed for gathering
+| | ... | additional data for debug purposes.
+| | ... | Currently, just "Traffic should pass with no loss" is called.
+| | ... | TODO: Move latency measurements from optimized search here.
+| | ...
+| | ... | *Arguments:*
+| | ... | - result - Measured result data per stream [pps]. Type: NdrPdrResult
+| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str
+| | ... | - traffic_profile - Topology profile. Type: string
+| | ...
+| | ... | *Example:*
+| | ... | \| Perform additional measurements based on NDRPDR result \
+| | ... | \| \${result} \| ${64} \| 3-node-IPv4 \|
+| | ...
+| | [Arguments] | ${result} | ${framesize} | ${traffic_profile}
+| | ...
+| | ${duration}= | Set Variable | 2.0
+| | ${rate_per_stream}= | Evaluate
+| | ... | ${result.ndr_interval.measured_low.target_tr} / 2.0
+| | Traffic should pass with no loss | ${duration} | ${rate_per_stream}pps
+| | ... | ${framesize} | ${traffic_profile} | fail_on_loss=${False}
+
| Traffic should pass with no loss
| | [Documentation]
| | ... | Send traffic at specified rate. No packet loss is accepted at loss
| | ... | - duration - Duration of traffic run [s]. Type: integer
| | ... | - rate - Rate for sending packets. Type: string
| | ... | - frame_size - L2 Frame Size [B] or IMIX_v4_1. Type: integer/string
-| | ... | - traffic_profile - Topology type. Type: string
+| | ... | - traffic_profile - Name of module defining traffc for measurements.
+| | ... | Type: string
| | ... | - fail_on_loss - If True, the keyword fails if loss occurred.
| | ... | Type: boolean
| | ...
| | ... | Some inputs are read from variables to streamline suites.
| | ...
| | ... | *Test (or broader scope) variables read:*
-| | ... | - traffic_profile - Topology type. Type: string
-| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str
-| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps].
-| | ... | Type: float
+| | ... | - traffic_profile - Name of module defining traffc for measurements.
+| | ... | Type: string
+| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str
+| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps].
+| | ... | Type: float
| | ...
| | ... | *Arguments:*
| | ... | - subsamples - How many trials in this measurement. Type: int
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Traffic should pass with maximum rate \| ${1} \| ${10.0} \|
+| | ... | \| Traffic should pass with maximum rate \| ${1} \| ${10.0} \
| | ... | \| ${False} \| ${False} \| ${0} \| ${1} \|
| | ...
| | [Arguments] | ${trial_duration}=${perf_trial_duration}
| | ... | - trial_duration - Duration of single trial [s]. Type: float
| | ... | - rate - Rate for sending packets. Type: string
| | ... | - frame_size - L2 Frame Size [B]. Type: integer/string
-| | ... | - traffic_profile - Topology type. Type: string
+| | ... | - traffic_profile - Name of module defining traffc for measurements.
+| | ... | Type: string
| | ... | - subsamples - How many trials in this measurement. Type: int
| | ... | - unidirection - False if traffic is bidirectional. Type: boolean
| | ... | - tx_port - TX port of TG, default 0. Type: integer
| | ... | - rx_port - RX port of TG, default 1. Type: integer
+| | ... | - pkt_trace - True to enable packet trace. Type: boolean
| | ...
| | ... | *Example:*
| | ...
| | ... | \| Send traffic at specified rate \| ${1.0} \| 4.0mpps \| 64 \
-| | ... | \| 3-node-IPv4 \| ${10} \| ${False} \| ${0} | ${1} \|
+| | ... | \| 3-node-IPv4 \| ${10} \| ${False} \| ${0} | ${1} \| ${False}
| | ...
| | [Arguments] | ${trial_duration} | ${rate} | ${frame_size}
| | ... | ${traffic_profile} | ${subsamples}=${1} | ${unidirection}=${False}
-| | ... | ${tx_port}=${0} | ${rx_port}=${1}
+| | ... | ${tx_port}=${0} | ${rx_port}=${1} | ${pkt_trace}=${False}
| | ...
| | Clear and show runtime counters with running traffic | ${trial_duration}
| | ... | ${rate} | ${frame_size} | ${traffic_profile}
| | ... | ${unidirection} | ${tx_port} | ${rx_port}
| | Run Keyword If | ${dut_stats}==${True} | Clear all counters on all DUTs
| | Run Keyword If | ${dut_stats}==${True} and ${pkt_trace}==${True}
-| | ... | VPP Enable Traces On All DUTs | ${nodes}
+| | ... | VPP Enable Traces On All DUTs | ${nodes} | fail_on_error=${False}
| | Run Keyword If | ${dut_stats}==${True}
| | ... | VPP enable elog traces on all DUTs | ${nodes}
| | ${results} = | Create List
| | ...
| | ... | *Arguments:*
| | ... | - duration - Duration of traffic run [s]. Type: integer
-| | ... | - rate - Rate for sending packets. Type: string
+| | ... | - rate - Unidirectional rate for sending packets. Type: string
| | ... | - frame_size - L2 Frame Size [B] or IMIX_v4_1. Type: integer/string
-| | ... | - traffic_profile - Topology type. Type: string
+| | ... | - traffic_profile - Name of module defining traffc for measurements.
+| | ... | Type: string
| | ... | - unidirection - False if traffic is bidirectional. Type: boolean
| | ... | - tx_port - TX port of TG, default 0. Type: integer
| | ... | - rx_port - RX port of TG, default 1. Type: integer
| | ... | Show runtime counters on all DUTs | ${nodes}
| | Stop traffic on tg
-| Create network function CPU list
-| | # TODO: Is there a better place for this keyword?
-| | # It is not exactly a performance utility.
+| Start Traffic on Background
| | [Documentation]
-| | ... | Create list of CPUs allocated for network function base on SUT/DUT
-| | ... | placement and other network functions placement.
+| | ... | Start traffic at specified rate then return control to Robot.
| | ...
+| | ... | This keyword is useful if the test needs to do something
+| | ... | while traffic is running.
+| | ... | Just a wrapper around L1 keyword.
+| | ... |
+| | ... | TODO: How to make sure the traffic is stopped on any failure?
+| | ...
+| | ... | *Test (or broader scope) variables read:*
+| | ... | - traffic_profile - Name of module defining traffc for measurements.
+| | ... | Type: string
+| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str
| | ... | *Arguments:*
-| | ... | - dut - DUT node. Type: dictionary
-| | ... | - chains: Total number of chains. Type: integer
-| | ... | - nodeness: Total number of nodes per chain. Type: integer
-| | ... | - chain_id - Network function chain ID. Type: integer
-| | ... | - node_id - Network function node ID within chain. Type: integer
-| | ... | - mtcr - Main thread to core ratio. Type: integer
-| | ... | - dtcr - Dataplane thread to core ratio. Type: integer
-| | ... | - auto_scale - If True, use same amount of Dataplane threads for
-| | ... | network function as DUT, otherwise use single physical core for
-| | ... | every network function. Type: boolean
-| | ...
-| | ... | *Note:*
-| | ... | KW uses test variables \${cpu_count_int} set by
-| | ... | "Add worker threads and rxqueues to all DUTs"
+| | ... | - rate - Unidirectional rate for sending packets. Type: string
+| | ... | - unidirection - False if traffic is bidirectional. Type: boolean
+| | ... | - tx_port - TX port of TG, default 0. Type: integer
+| | ... | - rx_port - RX port of TG, default 1. Type: integer
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Start Traffic on Background \| 4.0mpps \| ${False} \| ${0} \
+| | ... | \| ${1} \|
+| | ...
+| | [Arguments] | ${rate} | ${unidirection}=${False} | ${tx_port}=${0}
+| | ... | ${rx_port}=${1}
+| | ...
+| | # Duration of -1 means we will stop traffic manually.
+| | Send traffic on tg | ${-1} | ${rate} | ${frame_size} | ${traffic_profile}
+| | ... | warmup_time=${0} | async_call=${True} | latency=${False}
+| | ... | unidirection=${unidirection} | tx_port=${tx_port} | rx_port=${rx_port}
+
+| Stop Running Traffic
+| | [Documentation]
+| | ... | Stop the running traffic, return measurement result.
+| | ... | For bidirectional traffic, the reported values are bi-directional.
+| | ...
+| | ... | Just a wrapper around L1 keyword.
+| | ... |
+| | ... | TODO: Tolerate if traffic was not started.
+| | ...
+| | ... | *Returns:*
+| | ... | - Measurement result. Type: ReceiveRateMeasurement
| | ...
| | ... | *Example:*
| | ...
-| | ... | \| Create network function CPU list \| ${nodes['DUT1']} \
-| | ... | \| 1 \| 1 \| 1 \| 1 \|
-| | ...
-| | [Arguments] | ${dut} | ${chains}=${1} | ${nodeness}=${1} | ${chain_id}=${1}
-| | ... | ${node_id}=${1} | ${mtcr}=${2} | ${dtcr}=${1} | ${auto_scale}=${False}
-| | ...
-| | ${sut_sc}= | Set Variable | ${1}
-| | ${dut_mc}= | Set Variable | ${1}
-| | ${dut_dc}= | Set Variable | ${cpu_count_int}
-| | ${skip}= | Evaluate | ${sut_sc} + ${dut_mc} + ${dut_dc}
-| | ${dtc}= | Set Variable If | ${auto_scale} | ${cpu_count_int} | ${1}
-| | ${if1_status} | ${value}= | Run Keyword And Ignore Error
-| | ... | Variable Should Exist | ${${dut}_if1}
-| | @{if_list}= | Run Keyword If | '${if1_status}' == 'PASS'
-| | ... | Create List | ${${dut}_if1}
-| | ... | ELSE | Create List | ${${dut}_if1_1} | ${${dut}_if1_2}
-| | ${if2_status} | ${value}= | Run Keyword And Ignore Error
-| | ... | Variable Should Exist | ${${dut}_if2}
-| | Run Keyword If | '${if2_status}' == 'PASS'
-| | ... | Append To List | ${if_list} | ${${dut}_if2}
-| | ... | ELSE | Append To List | ${if_list} | ${${dut}_if2_1} | ${${dut}_if2_2}
-| | ${dut_numa}= | Get interfaces numa node | ${nodes['${dut}']} | @{if_list}
-| | ${nf_cpus}= | Cpu slice of list for NF | node=${nodes['${dut}']}
-| | ... | cpu_node=${dut_numa} | chains=${chains} | nodeness=${nodeness}
-| | ... | chain_id=${chain_id} | node_id=${node_id} | mtcr=${mtcr}
-| | ... | dtcr=${dtcr} | dtc=${dtc} | skip_cnt=${skip}
-| | Return From Keyword | ${nf_cpus}
+| | ... | \${result}= \| Stop Running Traffic \|
+| | ...
+| | ${result}= | Stop traffic on tg
+| | Return From Keyword | ${result}
Code Review / csit.git / blobdiff