Change-Id: Id1c9684c42c19eccfdc347e43221e097aed10514
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
---
title: Probabilistic Loss Ratio Search for Packet Throughput (PLRsearch)
# abbrev: PLRsearch
---
title: Probabilistic Loss Ratio Search for Packet Throughput (PLRsearch)
# abbrev: PLRsearch
-docname: draft-vpolak-bmwg-plrsearch-02
-date: 2019-07-08
+docname: draft-vpolak-bmwg-plrsearch-03
+date: 2020-03-06
ipr: trust200902
area: ops
ipr: trust200902
area: ops
+
+ FDio-CSIT-PLRsearch:
+ target: https://docs.fd.io/csit/rls2001/report/introduction/methodology_data_plane_throughput/methodology_plrsearch.html
+ title: "FD.io CSIT Test Methodology - PLRsearch"
+ date: 2020-02
+
draft-vpolak-mkonstan-bmwg-mlrsearch:
target: https://tools.ietf.org/html/draft-vpolak-mkonstan-bmwg-mlrsearch
title: "Multiple Loss Ratio Search for Packet Throughput (MLRsearch)"
draft-vpolak-mkonstan-bmwg-mlrsearch:
target: https://tools.ietf.org/html/draft-vpolak-mkonstan-bmwg-mlrsearch
title: "Multiple Loss Ratio Search for Packet Throughput (MLRsearch)"
-Network providers are interested in throughput a system can sustain.
+Network providers are interested in throughput a networking system can
+sustain.
[RFC2544] assumes loss ratio is given by a deterministic function of
offered load. But NFV software systems are not deterministic enough.
[RFC2544] assumes loss ratio is given by a deterministic function of
offered load. But NFV software systems are not deterministic enough.
+Due to the indeterministic nature of certain NFV systems that are the
+targetted by PLRsearch algorithm, existing network benchmarking terms
+are explicated and a number of new terms and assumptions are introduced.
+
## Device Under Test
In software networking, "device" denotes a specific piece of software
## Device Under Test
In software networking, "device" denotes a specific piece of software
Alternatively, trend analysis may be a part of exit conditions,
requiring longer searches for systems displaying trends.
Alternatively, trend analysis may be a part of exit conditions,
requiring longer searches for systems displaying trends.
-# Sample Implementation Specifics: FD.io CSIT
+# Known Implementations
+
+The only known working implementation of PLRsearch is in Linux
+Foundation FD.io CSIT open-source project [FDio-CSIT-PLRsearch].
+
+## FD.io CSIT Implementation Specifics
The search receives min_rate and max_rate values, to avoid measurements
at offered loads not supporeted by the traffic generator.
The search receives min_rate and max_rate values, to avoid measurements
at offered loads not supporeted by the traffic generator.
packets sent and packets lost), but debug output from traffic generator
lists unidirectional values.
packets sent and packets lost), but debug output from traffic generator
lists unidirectional values.
In a sample implemenation in FD.io CSIT project, there is roughly 0.5
second delay between trials due to restrictons imposed by packet traffic
In a sample implemenation in FD.io CSIT project, there is roughly 0.5
second delay between trials due to restrictons imposed by packet traffic
seconds for the first trial, each subsequent trial being 0.1 second
longer).
seconds for the first trial, each subsequent trial being 0.1 second
longer).
-## Rounding Errors and Underflows
+### Rounding Errors and Underflows
In order to avoid them, the current implementation tracks natural
logarithm (instead of the original quantity) for any quantity which is
In order to avoid them, the current implementation tracks natural
logarithm (instead of the original quantity) for any quantity which is
frequent operations (such as "logarithm of sum of exponentials") are
defined to handle None correctly.
frequent operations (such as "logarithm of sum of exponentials") are
defined to handle None correctly.
Current implementation uses two fitting functions. In general, their
estimates for critical rate differ, which adds a simple source of
Current implementation uses two fitting functions. In general, their
estimates for critical rate differ, which adds a simple source of
multiple "if" branches, discontinuities are a possibility at range
boundaries.
multiple "if" branches, discontinuities are a possibility at range
boundaries.
The original function (before applying logarithm) is Primitive Function
to Logistic Function. The name "stretch" is used for related a function
The original function (before applying logarithm) is Primitive Function
to Logistic Function. The name "stretch" is used for related a function
r = (a*(1 + E^(m/a))*Log[(E^(b/a) + E^(m/a))/(1 + E^(m/a))])/E^(m/a)
r = (a*(1 + E^(m/a))*Log[(E^(b/a) + E^(m/a))/(1 + E^(m/a))])/E^(m/a)
The original function is double Primitive Function to Gaussian Function.
The name "erf" comes from error function, the first primitive to
The original function is double Primitive Function to Gaussian Function.
The name "erf" comes from error function, the first primitive to
r = ((a*(E^(-((b - m)^2/a^2)) - E^(-(m^2/a^2))))/Sqrt[Pi] + m*Erfc[m/a]
+ (b - m)*Erfc[(-b + m)/a])/(1 + Erf[m/a])
r = ((a*(E^(-((b - m)^2/a^2)) - E^(-(m^2/a^2))))/Sqrt[Pi] + m*Erfc[m/a]
+ (b - m)*Erfc[(-b + m)/a])/(1 + Erf[m/a])
The numeric integrator expects all the parameters to be distributed
(independently and) uniformly on an interval (-1, 1).
The numeric integrator expects all the parameters to be distributed
(independently and) uniformly on an interval (-1, 1).
a random power between zero and one; thus it follows a Reciprocal
Distribution.
a random power between zero and one; thus it follows a Reciprocal
Distribution.
After few measurements, the posterior distribution of fitting function
arguments gets quite concentrated into a small area. The integrator is
After few measurements, the posterior distribution of fitting function
arguments gets quite concentrated into a small area. The integrator is
(dominated by whole sample population) is actually relevant for the
critical rate estimation.
(dominated by whole sample population) is actually relevant for the
critical rate estimation.
-## Offered Load Selection
+### Offered Load Selection
First two measurements are hardcoded to happen at the middle of rate interval
and at max_rate. Next two measurements follow MRR-like logic,
First two measurements are hardcoded to happen at the middle of rate interval
and at max_rate. Next two measurements follow MRR-like logic,
--- back
\ No newline at end of file
--- back
\ No newline at end of file
Code Review / csit.git / commitdiff