Tibor Frank [Mon, 7 Oct 2019 08:13:42 +0000 (10:13 +0200)]
Report: Show TSH results
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: Ibdb2b098dfe57280ba60b0df6e4e330e44ed783a
Tibor Frank [Mon, 7 Oct 2019 06:45:04 +0000 (08:45 +0200)]
Report: Add data
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: I55abf8d6d5b35beb7af6c887e0c6d8c3d94c5f83
Vratko Polak [Wed, 2 Oct 2019 15:08:39 +0000 (17:08 +0200)]
PAL: Accept reconf time in scientific notation
+ Otherwise "9.
58828205237e-05" is understood as "9.
58828205237".
+ Replace "[0-9]" with "\d".
Change-Id: I7ecf6d0116fe704ecdba46f3a0e7975aba79aa62
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit
2eb470aa8b86bd11fb6365f0fd9a77ee026dc5aa)
Tibor Frank [Fri, 4 Oct 2019 07:26:07 +0000 (09:26 +0200)]
Report: Add data
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: Ia040942d3f4163377493ec6d8408766730819ad8
Tibor Frank [Thu, 3 Oct 2019 07:35:56 +0000 (09:35 +0200)]
Report: Add data and configure 1908.1.41
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: I3cad74cc1f183ba86d5e7377d68bbdd58c8ad42c
Tibor Frank [Wed, 2 Oct 2019 08:31:15 +0000 (10:31 +0200)]
Report: Add data
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: I6b361efc592985ff068bd4e4bf50595dc68d51ab
Tibor Frank [Tue, 1 Oct 2019 07:18:22 +0000 (09:18 +0200)]
Report: Add data
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: Iab81330ae8945748d9235293932de762616d624d
Tibor Frank [Mon, 30 Sep 2019 10:51:22 +0000 (12:51 +0200)]
Report: Add data
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: I89234d2a5c4aeb26c47da73943686d402374ddcc
Jan Gelety [Thu, 26 Sep 2019 05:34:04 +0000 (07:34 +0200)]
docs: Update content of test_select_lists
Signed-off-by: Jan Gelety <jgelety@cisco.com>
Change-Id: I2e993f342920ee7516cc6e90a016ecb205b3931c
Tibor Frank [Fri, 27 Sep 2019 08:29:13 +0000 (10:29 +0200)]
Report: Add data
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: I5160f54b01e6ee002cbc49c59dd8232d338cf5c4
Tibor Frank [Thu, 26 Sep 2019 11:39:59 +0000 (13:39 +0200)]
Report: Reconf tests: Fix legend
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: I6ff5505e2d33d4dc1ce7cdfd028f6c8dc8143eb8
Tibor Frank [Thu, 26 Sep 2019 10:48:38 +0000 (12:48 +0200)]
Report: Reconf tests: set autorange for y axis
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: I1811fb1b88e6e2c2f687d2c7adb6f744740803e2
Jan Gelety [Wed, 25 Sep 2019 18:10:58 +0000 (20:10 +0200)]
doc: perf release notes
Signed-off-by: Jan Gelety <jgelety@cisco.com>
Change-Id: I5a6a6606ec111e01408f8636074ec3695c50dc0f
Tibor Frank [Thu, 26 Sep 2019 08:30:34 +0000 (10:30 +0200)]
Report: Configure graphs, add data
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: Id82d3e321244e54b522ea160a688cc3398703e23
Tibor Frank [Thu, 26 Sep 2019 07:16:32 +0000 (09:16 +0200)]
Report: Configure 1908.1.40
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: I5f92b1cf84c61ebab5606092e20361ea4c416f1a
Jan Gelety [Fri, 20 Sep 2019 21:16:36 +0000 (23:16 +0200)]
report: func release notes
Signed-off-by: Jan Gelety <jgelety@cisco.com>
Change-Id: I94578bf0d70d32387dd956f5c7924976ebf362d2
Tibor Frank [Wed, 25 Sep 2019 09:33:52 +0000 (11:33 +0200)]
Report: Remove disclaimer from reconf tests.
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: I7127e0b0a08d0c1cebda6ad06f6d89994a1903b5
Tibor Frank [Fri, 20 Sep 2019 07:46:58 +0000 (09:46 +0200)]
Report: Configure 1908.1.39
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Change-Id: Ibf041efa086e5e3965582cefc37327f32ec24dfd
Vratko Polak [Mon, 23 Sep 2019 13:00:00 +0000 (15:00 +0200)]
Reconf tests: Fix async measurements
TRex does not zero the server counters.
It copies the values to use as reference,
and subtracts them when asked for results.
But the reference is stored in the client (not the server).
And CSIT uses different scripts to start and stop async traffic,
which means different clients.
This patch introduces a workaround.
Async start will return xstats objects to use as reference,
and async stop will use the objects to compute the correct results.
The xstats objects are stored in TrafficGenerator instance.
Sync measurement does not export the counters, to shorten logs.
Other improvements:
+ Make stop_traffic_on_tg return measurement results directly.
+ Rename --async to --async_start as "async" is reserved in Python 3.7
+ Minor pylint, docstring and typo fixes.
Change-Id: I5fc56a0763afb7d62cfa7c0651f96b6867de3e15
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Jan Gelety [Thu, 19 Sep 2019 11:29:10 +0000 (13:29 +0200)]
Use vpp 19.08.1-release builds
Signed-off-by: Jan Gelety <jgelety@cisco.com>
Change-Id: I3a35176e72e5805b39df03576c9d5b96a3875794
Tibor Frank [Wed, 18 Sep 2019 14:42:22 +0000 (16:42 +0200)]
Report: Fix formating of static content for reconf tests
Change-Id: I5b93749b13af870955c1cf9eb8053c446356c329
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
0608915e1a87f9a5f33e61c9c7cb93c98e9fc5ba)
Tibor Frank [Wed, 18 Sep 2019 08:40:17 +0000 (10:40 +0200)]
Report: Add disclaimer for reconf tests
+ Improve description for reconf tests
Change-Id: Ib1f397eda88a6c1db79b7a3a518584420770fd5c
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
2fc48e8d0a4712ec99dd2bb3a814ec9cb2bc12cf)
Tibor Frank [Tue, 17 Sep 2019 07:23:07 +0000 (09:23 +0200)]
Report: Enable pdf version
Change-Id: I2c88a8b5939cbe4498d5bc44de3618cb717a9a7f
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
73cabdc86a0dd86755cfa921379fc3c63142ed6e)
Tibor Frank [Tue, 17 Sep 2019 09:15:03 +0000 (11:15 +0200)]
Report: Fix Report History
Change-Id: I654677ca9252e592e635eab28577bb3bb3b8bf85
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
b61e34657e58cbe6c8476f6708d62168ffd80775)
Tibor Frank [Tue, 17 Sep 2019 07:07:06 +0000 (09:07 +0200)]
Report: Update Report History
Change-Id: Ie16a2f2298eeaf0793171c7268a95abb6986246a
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
2007221103d73be24c6d547ebaf6c86dd6dff1bf)
Vratko Polak [Mon, 16 Sep 2019 16:34:29 +0000 (18:34 +0200)]
Report: Add methodology for reconf tests
Change-Id: I833d63f4e305ba60c94aa16cbb3a31a52ee3b3bb
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit
abca97bfc198c4f947571462cc57732d187c3c20)
Tibor Frank [Fri, 13 Sep 2019 08:17:08 +0000 (10:17 +0200)]
Report: Add data
Change-Id: Ibef7599c5438624512d547af25b2026580e2bc92
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
48b37104e999ce75069672e5b901afeb06b9312b)
Tibor Frank [Fri, 13 Sep 2019 11:53:23 +0000 (13:53 +0200)]
Report: Fixes
Change-Id: Id14704f3c1526655adeaa51f0a4198b6309a64eb
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
e2d15edc355f0c70df578dde0495f8ffd85d8c12)
Tibor Frank [Fri, 13 Sep 2019 06:56:08 +0000 (08:56 +0200)]
Report: Add data
Change-Id: I589a236919af913bb37d2db3b4079b16f1e64242
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
17220bfce36c723a3ca6fdd1ef757709ab50f4eb)
Tibor Frank [Thu, 12 Sep 2019 13:16:35 +0000 (15:16 +0200)]
Report: Add nfv reconf
Change-Id: If773f5d05c845b0d15bd8f60a44a36adcbf65b82
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
8f03b515387df1ebb7e5a7231a287f7816e57726)
Tibor Frank [Wed, 11 Sep 2019 09:55:13 +0000 (11:55 +0200)]
PAL: Add NFV reconf tests
Change-Id: I6fe36f05423c3c43081872153adf0f1a899381b6
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
5979e1020d4e41e08f1fe4e6d195a0a5bc020bb1)
Tibor Frank [Thu, 12 Sep 2019 07:53:49 +0000 (09:53 +0200)]
Report: Temporarily remove pdf version
Change-Id: Iab5909defc86ec31ba726eb2a1a4b6530cf7d63d
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
c77230ed462b0a79ca0200fabdba41b78e8d7923)
Tibor Frank [Thu, 12 Sep 2019 06:53:21 +0000 (08:53 +0200)]
Report: Configure 1908.38
Change-Id: I0239e6c567c50c211ddabffa4b1bf22cdcec6462
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
e248c3fd06b4126b780eca85c5aae282cf872524)
Tibor Frank [Wed, 11 Sep 2019 11:14:40 +0000 (13:14 +0200)]
Report: Remove NFV full data
Change-Id: I51402892df91cfc5e8f2c18f18ef67bab634bcdf
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
2fe2a2c140bffa10678e3e217c8a5cba0fd4dbd0)
Tibor Frank [Wed, 11 Sep 2019 08:18:49 +0000 (10:18 +0200)]
Report: Add data
Change-Id: I2092c6420a40e1c3199183e580f00eaca37706c6
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
27cff25483728d50e0215b895c680fb88f6ffe99)
Tibor Frank [Tue, 10 Sep 2019 11:50:24 +0000 (13:50 +0200)]
Report: Add pdf version
Change-Id: I8897a6ef87b3813db0485fa081fdd44b4e501e41
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
d44bf1d08991f9698b8d02d72ecbc88fbfff9515)
Tibor Frank [Fri, 6 Sep 2019 11:54:11 +0000 (13:54 +0200)]
PAL: Add replacing of data in tables
+ vhost data
Change-Id: I86fc97ba5c3b0feffec2dfb0ac11c5d3c9d53a81
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
4dc7d6cc7b695c21ef1e37fa6b754958fc50e484)
Tibor Frank [Mon, 9 Sep 2019 12:55:26 +0000 (14:55 +0200)]
Report: Report history
Change-Id: Ib74a10ccd589111dd3c830adec9293ff2c698772
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
fa6124e5a9ce40348a4113b7da009e99d231d437)
Tibor Frank [Mon, 9 Sep 2019 10:58:25 +0000 (12:58 +0200)]
Report: Add dnv mrr data
Change-Id: I3dcc813dce7b8938be6b2caaaf1e64ce9abb7fe9
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
80fd7db87eb177b2369cd01716662c63a642a513)
Tibor Frank [Mon, 9 Sep 2019 10:16:36 +0000 (12:16 +0200)]
Report: Fix tsh and dnv plots
Change-Id: I1d40849bc15f4433117af91f71e951a61924d732
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
620774fdc9a0fa2703fcaa8e18915d65cb2b1ed3)
Tibor Frank [Mon, 9 Sep 2019 07:01:18 +0000 (09:01 +0200)]
Report: Add soak tests
Change-Id: I2161c8d8e54cfa35cd3febdda1cb47d6cec424a0
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
ba14f2b0ac026e537c79a61e800176b3404bde4d)
Tibor Frank [Fri, 6 Sep 2019 17:41:15 +0000 (17:41 +0000)]
Report: Fix2 - Add 3n-dnv plots and tables
Change-Id: Ia182e6e52a37f9b3248eab60ffe4dd4746ed7fa6
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
81907979cdf2ca6ca991410e8151170deb380bc1)
Jan Gelety [Tue, 3 Sep 2019 20:33:45 +0000 (22:33 +0200)]
docs: add select list of tests for report - tsh and dnv
Change-Id: I63a69d998c80029422bae9f90e186eb2286196d1
Signed-off-by: Jan Gelety <jgelety@cisco.com>
(cherry picked from commit
a3da59a916817208f69cb718d00a5017d0e5c0e2)
Tibor Frank [Fri, 6 Sep 2019 07:43:53 +0000 (09:43 +0200)]
Report: Fix - Add 3n-dnv plots and tables
Change-Id: I5f09f1f20e532e71d6abde02111a90f5868035f3
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
1c77031813057a50781784fc8e34019805d222b8)
Tibor Frank [Fri, 6 Sep 2019 06:13:09 +0000 (08:13 +0200)]
Report: Add 3n-dnv plots and tables
Change-Id: I6b65231f928b6168e0696f35fc253ce513163e50
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
1082d79a51e3caf60586e6176ee89ac3aadf04fc)
Tibor Frank [Thu, 5 Sep 2019 12:01:09 +0000 (14:01 +0200)]
Report: Add 2n-dnv plots and tables
Change-Id: Ibbd1775bec26817a4c193bc67ba03b29b8ddfff7
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
99219815466588419bd6705ff2ad8a9d9fc90dff)
Tibor Frank [Thu, 5 Sep 2019 08:49:33 +0000 (10:49 +0200)]
Report: Add DNV data
Change-Id: I020373b1553300459654ded31d940b0a5562eb8a
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
3eb09745cc7a936dc91a68a4a723867c2311f3a1)
Tibor Frank [Thu, 5 Sep 2019 07:54:39 +0000 (09:54 +0200)]
Report: Fix listing order of new vs. see footnote
Change-Id: I01426c3e4d40af0902e7776088e0489e141a2362
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
4751baad9a204c27b0566e4faf00bf0b30f7e170)
Tibor Frank [Thu, 5 Sep 2019 06:40:20 +0000 (08:40 +0200)]
Report: Fix hsw vs. skx comparison table
Change-Id: Id16b161ad27102891a58d084185d17a2b8fcae6a
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
ac02e01bd4b0625e91a8eb3fa0f13f6f32f74527)
Tibor Frank [Thu, 5 Sep 2019 06:27:10 +0000 (08:27 +0200)]
Report: Fix fonts in release notes table
Change-Id: I4a3486c313c4aa4754c455389b8ebb363ead469a
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
1a330e122eba0526960a581b32c0d7031c72f394)
Tibor Frank [Thu, 5 Sep 2019 06:20:32 +0000 (08:20 +0200)]
Report: Configure Report 1908.37
Change-Id: I1d883dbb299ecef151c7d7a9c5612ae0330e3538
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
d7813d33fef48421377c7eb6f24cd3998d8b0594)
Jan Gelety [Mon, 2 Sep 2019 21:28:30 +0000 (23:28 +0200)]
FIX: Use vhost macs when testpmd_mac forwarding used in VM
Change-Id: I291c3dc852e7c1bde6fb6102a9168cfec9992682
Signed-off-by: Jan Gelety <jgelety@cisco.com>
(cherry picked from commit
474034ac24ce89cdaa94a09ae4d9ebbd3fb0f219)
Maciek Konstantynowicz [Wed, 4 Sep 2019 11:56:55 +0000 (12:56 +0100)]
docs: report vpp performance rls notes update.
Change-Id: Ie4a7a80cad9ff92b9f30953096cca8a029edc687
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
(cherry picked from commit
1abf61ca0224226e3f39ac491b2b84e49ac6092f)
Vratko Polak [Wed, 4 Sep 2019 10:51:50 +0000 (12:51 +0200)]
Report methodology: PLRsearch update with graphs
Change-Id: Ib73a64c93bdba3945b8237783a24134b47d4eb99
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit
55a8e9aa3a8583b46192ffdad9ef7265fa63d537)
Maciek Konstantynowicz [Wed, 4 Sep 2019 11:11:54 +0000 (12:11 +0100)]
docs: report packet_throughput_graphs text formatting fixes.
Change-Id: Ibea6e70e2fcf9cfeb7627ee1cbcfa6e177c0b2c6
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
(cherry picked from commit
32605fc1b4ff207776b6ff3f6a1975f77d8e4629)
Peter Mikus [Wed, 4 Sep 2019 09:04:40 +0000 (09:04 +0000)]
Doc: Report release notes (perf)
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I8c1667efec67febd375818bc1dfe1d9bebc70708
(cherry picked from commit
b8ae239848d484cd0660fe406ae2c4a34c0551e0)
Maciek Konstantynowicz [Wed, 28 Aug 2019 13:41:07 +0000 (14:41 +0100)]
docs: added methodology terminology section to report.
Change-Id: Id787d12b577165a1081dba84b610c32df57a7c44
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
(cherry picked from commit
622726926fc12522e439455ade436f6dfcc0da06)
Maciek Konstantynowicz [Mon, 2 Sep 2019 18:29:05 +0000 (19:29 +0100)]
docs: report updates in physical_testbeds section.
Change-Id: Id31e6ed294dc80619a108296be580925509bb7d7
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
(cherry picked from commit
a10342c5b6d989f64bd1fe49f0edd0c7585e54ea)
Maciek Konstantynowicz [Tue, 3 Sep 2019 21:16:30 +0000 (22:16 +0100)]
docs: updated report intro in packet_throughput_graphs
Change-Id: I3bf70cc7e48ef2134841bc4bf9300cda6a118cb3
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
(cherry picked from commit
6898a7671b00c5e5c48c58fdfaeb47bb7148d4bb)
Maciek Konstantynowicz [Tue, 3 Sep 2019 21:37:45 +0000 (22:37 +0100)]
docs: added 3n-tsh testbed tests to report_test_groups.txt
Change-Id: Ib2249ce6c28f4c07878eb5fc2919c56dd8637376
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
(cherry picked from commit
7d7d7ff65943fa370c515c5d350b141a0515a074)
Tibor Frank [Wed, 4 Sep 2019 07:21:26 +0000 (09:21 +0200)]
Report: ixgbe for tsh
Change-Id: I707a654b8853926f4cb1b78f3c65b55925e95ac1
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
d53a4da7e088fa8330e851cee3c6309ef13c8360)
Tibor Frank [Wed, 4 Sep 2019 06:47:28 +0000 (08:47 +0200)]
Report: Add data
Change-Id: I6a32e0bdd1a693550214e7bea49b76313449f24d
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
4be26a2f7f0cf0539e198864d69bd34bd278dda4)
Tibor Frank [Tue, 3 Sep 2019 20:26:25 +0000 (20:26 +0000)]
Report: Comparison tables
Change-Id: I108e1292c5de8024060c220b877b668b8843c68f
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
6a5bc78e5fca4f43619c9199ef05ecbe2bc5d356)
Tibor Frank [Tue, 3 Sep 2019 17:52:02 +0000 (17:52 +0000)]
Report: Comparison tables
Change-Id: I90f635f62573154e37a319ca1cbb760401d17c05
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
92d09115493a50599ed5bea9126be28229b4211f)
Tibor Frank [Tue, 3 Sep 2019 12:08:37 +0000 (14:08 +0200)]
Report: Comparison tables
Change-Id: Ib4132c4d25f6a7c936793f403877a5819a6dd55a
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
ac688f80adb649bd04ecbe1daa79748a439a9e19)
Tibor Frank [Tue, 3 Sep 2019 09:07:36 +0000 (11:07 +0200)]
Report: add data
Change-Id: I51fd0b9e07d09224452f842689925d1166246f19
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
54b25e85bf3efd1709043ded2d6a525fa0c73962)
Tibor Frank [Tue, 3 Sep 2019 06:56:27 +0000 (08:56 +0200)]
CSIT-1590: Performance comparison analysis
Change-Id: I802b5b9cbeb486587f395aeca76cbe3df68b40c4
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
259620ea6ca1c5864d4864f0f04d260e89ecb72a)
Tibor Frank [Tue, 3 Sep 2019 06:45:31 +0000 (08:45 +0200)]
CSIT-1590: Performance comparison analysis
Change-Id: I09d502fcd7b286e41b141e43933bcf09262a8749
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
3e5dbfab796da168865ba787382224b1b04b83a4)
Tibor Frank [Mon, 2 Sep 2019 16:48:37 +0000 (18:48 +0200)]
PAL: list all sel tests in table_performance_comparison
Change-Id: I99ccbca3e56a58a662c569d195ecdef20d3ea348
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
ff3201774cf904bad6c64adf11f1fae92d5f8588)
Tibor Frank [Mon, 2 Sep 2019 13:52:06 +0000 (15:52 +0200)]
PAL: list all sel tests in table_performance_comparison_nic
Change-Id: I21bb76aab4a1e627a13057f896ea7bfc34c03463
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
0a67083bea6b5e740c89fcc6c9143e0f035eadd1)
Tibor Frank [Mon, 2 Sep 2019 13:11:20 +0000 (15:11 +0200)]
Report: Add data
Change-Id: If799a7a1d1b8f6ea59ebfba3dd4724b09e7680f2
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
6d981f513fba8cbec434f5c10f83081034372f20)
Peter Mikus [Sun, 1 Sep 2019 05:51:07 +0000 (05:51 +0000)]
Add: Testpmd VM test
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: Id39667515e304f4d92f230f82d3f1dbc1e22ac32
(cherry picked from commit
8beb22fb0a13ccc69281bb4a68b62a733d1f20e2)
Jan Gelety [Mon, 26 Aug 2019 03:41:53 +0000 (05:41 +0200)]
Report: functional tests static content
Change-Id: I17238ff6d1760f6a956459e072ba0303abef3199
Signed-off-by: Jan Gelety <jgelety@cisco.com>
(cherry picked from commit
97881b4e0142f98bbb8e1d54828bbc3c1b34d4ac)
Tibor Frank [Mon, 2 Sep 2019 10:32:32 +0000 (12:32 +0200)]
Report: Add 3n-tsh
Change-Id: I6027b635191c7e64397915df8944699bb320daf3
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
a7886e481f492d996643fad5ab48944f1cd4f725)
Tibor Frank [Fri, 30 Aug 2019 09:55:42 +0000 (11:55 +0200)]
Report: Add data
Change-Id: Ie9350530879fef7d2ae8c0b58f6fd799fde9fff8
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
603931f00dab6e4375f1ca9dff9f5877c7ed5135)
Maciek Konstantynowicz [Wed, 28 Aug 2019 12:38:34 +0000 (13:38 +0100)]
presentation: fixed suite names 10ge>25ge, 10ge>40ge based on nic model.
Change-Id: I90dd569ce42fe6bf29751ca29086fe458d40a095
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
(cherry picked from commit
25827ae5d147343408da1eb1e74bef3e2c1a8812)
Tibor Frank [Wed, 28 Aug 2019 19:09:21 +0000 (21:09 +0200)]
Report: Add data
Change-Id: I5ae5fa7cb40ddc31e3fe3fd0d6182bf54c84ad00
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
a907de589b3f49368b81394b4faf6f4202c1f404)
Jan Gelety [Mon, 26 Aug 2019 09:03:38 +0000 (11:03 +0200)]
Report: Add data
Change-Id: I66c91e083138b50a575cbbd32d309d0c198e6a71
Signed-off-by: Jan Gelety <jgelety@cisco.com>
(cherry picked from commit
f18bab9daaa9ac4108cc7bc04213db9051c96567)
Maciek Konstantynowicz [Tue, 20 Aug 2019 16:07:21 +0000 (17:07 +0100)]
docs: created full list of tests for report detailed results.
Change-Id: I82d298399b5899736c8e5ebb7204dbbddace79f4
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
(cherry picked from commit
97ff6490bbd4656e15422576556aeba8aa799345)
Maciek Konstantynowicz [Tue, 20 Aug 2019 12:51:51 +0000 (13:51 +0100)]
docs: created select list of tests for report and trending.
Change-Id: I2ad255c35490e52d9a55e24e64539549e6bdd36b
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
(cherry picked from commit
218572c4932c47f08cb0b64b5e493ac939ce73ae)
Ludovit Mikula [Tue, 27 Aug 2019 11:51:18 +0000 (11:51 +0000)]
NEW: Added trex profiles for 60k ip4 destinations.
Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech>
Change-Id: Ib9e42914ad5ab6bf0febea3edbeea8d09c8918b1
(cherry picked from commit
e895f7fad03c2d805c3b16ee490b51cb304f49c0)
Dave Wallace [Tue, 27 Aug 2019 21:02:07 +0000 (17:02 -0400)]
DOC: 1908rls release notes for VPP Hoststack TCP performance tests.
Change-Id: I56d0519b5cbdfb8a60aab85a9a4d721fdfe022ce
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
(cherry picked from commit
f4952736d2f356cd3423c55891e133536ddc9552)
Maciek Konstantynowicz [Sun, 25 Aug 2019 20:11:02 +0000 (21:11 +0100)]
docs: added csit tests job specs.
Change-Id: I271b1c700f72204dcf03bee6f2cd30b25ca5663e
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
(cherry picked from commit
a0465dc55b2f7edfa86e7803704bdb1415fd2b8e)
Tibor Frank [Tue, 27 Aug 2019 09:46:48 +0000 (11:46 +0200)]
Report: Add data
Change-Id: I534cbb4141b60609387f894e51c429e368520a23
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
823e79aca6d19531f3abb080ab6510418f6af94d)
Ludovit Mikula [Mon, 26 Aug 2019 11:39:39 +0000 (11:39 +0000)]
- Removed 9000B test cases from ipsec test suites.
- Added new traffic profiles.
Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech>
Change-Id: Ie06b7a1cf4d9df403d098b7a495219fd95e5869e
(cherry picked from commit
ac37499ef8754d7ac8f6350ab80d389727bbcd03)
Tibor Frank [Mon, 26 Aug 2019 15:13:25 +0000 (17:13 +0200)]
Report: Comp Tables
Change-Id: I2cedef4dc274dc45ebc82911d53f3d5d14ffa381
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
7186f868f6fd24f8727e088301c4411c2a8e8a8c)
Tibor Frank [Mon, 26 Aug 2019 13:46:29 +0000 (15:46 +0200)]
Report: tables
Change-Id: Ideec743ae10909d91625ea5cfc7901624459f455
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
479b30c17e28e30b7c2e01e616e96b67d6ac1779)
Tibor Frank [Mon, 26 Aug 2019 13:29:04 +0000 (15:29 +0200)]
Report: NFV
Change-Id: Idf369766cb9acf82475b1d86860d72077b88dc35
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
320f82cc987dbc6d9c4ab1632de1005ac4c0077f)
Tibor Frank [Mon, 26 Aug 2019 12:32:48 +0000 (14:32 +0200)]
Report: Tables
Change-Id: I5084163758c5f15f482c6c8e08ee70c4a4f50431
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
af4d3ba04cddbfb198af47b2d1d74c0655e35a08)
Tibor Frank [Mon, 26 Aug 2019 11:40:39 +0000 (13:40 +0200)]
Report: Modify comparison tables, add data
Change-Id: I6d5e7cc0618a5b1889ab54c020fd98c661d20989
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
db49a120f90f59f5277a5c812544ea0ad1592eb8)
Jan Gelety [Sun, 25 Aug 2019 12:05:34 +0000 (14:05 +0200)]
FIX: Copy-paste typo in 2n-10ge2p1x710-eth-l2bd-6ch-48vh-24vm1t-vppip4-ndrpdr.robot
Change-Id: Ibe231547e26d9b379554eec8ca6a9b4d7bc1224b
Signed-off-by: Jan Gelety <jgelety@cisco.com>
(cherry picked from commit
d8271912d325ca9d4dc57f0d667c26b51f70e2fd)
Tibor Frank [Mon, 19 Aug 2019 13:25:47 +0000 (15:25 +0200)]
Report: Select tests by name, specifications
Change-Id: Ibcc5e67d49fba9c02b07af462a74d2a062412979
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
33cfe1217cb2d6bc9655b01f200b0ec8d5ccb466)
Jan Gelety [Sun, 25 Aug 2019 08:23:27 +0000 (10:23 +0200)]
Oper: TB31 back in operation
Signed-off-by: Jan Gelety <jgelety@cisco.com>
Change-Id: I8a65efa632bfb095cd9ed8716ea2496288a88d0d
Vratko Polak [Fri, 23 Aug 2019 15:00:29 +0000 (17:00 +0200)]
Introduce reconfig suites, for dot1q+ip4+vxlan
Ticket: CSIT-1551
+ Config copied from other suites.
+ Layer keywords prepared for additional chains.
+ Renamed "create" argument to "vlan_per_chain".
+ TrafficGenerator result extraction improvements.
+ Also unified TG type and subtype checking.
+ Throughput estimated as PDR for zero PLR.
+ Actual Qemu start is skipped.
+ Parse xstats instead of stats in trex stop scipt.
- Because stats are unconditionally cleared on each script connect.
+ Remember start values to get better ReceiveMeasurementResult at stop.
+ Expose loss count and time as appropriate test message.
+ Autogen support.
+ Multiple chain amounts and core densities.
- Only 1 additional chain.
+ Proper TAGs. Documented.
+ Clarified min framesize tags are for TG-DUT.
+ Added a TODO concerning overhead values.
+ Added 118B tag.
+ Robot performance keywords improvements.
+ Added missing return value documentation.
+ Frame size as required test variable, not an explicit argument.
+ MLRsearch keyword to store result (not into test message).
- TODO: De-duplicate: main keyword to measure, minor to set test message.
- But implementation uses the fact default arguments are different.
+ Keywords for starting and stopping background traffic.
- No checks whether start-stop pair is complete.
+ Multiple minor indentation and logging improvements.
- Missing / TODOs:
- Support also vlan_per_chain=True?
- Support dot1q without vxlan?
- Support also deletion?
- How to package config steps so -reconf can be generated from -ndrpdr?
Change-Id: If0270697f9ffa837c34ef802917220a53f156703
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit
ff820e3d53434889aacde2373314c3be8d00f765)
Vratko Polak [Thu, 22 Aug 2019 13:04:34 +0000 (15:04 +0200)]
Improve PLRsearch yet again
Logic improvements for rls1908 and post-pylint fixes:
+ Reduce search time to 30 minutes.
+ Use average instead alternating loads.
+ Rework log/exp avg/stdev from two estimates.
+ Introduce and use pessimistic variance of dual trackers.
+ Introduce safe_exp to use when None does not skip code.
+ Use dot relative imports (instead of disabling pylint).
+ Complete docstrings for simpler functions.
+ Append docstrings to named tuples.
+ Somewhat unify docstrings related to the same arguments.
+ Slightly improve intentation.
+ State named tuples as rtype where used.
+ Add returns and rtype to __repr__ where missing.
+ Return what docstring say (or update them).
+ Explicit copying for vector and matrix values.
Change-Id: I884c68b4839c5df5e8bef95e463666599603a0ff
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit
fbbc47359e3f7b59bbd5a84d85c673374933a50a)
Miroslav Los [Fri, 16 Aug 2019 13:09:39 +0000 (15:09 +0200)]
Framework: Refactor complex functions in PLRSearch
Signed-off-by: Miroslav Los <miroslav.los@pantheon.tech>
Change-Id: Ie2f19a2e3b37e8d85656ab31ece59b89c76bea25
(cherry picked from commit
281b230ba982f9f6ad589fb6e44f121a6a46531f)
Tibor Frank [Tue, 20 Aug 2019 08:52:11 +0000 (10:52 +0200)]
Report: Specify groups of tests for graphs
Groups of tests for graphs:
- Packet Throughput
- Packet Latency
- Speedup Multi-Core
Change-Id: Ib37c1231abb63c234fb1e98d059bb3338133da0c
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit
719a84662ad1482f09224543c07053d27d7b94ea)
Jan Gelety [Wed, 21 Aug 2019 05:54:06 +0000 (07:54 +0200)]
Use vpp 19.08-release builds
- vpp repo updated to release branch
- vpp stable builds updated to 19.08-release
Signed-off-by: Jan Gelety <jgelety@cisco.com>
Change-Id: I47cdb90d2a9714d49b84648b8e3979815e4d4439
Jan Gelety [Wed, 21 Aug 2019 06:26:09 +0000 (08:26 +0200)]
Add: 2-node ip4 oacl tests
Change-Id: I786fe9f75777e1cd0daee70bc350185dd7fd17e1
Signed-off-by: Jan Gelety <jgelety@cisco.com>
(cherry picked from commit
01be3c2278c79d83ae36290868ab66f3741abc38)
Vratko Polak [Fri, 16 Aug 2019 13:48:02 +0000 (15:48 +0200)]
Autogen: Create NICs for TCP suites.
+ Put Regenerator internal functions ourside class.
+ Duplicate into _default_ and _tcp_ functions.
+ Rework Testcase to be a single class with two factories.
+ Improve suite and testcase naming.
Change-Id: I1f835b576c476c1a40c8c52469b1876695f9ae73
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
(cherry picked from commit
ae7dc0011f376861ba9259b7cb5166777520799c)