Changes in CSIT |release|\r
-------------------------\r
\r
-#. Added VPP performance tests\r
+#. **Added VPP performance tests**\r
\r
- - **Container Service Chain Topologies Orchestrated by K8s with VPP Memif**\r
+ - **MRR tests** : New MRR tests measure the packet forwarding rate\r
+ under the maximum load offered by traffic generator over a set\r
+ trial duration, regardless of packet loss. Maximum load for\r
+ specified Ethernet frame size is set to the bi-directional link\r
+ rate. MRR tests are used for continuous performance trending and\r
+ for comparison between releases.\r
\r
- - Added tests with VPP vswitch in container connecting a number of VPP-\r
- in-container service chain topologies with L2 Cross-Connect and L2\r
- Bridge-Domain configurations, orchestrated by Kubernetes. Added\r
- following forwarding topologies: i) "Parallel" with packets flowing from\r
- NIC via VPP to container and back to VPP and NIC; ii) "Chained" (a.k.a.\r
- "Snake") with packets flowing via VPP to container, back to VPP, to next\r
- container, back to VPP and so on until the last container in a chain,\r
- then back to VPP and NIC; iii) "Horizontal" with packets flowing via VPP\r
- to container, then via "horizontal" memif to next container, and so on\r
- until the last container, then back to VPP and NIC;\r
+ - **Service Chaining with SRv6** : SRv6 (Segment Routing IPv6) proxy tests\r
+ verifying performance of Endpoint to SR-unaware appliance via\r
+ masquerading (End.AM), dynamic proxy (End.AD) or static proxy (End.AS)\r
+ functions.\r
\r
- - **VPP TCP/IP stack**\r
+#. **Presentation and Analytics Layer (PAL)**\r
\r
- - Added tests for VPP TCP/IP stack using VPP built-in HTTP server.\r
- WRK traffic generator is used as a client-side;\r
+ - Added continuous performance measuring, trending and anomaly\r
+ detection. Includes new PAL code and Jenkins jobs for Performance\r
+ Trending (PT) and Performance Analysis (PA) producing performance\r
+ trending dashboard and trendline graphs with summary and drill-\r
+ down views across all specified tests that can be reviewed and\r
+ inspected regularly by FD.io developers and users community.\r
\r
- - **SRv6**\r
+#. **Test Framework Optimizations**\r
\r
- - Initial SRv6 (Segment Routing IPv6) tests verifying performance of\r
- IPv6 and SRH (Segment Routing Header) encapsulation, decapsulation,\r
- lookups and rewrites based on configured End and End.DX6 SRv6 egress\r
- functions;\r
+ - **Performance tests efficiency** : Qemu build/install\r
+ optimizations, warmup phase handling, vpp restart handling.\r
+ Resulted in improved stability and reduced total execution time by\r
+ 30% for single pkt size e.g. 64B/78B.\r
\r
- - **IPSecSW**\r
-\r
- - SW computed IPSec encryption with AES-GCM, CBC-SHA1 ciphers, in\r
- combination with IPv4 routed-forwarding;\r
-\r
-#. Presentation and Analytics Layer\r
-\r
- - Added throughput speedup analysis for multi-core and multi-thread\r
- VPP tests into Presentation and Analytics Layer (PAL) for automated\r
- CSIT test results analysis;\r
-\r
-#. Other changes\r
-\r
- - **Framework optimizations**\r
-\r
- - Ability to run CSIT framework on ARM architecture;\r
-\r
- - Overall stability improvements;\r
-\r
- - **NDR and PDR throughput binary search change**\r
-\r
- - Increased binary search resolution by reducing final step from\r
- 100kpps to 50kpps;\r
-\r
- - **VPP plugin loaded as needed by tests**\r
-\r
- - From this release only plugins required by tests are loaded at\r
- VPP initialization time. Previously all plugins were loaded for\r
- all tests;\r
+ - **General code housekeeping** : ongoing RF keywords\r
+ optimizations, removal of redundant RF keywords.\r
\r
Performance Changes\r
-------------------\r
- `pretty ASCII format for 1t1c <../_static/vpp/performance-changes-pdr-1t1c-full.txt>`_,\r
- `pretty ASCII format for 2t2c <../_static/vpp/performance-changes-pdr-2t2c-full.txt>`_.\r
\r
-Measured improvements are in line with VPP code optimizations listed in\r
-`VPP-18.01 release notes\r
-<https://docs.fd.io/vpp/18.01/release_notes_1801.html>`_.\r
-\r
MRR Throughput Changes\r
~~~~~~~~~~~~~~~~~~~~~~\r
\r
Known Issues\r
------------\r
\r
+<to be updated before rls1804 release>\r
+\r
Here is the list of known issues in CSIT |release| for VPP performance tests:\r
\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
| 3 | Lower than expected NDR throughput with | CSIT-571 | Suspected NIC firmware or DPDK driver issue affecting NDR and |\r
| | xl710 and x710 NICs, compared to x520 NICs. | | PDR throughput. Applies to XL710 and X710 NICs. |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 4 | QAT IPSec scale with 1000 tunnels (interfaces) | VPP-1121 | VPP crashes during configuration of 1000 IPsec tunnels. |\r
-| | in 2t2c config, all tests are failing. | | 1t1c tests are not affected |\r
-+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 5 | rls1801 plugin related performance regression | CSIT-925 | With all plugins loaded NDR, PDR and MaxRates vary |\r
+| 4 | rls1801 plugin related performance regression | CSIT-925 | With all plugins loaded NDR, PDR and MaxRates vary |\r
| | | | intermittently from 3% to 5% across multiple test executions. |\r
| | | | Requires plugin code bisecting. |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 6 | rls1801 generic small performance regression | CSIT-926 | Generic performance regression of discovered NDR, PDR and |\r
+| 5 | rls1801 generic small performance regression | CSIT-926 | Generic performance regression of discovered NDR, PDR and |\r
| | ip4base, l2xcbase, l2bdbase | | MaxRates of -3%..-1% vs. rls1710, affects ip4base, l2xcbase, |\r
| | | | l2bdbase test suites. Not detected by CSIT performance trending |\r
| | | | scheme as it was masked out by another issue CSIT-925. |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 7 | rls1801 substantial NDR performance regression | CSIT-927 | Much lower NDR for vhostvr1024 tests, with mean values |\r
-| | for vhost-user vring size of 1024 | | regression of -17%..-42% vs. rls1710, but also very high |\r
-| | | | standard deviation of up to 1.46 Mpps => poor repeatibility. |\r
-| | | | Making mean values not fully representative. |\r
-+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
-| 8 | rls1801 substantial NDR/PDR regression for | CSIT-928 | NDR regression of -7%..-15%, PDR regression of -3%..-15% |\r
+| 6 | rls1801 substantial NDR/PDR regression for | CSIT-928 | NDR regression of -7%..-15%, PDR regression of -3%..-15% |\r
| | IPSec tunnel scale with HW QAT crypto-dev | | compared to rls1710. |\r
+---+-------------------------------------------------+------------+-----------------------------------------------------------------+\r
Code Review / csit.git / blobdiff