From 61ef529d33efd51acd1b1986c325d25a9d7ba705 Mon Sep 17 00:00:00 2001 From: Vratko Polak Date: Mon, 9 Jul 2018 15:14:36 +0200 Subject: [PATCH] Remove THIS tag and use unix endlines The THIS tag has sneaked in 10Ge2P1X520-Eth-L2Xcbase-Eth-1Memif-1Dcr-Mrr but usually it is only to be used in draft testing. Not sure about the way non-unix endlines crept in. Is not git supposed to perform endline conversions on text files? Or are .robot, .rst, .yaml and .vat not recognized as text? Change-Id: Ia78cf942c05c3029c340a8639782786f37e57469 Signed-off-by: Vratko Polak --- .../vpp_unittest_results/index.rst | 14 +- .../vpp_unittest_results/vpp_unittest_results.rst | 2356 ++++++++++---------- .../dpdk_performance_tests/documentation.rst | 10 +- .../dpdk_performance_tests/test_environment.rst | 136 +- .../csit_release_notes.rst | 62 +- .../honeycomb_functional_tests/documentation.rst | 10 +- .../test_environment.rst | 10 +- .../nsh_sfc_functional_tests/documentation.rst | 12 +- docs/report/nsh_sfc_functional_tests/overview.rst | 236 +- .../nsh_sfc_functional_tests/test_environment.rst | 2 +- docs/report/test_operational_data/index.rst | 16 +- docs/report/vpp_functional_tests/documentation.rst | 12 +- docs/report/vpp_functional_tests/overview.rst | 298 +-- .../vpp_functional_tests/test_environment.rst | 1244 +++++------ .../vpp_performance_tests/csit_release_notes.rst | 300 +-- .../vpp_performance_tests/test_environment.rst | 372 ++-- docs/report/vpp_unit_tests/documentation.rst | 12 +- docs/report/vpp_unit_tests/overview.rst | 174 +- resources/templates/vat/memif_create.vat | 2 +- .../vat/nat/nat44_deterministic_forward.vat | 2 +- .../vat/nat/nat44_deterministic_reverse.vat | 2 +- .../templates/vat/nat/nat44_set_deterministic.vat | 2 +- .../templates/vat/nat/nat44_set_interfaces.vat | 2 +- resources/templates/vat/nat/nat_set_workers.vat | 2 +- resources/templates/vat/nat/nat_show_nat.vat | 2 +- .../templates/vat/nat/snat_interface_dump.vat | 2 +- resources/templates/vat/show_memif.vat | 2 +- resources/tools/testbed-setup/ansible/production | 106 +- .../ansible/roles/common/tasks/main.yaml | 92 +- .../ansible/roles/common/tasks/ubuntu_x86_64.yaml | 100 +- .../ansible/roles/sut/tasks/main.yaml | 28 +- .../ansible/roles/sut/tasks/ubuntu_x86_64.yaml | 180 +- .../testbed-setup/ansible/roles/tg/tasks/main.yaml | 12 +- .../ansible/roles/tg/tasks/ubuntu_x86_64.yaml | 18 +- .../ansible/roles/tg_sut/tasks/main.yaml | 202 +- .../ansible/roles/tg_sut/tasks/ubuntu_x86_64.yaml | 364 +-- resources/tools/testbed-setup/ansible/site.yaml | 28 +- resources/tools/testbed-setup/ansible/sut.yaml | 22 +- resources/tools/testbed-setup/ansible/tg.yaml | 22 +- resources/tools/testbed-setup/ansible/virl.yaml | 20 +- .../tools/testbed-setup/ansible/vppdevice.yaml | 20 +- ...e2p1x520-eth-l2xcbase-eth-1memif-1dcr-mrr.robot | 440 ++-- ...0-eth-l2xcbase-eth-1memif-1dcr-ndrpdrdisc.robot | 722 +++--- ...e2p1x710-eth-l2xcbase-eth-1memif-1dcr-mrr.robot | 438 ++-- ...0-eth-l2xcbase-eth-1memif-1dcr-ndrpdrdisc.robot | 720 +++--- 45 files changed, 4414 insertions(+), 4414 deletions(-) diff --git a/docs/report/detailed_test_results/vpp_unittest_results/index.rst b/docs/report/detailed_test_results/vpp_unittest_results/index.rst index f77153d6b9..5659d5e1ab 100644 --- a/docs/report/detailed_test_results/vpp_unittest_results/index.rst +++ b/docs/report/detailed_test_results/vpp_unittest_results/index.rst @@ -1,7 +1,7 @@ -VPP Unit Test Results -===================== - -.. toctree:: - - vpp_unittest_results - +VPP Unit Test Results +===================== + +.. toctree:: + + vpp_unittest_results + diff --git a/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst b/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst index 3e1fd5ac57..ba35f2b831 100644 --- a/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst +++ b/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst @@ -1,1178 +1,1178 @@ -ACL Security Groups -``````````````````` -:: - - ============================================================================== - ACL plugin Test Case - ============================================================================== - ACL plugin version check; learn MACs OK - ACL create/delete test OK - permit ACL apply test OK - deny ACL apply test OK - VPP_624 permit ICMPv4 OK - VPP_624 permit ICMPv6 OK - VPP_624 deny ICMPv4 OK - VPP_624 deny ICMPv6 OK - permit TCPv4 OK - permit TCPv6 OK - permit UDPv4 OK - permit UDPv6 OK - deny TCPv4/v6 OK - deny UDPv4/v6 OK - verify add/dump acls OK - permit single TCPv4 OK - permit single UDPv4 OK - permit single TCPv6 OK - permit single UPPv6 OK - deny single TCPv4/v6 OK - deny single UDPv4/v6 OK - deny single UDPv4/v6, permit ip any, verify non-initial fragment blocked OK - VPP-687 zero length udp ipv4 packet OK - VPP-687 zero length udp ipv6 packet OK - permit TCPv4 + non-match range OK - permit TCPv6 + non-match range OK - permit UDPv4 + non-match range OK - permit UDPv6 + non-match range OK - deny TCPv4/v6 + non-match range OK - deny UDPv4/v6 + non-match range OK - - ============================================================================== - IRB Test Case - ============================================================================== - ACL plugin prepare OK - ACL IPv6 routed -> bridged, L2 ACL deny OK - ACL IPv6 routed -> bridged, L3 ACL deny OK - ACL IPv4 routed -> bridged, L2 ACL deny OK - ACL IPv4 routed -> bridged, L3 ACL deny OK - ACL IPv6 bridged -> routed, L2 ACL deny OK - ACL IPv6 bridged -> routed, L3 ACL deny OK - ACL IPv4 bridged -> routed, L2 ACL deny OK - ACL IPv4 bridged -> routed, L3 ACL deny OK - ACL IPv6 routed -> bridged, L2 ACL permit+reflect OK - ACL IPv6 bridged -> routed, L2 ACL permit+reflect OK - ACL IPv4 routed -> bridged, L2 ACL permit+reflect OK - ACL IPv4 bridged -> routed, L2 ACL permit+reflect OK - ACL IPv6 routed -> bridged, L3 ACL permit+reflect OK - ACL IPv6 bridged -> routed, L3 ACL permit+reflect OK - ACL IPv4 routed -> bridged, L3 ACL permit+reflect OK - ACL IPv4 bridged -> routed, L3 ACL permit+reflect OK - ACL IPv6+EH routed -> bridged, L2 ACL deny OK - ACL IPv6+EH routed -> bridged, L3 ACL deny OK - ACL IPv6+EH bridged -> routed, L2 ACL deny OK - ACL IPv6+EH bridged -> routed, L3 ACL deny OK - ACL IPv6+EH routed -> bridged, L2 ACL permit+reflect OK - ACL IPv6+EH bridged -> routed, L2 ACL permit+reflect OK - ACL IPv6+EH routed -> bridged, L3 ACL permit+reflect OK - ACL IPv6+EH bridged -> routed, L3 ACL permit+reflect OK - ACL IPv4+MF routed -> bridged, L2 ACL deny OK - ACL IPv4+MF routed -> bridged, L3 ACL deny OK - ACL IPv4+MF bridged -> routed, L2 ACL deny OK - ACL IPv4+MF bridged -> routed, L3 ACL deny OK - ACL IPv4+MF routed -> bridged, L2 ACL permit+reflect OK - ACL IPv4+MF bridged -> routed, L2 ACL permit+reflect OK - ACL IPv4+MF routed -> bridged, L3 ACL permit+reflect OK - ACL IPv4+MF bridged -> routed, L3 ACL permit+reflect OK - - ============================================================================== - ACL plugin connection-oriented extended testcases - ============================================================================== - Prepare the settings SKIP - IPv4: Basic conn timeout test reflect on ingress SKIP - IPv4: Basic conn timeout test reflect on egress SKIP - IPv4: reflect egress, clear conn SKIP - IPv4: reflect ingress, clear conn SKIP - IPv4: Idle conn behind active conn, reflect on ingress SKIP - IPv4: Idle conn behind active conn, reflect on egress SKIP - IPv6: Basic conn timeout test reflect on ingress SKIP - IPv6: Basic conn timeout test reflect on egress SKIP - IPv6: reflect egress, clear conn SKIP - IPv6: reflect ingress, clear conn SKIP - IPv6: Idle conn behind active conn, reflect on ingress SKIP - IPv6: Idle conn behind active conn, reflect on egress SKIP - Prepare for TCP session tests SKIP - IPv4: transient TCP session (incomplete 3WHS), ref. on ingress SKIP - IPv4: transient TCP session (incomplete 3WHS), ref. on egress SKIP - IPv4: established TCP session (complete 3WHS), ref. on ingress SKIP - IPv4: established TCP session (complete 3WHS), ref. on egress SKIP - IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP - IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP - IPv6: transient TCP session (incomplete 3WHS), ref. on ingress SKIP - IPv6: transient TCP session (incomplete 3WHS), ref. on egress SKIP - IPv6: established TCP session (complete 3WHS), ref. on ingress SKIP - IPv6: established TCP session (complete 3WHS), ref. on egress SKIP - IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP - IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP - - ============================================================================== - ACL on dot1q bridged subinterfaces Tests - ============================================================================== - IP4 ACL SubIf Dot1Q bridged traffic OK - IP6 ACL SubIf Dot1Q bridged traffic OK - - ============================================================================== - ACL on dot1ad bridged subinterfaces Tests - ============================================================================== - IP4 ACL SubIf Dot1AD bridged traffic OK - IP6 ACL SubIf Dot1AD bridged traffic OK - - ============================================================================== - ACL on dot1ad routed subinterfaces Tests - ============================================================================== - IP4 ACL SubIf Dot1AD routed traffic OK - IP4 ACL SubIf wrong tags Dot1AD routed traffic OK - IP6 ACL SubIf Dot1AD routed traffic OK - IP6 ACL SubIf wrong tags Dot1AD routed traffic OK - - ============================================================================== - ACL on dot1q routed subinterfaces Tests - ============================================================================== - IP4 ACL SubIf Dot1Q routed traffic OK - IP4 ACL SubIf wrong tags Dot1Q routed traffic OK - IP6 ACL SubIf Dot1Q routed traffic OK - IP6 ACL SubIf wrong tags Dot1Q routed traffic OK - -APIs -```` -:: - - ============================================================================== - VAPI test - ============================================================================== - run C VAPI tests SKIP - run C++ VAPI tests SKIP - - ============================================================================== - VPP Object Model Test - ============================================================================== - run C++ VOM tests SKIP - - ============================================================================== - PAPI Test Case - ============================================================================== - show version OK - show version - invalid parameters OK - u8 array OK - - ============================================================================== - PAPI Message parsing Test Case - ============================================================================== - New compound type with array OK - Add new types OK - Add new types 2 OK - Add new message object OK - New message with array OK - Argument name OK - VLA with aribtrary length field placement OK - Message to byte encoding OK - Nested array type OK - Old style VLA array OK - Old VLA compound type OK - Old VLA array arbitrary placement OK - Old VLA u32 OK - Simple array OK - - ============================================================================== - JVPP Core Test Case - ============================================================================== - JVPP Acl Callback Api Test Case OK - JVPP Acl Future Api Test Case OK - JVPP Core Callback Api Test Case OK - JVPP Core Future Api Test Case OK - JVPP Ioamexport Callback Api Test Case OK - JVPP Ioamexport Future Api Test Case OK - JVPP Ioampot Callback Api Test Case OK - JVPP Ioampot Future Api Test Case OK - JVPP Ioamtrace Callback Api Test Case OK - JVPP Ioamtrace Future Api Test Case OK - JVPP Snat Callback Api Test Case OK - JVPP Snat Future Api Test Case OK - -ARP -``` -:: - - ============================================================================== - ARP Test Case - ============================================================================== - ARP OK - ARP Duplicates OK - ARP Static OK - ARP reply with VRRP virtual src hw addr OK - MPLS OK - Proxy ARP OK - Interface Mirror Proxy ARP OK - - ============================================================================== - L2BD arp termination Test Case - ============================================================================== - L2BD arp term - add 5 hosts, verify arp responses OK - L2BD arp term - delete 3 hosts, verify arp responses OK - L2BD arp term - recreate BD1, readd 3 hosts, verify arp responses OK - L2BD arp term - 2 IP4 addrs per host OK - L2BD arp term - create and update 10 IP4-mac pairs OK - L2BD arp/ND term - hosts with both ip4/ip6 OK - L2BD ND term - Add and Del hosts, verify ND replies OK - L2BD ND term - Add and update IP+mac, verify ND replies OK - L2BD arp term - send garps, verify arp event reports OK - L2BD arp term - send duplicate garps, verify suppression OK - L2BD arp term - disable ip4 arp events,send garps, verify no events OK - L2BD ND term - send NS packets verify reports OK - L2BD ND term - send duplicate ns, verify suppression OK - L2BD ND term - disable ip4 arp events,send ns, verify no events OK - -BFD API -```````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) - API - ============================================================================== - activate SHA1 authentication SKIP - create BFD session using non-existent SHA1 (negative case) SKIP - create a BFD session SKIP - create IPv6 BFD session SKIP - create a BFD session (SHA1) SKIP - add SHA1 keys SKIP - change SHA1 key SKIP - deactivate SHA1 authentication SKIP - create the same BFD session twice (negative case) SKIP - create the same BFD session twice (negative case) (SHA1) SKIP - modify BFD session parameters SKIP - share single SHA1 key between multiple BFD sessions SKIP - -BFD Authentication -`````````````````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) (SHA1 auth) - ============================================================================== - hold BFD session up SKIP - hold BFD session up - meticulous auth SKIP - session is not brought down by unauthenticated msg SKIP - session is not brought down by msg with non-existent key-id SKIP - session is not brought down by msg with wrong auth type SKIP - simulate remote peer restart and resynchronization SKIP - session is not kept alive by msgs with bad sequence numbers SKIP - bring BFD session up SKIP - -BFD Authentication Change -````````````````````````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) (changing auth) - ============================================================================== - change auth key without disturbing session state (delayed) SKIP - change auth key without disturbing session state (immediate) SKIP - turn auth off without disturbing session state (delayed) SKIP - turn auth off without disturbing session state (immediate) SKIP - turn auth on without disturbing session state (delayed) SKIP - turn auth on without disturbing session state (immediate) SKIP - -BFD CLI -```````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) (CLI) - ============================================================================== - create/modify/delete IPv4 BFD UDP session SKIP - create/modify/delete IPv6 BFD UDP session SKIP - create/modify/delete IPv6 BFD UDP session (authenticated) SKIP - create/modify/delete IPv4 BFD UDP session (authenticated) SKIP - put session admin-up and admin-down SKIP - turn authentication on and off SKIP - turn authentication on and off (delayed) SKIP - set/delete meticulous SHA1 auth key SKIP - set/delete SHA1 auth key SKIP - set/del udp echo source SKIP - show commands SKIP - -BFD IPv4 -```````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) - ============================================================================== - put session admin-up and admin-down SKIP - configuration change while peer in demand mode SKIP - verify session goes down after inactivity SKIP - echo function SKIP - session goes down if echo function fails SKIP - echo packets looped back SKIP - echo function stops if echo source is removed SKIP - echo function stops if peer sets required min echo rx zero SKIP - hold BFD session up SKIP - immediately honor remote required min rx reduction SKIP - interface with bfd session deleted SKIP - echo packets with invalid checksum don't keep a session up SKIP - large remote required min rx interval SKIP - modify detect multiplier SKIP - modify session - double required min rx SKIP - modify session - halve required min rx SKIP - no periodic frames outside poll sequence if remote demand set SKIP - test correct response to control frame with poll bit set SKIP - test poll sequence queueing SKIP - bring BFD session down SKIP - bring BFD session up SKIP - bring BFD session up - first frame looked up by address pair SKIP - verify slow periodic control frames while session down SKIP - stale echo packets don't keep a session up SKIP - no packets when zero remote required min rx interval SKIP - -BFD IPv6 -```````` -:: - - ============================================================================== - Bidirectional Forwarding Detection (BFD) (IPv6) - ============================================================================== - echo function used SKIP - echo packets looped back SKIP - hold BFD session up SKIP - interface with bfd session deleted SKIP - bring BFD session up SKIP - bring BFD session up - first frame looked up by address pair SKIP - - ============================================================================== - BFD-FIB interactions (IPv6) - ============================================================================== - BFD-FIB interactions SKIP - -BIER - Bit Indexed Explicit Replication -``````````````````````````````````````` -:: - - ============================================================================== - BIER Test Case - ============================================================================== - BIER end-to-end OK - BIER head OK - BIER head over UDP OK - BIER midpoint OK - BIER Tail OK - BIER Tail over UDP OK - - ============================================================================== - BIER FIB Test Case - ============================================================================== - BFIB Unit Tests OK - -Classifier -`````````` -:: - - ============================================================================== - Classifier Test Case - ============================================================================== - IP ACL test OK - MAC ACL test OK - IP PBR test OK - -Container Integration -````````````````````` -:: - - ============================================================================== - Container integration extended testcases - ============================================================================== - IPv4 basic connectivity test SKIP - IPv6 basic connectivity test SKIP - Create loopbacks overlapping with remote addresses SKIP - IPv4 local-spoof connectivity test SKIP - IPv6 local-spoof connectivity test SKIP - Configure container commands SKIP - IPv4 test after configuring container SKIP - IPv6 test after configuring container SKIP - Unconfigure container commands SKIP - IPv4 local-spoof after unconfig test SKIP - IPv6 local-spoof after unconfig test SKIP - -CRUD Loopback -````````````` -:: - - ============================================================================== - CRUD Loopback - ============================================================================== - test_crud (test_interface_crud.TestLoopbackInterfaceCRUD) OK - test_down (test_interface_crud.TestLoopbackInterfaceCRUD) OK - -DHCP -```` -:: - - ============================================================================== - DHCP Test Case - ============================================================================== - DHCPv6 Proxy OK - DHCP Client OK - DHCPv4 Proxy OK - -Distributed Virtual Router -`````````````````````````` -:: - - ============================================================================== - Distributed Virtual Router - ============================================================================== - Distributed Virtual Router OK - L2 Emulation OK - -DS-Lite Softwire -```````````````` -:: - - ============================================================================== - DS-Lite Test Cases - ============================================================================== - Test DS-Lite OK - -FIB -``` -:: - - ============================================================================== - FIB Test Case - ============================================================================== - FIB Unit Tests OK - -Flowprobe -````````` -:: - - ============================================================================== - Re-enable Flowprobe feature - ============================================================================== - disable flowprobe feature after first packets and re-enable SKIP - - ============================================================================== - collect information on Ethernet, IP4 and IP6 datapath (no timers) - ============================================================================== - no timers, one CFLOW packet, 9 Flows inside OK - no timers, two CFLOW packets (mtu=256), 3 Flows in each OK - L2 data on IP4 datapath OK - L2 data on IP6 datapath OK - L2 data on L2 datapath OK - L3 data on IP4 datapath OK - L3 data on IP6 datapath OK - L3 data on L2 datapath OK - L4 data on IP4 datapath OK - L4 data on IP6 datapath OK - L4 data on L2 datapath OK - verify templates on IP6 datapath OK - verify templates on IP4 datapath OK - verify template on L2 datapath OK - - ============================================================================== - Disable Flowprobe feature - ============================================================================== - disable flowprobe feature after first packets SKIP - - ============================================================================== - Re-enable IPFIX - ============================================================================== - disable IPFIX after first packets and re-enable after few packets SKIP - - ============================================================================== - Disable IPFIX - ============================================================================== - disable IPFIX after first packets SKIP - -Geneve Tunnels -`````````````` -:: - - ============================================================================== - GENEVE Test Case - ============================================================================== - Decapsulation test OK - Encapsulation test OK - Multicast flood test OK - Multicast receive test OK - Unicast flood test OK - -GRE Tunnels -``````````` -:: - - ============================================================================== - GRE Test Case - ============================================================================== - GRE IPv4 tunnel Tests OK - GRE IPv6 tunnel Tests OK - GRE tunnel L2 Tests OK - GRE tunnel VRF Tests OK - -GTPU Tunnels -```````````` -:: - - ============================================================================== - GTPU Test Case - ============================================================================== - Decapsulation test OK - Encapsulation test OK - Multicast flood test OK - Multicast receive test OK - Unicast flood test OK - -IP Multicast Routing -```````````````````` -:: - - ============================================================================== - IP Multicast Test Case - ============================================================================== - IP Multicast Bi-directional OK - IPv6 Multicast Replication OK - IPv6 Multicast Replication in non-default table OK - IP Multicast Replication OK - IP Multicast Connected Source check OK - IP Multicast Signal OK - IP Multicast Replication in non-default table OK - -IPSec -````` -:: - - ============================================================================== - Basic test for IPSEC using AH transport and Tunnel mode - ============================================================================== - ipsec ah v4 transport basic test OK - ipsec ah v4 transport burst test OK - ipsec ah 4o4 tunnel basic test OK - ipsec ah 4o4 tunnel burst test OK - - ============================================================================== - Basic test for ipsec esp sanity - tunnel and transport modes. - ============================================================================== - ipsec esp v4 transport basic test OK - ipsec esp v4 transport burst test OK - ipsec esp 4o4 tunnel basic test OK - ipsec esp 4o4 tunnel burst test OK - -IPv4 FIB CRUD -````````````` -:: - - ============================================================================== - FIB - add/update/delete - ip4 routes - ============================================================================== - Add 1k routes OK - Delete 100 routes OK - Add 1k routes OK - Delete 1.5k routes OK - -IPv4 Routing -```````````` -:: - - ============================================================================== - IPv4 Test Case - ============================================================================== - IPv4 FIB test OK - - ============================================================================== - IPv4 routes via NULL - ============================================================================== - IP NULL route OK - - ============================================================================== - IPv4 disabled - ============================================================================== - IP Disabled OK - - ============================================================================== - IPv4 Subnets - ============================================================================== - IP Sub Nets OK - - ============================================================================== - IPv4 VLAN-0 - ============================================================================== - IP VLAN-0 OK - - ============================================================================== - IPv4 Load-Balancing - ============================================================================== - IP Load-Balancing OK - - ============================================================================== - IPv4 Deaggregate Routes - ============================================================================== - IP Deag Routes OK - - ============================================================================== - IPv4 Input Exceptions - ============================================================================== - IP Input Exceptions OK - - ============================================================================== - IPv4 Punt Police/Redirect - ============================================================================== - IP punt police and redirect OK - -IPv4 VRF Multi-instance -``````````````````````` -:: - - ============================================================================== - IP4 VRF Multi-instance Test Case - ============================================================================== - IP4 VRF Multi-instance test 1 - create 5 BDs OK - IP4 VRF Multi-instance test 2 - delete 2 VRFs OK - IP4 VRF Multi-instance 3 - add 2 VRFs OK - IP4 VRF Multi-instance test 4 - delete 4 VRFs OK - -IPv6 Routing -```````````` -:: - - ============================================================================== - IPv6 Test Case - ============================================================================== - IPv6 FIB test OK - IPv6 Neighbour Solicitation Exceptions OK - ND Duplicates OK - IPv6 Router Solicitation Exceptions OK - - ============================================================================== - IPv6 Punt Police/Redirect - ============================================================================== - IP6 punt police and redirect OK - - ============================================================================== - IPv6 disabled - ============================================================================== - IP Disabled OK - - ============================================================================== - IPv6 ND ProxyTest Case - ============================================================================== - IPv6 Proxy ND OK - - ============================================================================== - IPv6 Load-Balancing - ============================================================================== - IPv6 Load-Balancing OK - - ============================================================================== - IPv6 routes via NULL - ============================================================================== - IP NULL route OK - - ============================================================================== - IPv6 Input Exceptions - ============================================================================== - IP6 Input Exceptions OK - -IPv6 VRF Multi-instance -``````````````````````` -:: - - ============================================================================== - IP6 VRF Multi-instance Test Case - ============================================================================== - IP6 VRF Multi-instance test 1 - create 4 VRFs OK - IP6 VRF Multi-instance test 2 - reset 2 VRFs OK - IP6 VRF Multi-instance 3 - add 2 VRFs OK - IP6 VRF Multi-instance test 4 - reset 4 VRFs OK - -IRB Integrated Routing-Bridging -``````````````````````````````` -:: - - ============================================================================== - IRB Test Case - ============================================================================== - IPv4 IRB test 1 OK - IPv4 IRB test 2 OK - -Kube-proxy -`````````` -:: - - ============================================================================== - Kube-proxy Test Case - ============================================================================== - Kube-proxy NAT44 OK - Kube-proxy NAT46 SKIP - Kube-proxy NAT64 SKIP - Kube-proxy NAT66 SKIP - -L2 FIB CRUD -``````````` -:: - - ============================================================================== - L2 FIB Test Case - ============================================================================== - L2 FIB - program 100 + 100 MACs OK - L2 FIB - program 100 + delete 12 MACs OK - L2 FIB - flush all OK - L2 FIB - flush BD OK - L2 FIB - flush interface OK - L2 FIB - mac learning events OK - L2 FIB - mac learning max macs in event OK - L2 FIB - program 100 MACs OK - L2 FIB - Program 10 MACs, learn 10 OK - -L2BD Multi-instance -``````````````````` -:: - - ============================================================================== - L2BD Multi-instance Test Case - ============================================================================== - L2BD Multi-instance test 1 - create 5 BDs OK - L2BD Multi-instance test 2 - update data of 5 BDs OK - L2BD Multi-instance test 3 - delete 2 BDs OK - L2BD Multi-instance test 4 - add 2 BDs OK - L2BD Multi-instance test 5 - delete 5 BDs SKIP - -L2BD Switching -`````````````` -:: - - ============================================================================== - L2BD Test Case - ============================================================================== - L2BD MAC learning dual-loop test OK - L2BD MAC learning single-loop test OK - -L2XC Multi-instance -``````````````````` -:: - - ============================================================================== - L2XC Multi-instance Test Case - ============================================================================== - L2XC Multi-instance test 1 - create 10 cross-connects OK - L2XC Multi-instance test 2 - delete 4 cross-connects OK - L2BD Multi-instance 3 - add new 4 cross-connects OK - L2XC Multi-instance test 4 - delete 10 cross-connects OK - -L2XC Switching -`````````````` -:: - - ============================================================================== - L2XC Test Case - ============================================================================== - L2XC dual-loop test OK - L2XC single-loop test OK - -LISP Tunnels -```````````` -:: - - ============================================================================== - Basic LISP test - ============================================================================== - Test case for basic encapsulation OK - -Load Balancer -````````````` -:: - - ============================================================================== - Load Balancer Test Case - ============================================================================== - Load Balancer IP4 GRE4 OK - Load Balancer IP4 GRE6 OK - Load Balancer IP6 GRE4 OK - Load Balancer IP6 GRE6 OK - -MACIP Access Control -```````````````````` -:: - - ============================================================================== - MACIP Tests - ============================================================================== - MACIP 10 ACLs each with 100+ entries OK - MACIP 10 ACLs each with 100+ entries with IP4 traffic OK - MACIP 10 ACLs each with 100+ entries with IP6 traffic OK - MACIP ACL with 10 entries OK - MACIP ACL with 100 entries OK - MACIP ACL with 2 entries OK - MACIP ACL with 20 entries OK - MACIP ACL with 5 entries OK - MACIP ACL with 50 entries OK - MACIP 2 ACLs each with 100+ entries OK - MACIP replace ACL OK - MACIP ACL delete intf with acl OK - - ============================================================================== - MACIP with IP6 traffic - ============================================================================== - IP6 MACIP exactMAC|exactIP ACL bridged traffic OK - IP6 MACIP exactMAC|subnetIP ACL bridged traffic OK - IP6 MACIP exactMAC|wildIP ACL bridged traffic OK - IP6 MACIP oui_MAC|exactIP ACL bridged traffic OK - IP6 MACIP ouiMAC|subnetIP ACL bridged traffic OK - IP6 MACIP ouiMAC|wildIP ACL bridged traffic OK - IP6 MACIP wildcardMAC|exactIP ACL bridged traffic OK - IP6 MACIP wildcardMAC|subnetIP ACL bridged traffic OK - IP6 MACIP wildcardMAC|wildIP ACL bridged traffic OK - MACIP replace ACL with IP6 traffic OK - IP6 MACIP exactMAC|exactIP ACL routed traffic OK - IP6 MACIP exactMAC|subnetIP ACL routed traffic OK - IP6 MACIP exactMAC|wildIP ACL routed traffic OK - IP6 MACIP ouiMAC|exactIP ACL routed traffic OK - IP6 MACIP ouiMAC|subnetIP ACL routed traffic OK - IP6 MACIP ouiMAC|wildIP ACL routed traffic OK - IP6 MACIP wildcardMAC|exactIP ACL routed traffic OK - IP6 MACIP wildcardMAC|subnetIP ACL routed traffic OK - IP6 MACIP wildcardMAC|wildIP ACL OK - - ============================================================================== - MACIP with IP4 traffic - ============================================================================== - IP4 MACIP wildcardMAC|exactIP ACL bridged traffic OK - IP4 MACIP exactMAC|exactIP ACL bridged traffic OK - IP4 MACIP exactMAC|subnetIP ACL bridged traffic OK - IP4 MACIP exactMAC|wildIP ACL bridged traffic OK - IP4 MACIP ouiMAC|exactIP ACL bridged traffic OK - IP4 MACIP ouiMAC|subnetIP ACL bridged traffic OK - IP4 MACIP ouiMAC|wildIP ACL bridged traffic OK - IP4 MACIP wildcardMAC|subnetIP ACL bridged traffic OK - IP4 MACIP wildcardMAC|wildIP ACL bridged traffic OK - MACIP replace ACL with IP4 traffic OK - IP4 MACIP exactMAC|exactIP ACL routed traffic OK - IP4 MACIP exactMAC|subnetIP ACL routed traffic OK - IP4 MACIP exactMAC|wildIP ACL routed traffic OK - IP4 MACIP ouiMAC|exactIP ACL routed traffic OK - IP4 MACIP ouiMAC|subnetIP ACL routed traffic OK - IP4 MACIP ouiMAC|wildIP ACL routed traffic OK - IP4 MACIP wildcardMAC|exactIP ACL routed traffic OK - IP4 MACIP wildcardMAC|subnetIP ACL routed traffic OK - IP4 MACIP wildcardMAC|wildIP ACL OK - -MAP Softwires -````````````` -:: - - ============================================================================== - MAP Test Case - ============================================================================== - MAP-E OK - -MFIB Multicast FIB -`````````````````` -:: - - ============================================================================== - MFIB Test Case - ============================================================================== - MFIB Unit Tests OK - -MPLS Switching -`````````````` -:: - - ============================================================================== - MPLS-L2 - ============================================================================== - Virtual Private LAN Service OK - Virtual Private Wire Service OK - - ============================================================================== - MPLS Test Case - ============================================================================== - MPLS Local Label Binding test OK - MPLS Deagg OK - MPLS label imposition test OK - MPLS Interface Receive OK - MPLS Multicast Head-end OK - MPLS IPv4 Multicast Tail OK - MPLS IPv6 Multicast Tail OK - MPLS Multicast Mid Point OK - MPLS label swap tests OK - MPLS Tunnel Tests OK - MPLS V4 Explicit NULL test OK - MPLS V6 Explicit NULL test OK - - ============================================================================== - MPLS PIC edge convergence - ============================================================================== - MPLS eBGP PIC edge convergence OK - MPLS iBGP PIC edge convergence OK - MPLSv6 eBGP PIC edge convergence OK - - ============================================================================== - MPLS disabled - ============================================================================== - MPLS Disabled OK - -NAT44 -````` -:: - - ============================================================================== - NAT44 Test Cases - ============================================================================== - Delete NAT44 session OK - NAT44 dynamic translation test OK - NAT44 handling of client packets with TTL=1 OK - NAT44 handling of error responses to client packets with TTL=2 OK - NAT44 handling of server packets with TTL=1 OK - NAT44 handling of error responses to server packets with TTL=2 OK - NAT44 interfaces without configured IP address OK - NAT44 forwarding test OK - NAT44 translate fragments arriving in order OK - NAT44 translate fragments arriving out of order OK - NAT44 hairpinning - 1:1 NAPT OK - NAT44 hairpinning - 1:1 NAT OK - 1:1 NAT translate packet with unknown protocol - hairpinning OK - NAT44 translate packet with unknown protocol - hairpinning OK - Identity NAT OK - NAT44 multiple inside interfaces with overlapping address space OK - Acquire NAT44 addresses from interface OK - Identity NAT with addresses from interface OK - Static mapping with addresses from interface OK - IPFIX logging NAT addresses exhausted OK - IPFIX logging NAT44 session created/delted OK - MAX translations per user - recycle the least recently used OK - NAT44 multiple non-overlapping address space inside interfaces OK - One armed NAT44 OK - NAT44 interface output feature (in2out postrouting) OK - NAT44 interface output feature hairpinning (in2out postrouting) OK - NAT44 interface output feature VRF aware (in2out postrouting) OK - Ping internal host from outside network OK - Ping NAT44 out interface from outside network OK - NAT44 add pool addresses to FIB OK - Port restricted NAT44 (MAP-E CE) OK - NAT44 fragments hairpinning OK - NAT44 set/get virtual fragmentation reassembly OK - 1:1 NAT initialized from inside network OK - NAT44 interfaces without configured IP address - 1:1 NAT OK - NAT44 local service load balancing OK - 1:1 NAT initialized from outside network OK - 1:1 NAT translate packet with unknown protocol OK - 1:1 NAT VRF awareness OK - 1:1 NAPT initialized from inside network OK - NAT44 interfaces without configured IP address - 1:1 NAPT OK - 1:1 NAPT initialized from outside network OK - Twice NAT44 OK - Acquire twice NAT44 addresses from interface OK - Twice NAT44 local service load balancing OK - NAT44 translate packet with unknown protocol OK - NAT44 tenant VRF independent address pool mode OK - NAT44 tenant VRF aware address pool mode OK - - ============================================================================== - Deterministic NAT Test Cases - ============================================================================== - Deterministic NAT translation test (TCP, UDP, ICMP) OK - NAT plugin run deterministic mode OK - Deterministic NAT multiple users OK - Deterministic NAT maximum sessions per user limit SKIP - Deterministic NAT session timeouts SKIP - Set deterministic NAT timeouts OK - Deterministic NAT TCP session close from inside network OK - Deterministic NAT TCP session close from outside network OK - -NAT64 -````` -:: - - ============================================================================== - NAT64 Test Cases - ============================================================================== - NAT64 dynamic translation test OK - NAT64 translate fragments arriving in order OK - NAT64 translate fragments arriving out of order OK - NAT64 hairpinning OK - NAT64 translate packet with unknown protocol - hairpinning OK - NAT64 ICMP Error message translation OK - Enable/disable NAT64 feature on the interface OK - Acquire NAT64 pool addresses from interface OK - One armed NAT64 OK - Add/delete address to NAT64 pool OK - NAT64 Network-Specific Prefix OK - NAT64 fragments hairpinning OK - NAT64 session timeout SKIP - Set NAT64 timeouts OK - NAT64 static translation test OK - Add/delete static BIB entry OK - NAT64 translate packet with unknown protocol OK - -P2P Ethernet Subinterface -````````````````````````` -:: - - ============================================================================== - P2P Ethernet tests - ============================================================================== - delete/create p2p subif OK - create 1k of p2p subifs OK - - ============================================================================== - P2P Ethernet IPv4 tests - ============================================================================== - receive ipv4 packet via p2p subinterface OK - route rx packet not matching p2p subinterface OK - send ip4 packet via p2p subinterface OK - drop tx ip4 packet not matching p2p subinterface OK - - ============================================================================== - P2P Ethernet IPv6 tests - ============================================================================== - receive ipv6 packet via p2p subinterface OK - drop rx packet not matching p2p subinterface OK - route rx ip6 packet not matching p2p subinterface OK - send packet via p2p subinterface OK - drop tx ip6 packet not matching p2p subinterface OK - standard routing without p2p subinterfaces OK - -PPPoE Encapsulation -``````````````````` -:: - - ============================================================================== - PPPoE Test Case - ============================================================================== - PPPoE Add Same Session Twice Test OK - PPPoE Decap Test OK - PPPoE Decap Multiple Sessions Test OK - PPPoE Delete Same Session Twice Test OK - PPPoE Encap Test OK - PPPoE Encap Multiple Sessions Test OK - -SPAN Switch Port Analyzer -````````````````````````` -:: - - ============================================================================== - SPAN Test Case - ============================================================================== - SPAN device rx mirror OK - SPAN l2 broadcast mirror OK - SPAN l2 rx tx mirror OK - SPAN l2 tx mirror OK - SPAN l2 rx mirror OK - SPAN l2 rx mirror into 1ad subif+vtr OK - SPAN l2 rx mirror into 1q subif+vtr OK - SPAN l2 rx mirror into gre-subif+vtr OK - SPAN l2 rx mirror into vxlan OK - -SRv6 Routing -```````````` -:: - - ============================================================================== - SRv6 Test Case - ============================================================================== - Test SRv6 End (without PSP) behavior. OK - Test SRv6 End.DT4 behavior. OK - Test SRv6 End.DT6 behavior. OK - Test SRv6 End.DX2 behavior. OK - Test SRv6 End.DX4 behavior. OK - Test SRv6 End.DX6 behavior. OK - Test SRv6 End.X (without PSP) behavior. OK - Test SRv6 End.X with PSP behavior. OK - Test SRv6 End with PSP behavior. OK - Test SRv6 Transit.Encaps behavior for IPv6. OK - Test SRv6 Transit.Encaps behavior for IPv4. OK - Test SRv6 Transit.Encaps behavior for L2. SKIP - Test SRv6 Transit.Insert behavior (IPv6 only). OK - Test SRv6 Transit.Insert behavior (IPv6 only). OK - -TCP/IP Stack -```````````` -:: - - ============================================================================== - TCP Test Case - ============================================================================== - TCP builtin client/server transfer OK - TCP Unit Tests OK - -UDP Stack -````````` -:: - - ============================================================================== - UDP Encap Test Case - ============================================================================== - UDP Encap test OK - -VTR VLAN Tag Rewrites -````````````````````` -:: - - ============================================================================== - VTR Test Case - ============================================================================== - 1AD VTR pop 1 test OK - 1AD VTR pop 2 test OK - 1AD VTR push 1 1AD test OK - 1AD VTR push 1 1Q test OK - 1AD VTR push 2 1AD test OK - 1AD VTR push 2 1Q test OK - 1AD VTR translate 1 -> 1 1AD test OK - 1AD VTR translate 1 -> 1 1Q test OK - 1AD VTR translate 1 -> 2 1AD test OK - 1AD VTR translate 1 -> 2 1Q test OK - 1AD VTR translate 2 -> 1 1AD test OK - 1AD VTR translate 2 -> 1 1Q test OK - 1AD VTR translate 2 -> 2 1AD test OK - 1AD VTR translate 2 -> 2 1Q test OK - 1Q VTR pop 1 test OK - 1Q VTR push 1 test OK - 1Q VTR push 2 test OK - 1Q VTR translate 1 -> 1 test OK - 1Q VTR translate 1 -> 2 test OK - -VXLAN Tunnels -````````````` -:: - - ============================================================================== - VXLAN Test Case - ============================================================================== - Decapsulation test OK - Encapsulation test OK - Multicast flood test OK - Multicast receive test OK - Unicast flood test OK - -VXLAN-GPE Tunnels -````````````````` -:: - - ============================================================================== - VXLAN-GPE Test Case - ============================================================================== - Decapsulation test SKIP - Encapsulation test SKIP - Multicast flood test SKIP - Multicast receive test SKIP - Unicast flood test SKIP - -Other Tests -``````````` -:: - - ============================================================================== - Ping Test Case - ============================================================================== - basic ping test OK - burst ping test OK - - ============================================================================== - Session Test Case - ============================================================================== - Session Unit Tests OK - - ============================================================================== - Template verification, timer tests - ============================================================================== - timer less than template timeout OK - timer greater than template timeout OK - verify cflow packet fields OK - +ACL Security Groups +``````````````````` +:: + + ============================================================================== + ACL plugin Test Case + ============================================================================== + ACL plugin version check; learn MACs OK + ACL create/delete test OK + permit ACL apply test OK + deny ACL apply test OK + VPP_624 permit ICMPv4 OK + VPP_624 permit ICMPv6 OK + VPP_624 deny ICMPv4 OK + VPP_624 deny ICMPv6 OK + permit TCPv4 OK + permit TCPv6 OK + permit UDPv4 OK + permit UDPv6 OK + deny TCPv4/v6 OK + deny UDPv4/v6 OK + verify add/dump acls OK + permit single TCPv4 OK + permit single UDPv4 OK + permit single TCPv6 OK + permit single UPPv6 OK + deny single TCPv4/v6 OK + deny single UDPv4/v6 OK + deny single UDPv4/v6, permit ip any, verify non-initial fragment blocked OK + VPP-687 zero length udp ipv4 packet OK + VPP-687 zero length udp ipv6 packet OK + permit TCPv4 + non-match range OK + permit TCPv6 + non-match range OK + permit UDPv4 + non-match range OK + permit UDPv6 + non-match range OK + deny TCPv4/v6 + non-match range OK + deny UDPv4/v6 + non-match range OK + + ============================================================================== + IRB Test Case + ============================================================================== + ACL plugin prepare OK + ACL IPv6 routed -> bridged, L2 ACL deny OK + ACL IPv6 routed -> bridged, L3 ACL deny OK + ACL IPv4 routed -> bridged, L2 ACL deny OK + ACL IPv4 routed -> bridged, L3 ACL deny OK + ACL IPv6 bridged -> routed, L2 ACL deny OK + ACL IPv6 bridged -> routed, L3 ACL deny OK + ACL IPv4 bridged -> routed, L2 ACL deny OK + ACL IPv4 bridged -> routed, L3 ACL deny OK + ACL IPv6 routed -> bridged, L2 ACL permit+reflect OK + ACL IPv6 bridged -> routed, L2 ACL permit+reflect OK + ACL IPv4 routed -> bridged, L2 ACL permit+reflect OK + ACL IPv4 bridged -> routed, L2 ACL permit+reflect OK + ACL IPv6 routed -> bridged, L3 ACL permit+reflect OK + ACL IPv6 bridged -> routed, L3 ACL permit+reflect OK + ACL IPv4 routed -> bridged, L3 ACL permit+reflect OK + ACL IPv4 bridged -> routed, L3 ACL permit+reflect OK + ACL IPv6+EH routed -> bridged, L2 ACL deny OK + ACL IPv6+EH routed -> bridged, L3 ACL deny OK + ACL IPv6+EH bridged -> routed, L2 ACL deny OK + ACL IPv6+EH bridged -> routed, L3 ACL deny OK + ACL IPv6+EH routed -> bridged, L2 ACL permit+reflect OK + ACL IPv6+EH bridged -> routed, L2 ACL permit+reflect OK + ACL IPv6+EH routed -> bridged, L3 ACL permit+reflect OK + ACL IPv6+EH bridged -> routed, L3 ACL permit+reflect OK + ACL IPv4+MF routed -> bridged, L2 ACL deny OK + ACL IPv4+MF routed -> bridged, L3 ACL deny OK + ACL IPv4+MF bridged -> routed, L2 ACL deny OK + ACL IPv4+MF bridged -> routed, L3 ACL deny OK + ACL IPv4+MF routed -> bridged, L2 ACL permit+reflect OK + ACL IPv4+MF bridged -> routed, L2 ACL permit+reflect OK + ACL IPv4+MF routed -> bridged, L3 ACL permit+reflect OK + ACL IPv4+MF bridged -> routed, L3 ACL permit+reflect OK + + ============================================================================== + ACL plugin connection-oriented extended testcases + ============================================================================== + Prepare the settings SKIP + IPv4: Basic conn timeout test reflect on ingress SKIP + IPv4: Basic conn timeout test reflect on egress SKIP + IPv4: reflect egress, clear conn SKIP + IPv4: reflect ingress, clear conn SKIP + IPv4: Idle conn behind active conn, reflect on ingress SKIP + IPv4: Idle conn behind active conn, reflect on egress SKIP + IPv6: Basic conn timeout test reflect on ingress SKIP + IPv6: Basic conn timeout test reflect on egress SKIP + IPv6: reflect egress, clear conn SKIP + IPv6: reflect ingress, clear conn SKIP + IPv6: Idle conn behind active conn, reflect on ingress SKIP + IPv6: Idle conn behind active conn, reflect on egress SKIP + Prepare for TCP session tests SKIP + IPv4: transient TCP session (incomplete 3WHS), ref. on ingress SKIP + IPv4: transient TCP session (incomplete 3WHS), ref. on egress SKIP + IPv4: established TCP session (complete 3WHS), ref. on ingress SKIP + IPv4: established TCP session (complete 3WHS), ref. on egress SKIP + IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP + IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP + IPv6: transient TCP session (incomplete 3WHS), ref. on ingress SKIP + IPv6: transient TCP session (incomplete 3WHS), ref. on egress SKIP + IPv6: established TCP session (complete 3WHS), ref. on ingress SKIP + IPv6: established TCP session (complete 3WHS), ref. on egress SKIP + IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP + IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP + + ============================================================================== + ACL on dot1q bridged subinterfaces Tests + ============================================================================== + IP4 ACL SubIf Dot1Q bridged traffic OK + IP6 ACL SubIf Dot1Q bridged traffic OK + + ============================================================================== + ACL on dot1ad bridged subinterfaces Tests + ============================================================================== + IP4 ACL SubIf Dot1AD bridged traffic OK + IP6 ACL SubIf Dot1AD bridged traffic OK + + ============================================================================== + ACL on dot1ad routed subinterfaces Tests + ============================================================================== + IP4 ACL SubIf Dot1AD routed traffic OK + IP4 ACL SubIf wrong tags Dot1AD routed traffic OK + IP6 ACL SubIf Dot1AD routed traffic OK + IP6 ACL SubIf wrong tags Dot1AD routed traffic OK + + ============================================================================== + ACL on dot1q routed subinterfaces Tests + ============================================================================== + IP4 ACL SubIf Dot1Q routed traffic OK + IP4 ACL SubIf wrong tags Dot1Q routed traffic OK + IP6 ACL SubIf Dot1Q routed traffic OK + IP6 ACL SubIf wrong tags Dot1Q routed traffic OK + +APIs +```` +:: + + ============================================================================== + VAPI test + ============================================================================== + run C VAPI tests SKIP + run C++ VAPI tests SKIP + + ============================================================================== + VPP Object Model Test + ============================================================================== + run C++ VOM tests SKIP + + ============================================================================== + PAPI Test Case + ============================================================================== + show version OK + show version - invalid parameters OK + u8 array OK + + ============================================================================== + PAPI Message parsing Test Case + ============================================================================== + New compound type with array OK + Add new types OK + Add new types 2 OK + Add new message object OK + New message with array OK + Argument name OK + VLA with aribtrary length field placement OK + Message to byte encoding OK + Nested array type OK + Old style VLA array OK + Old VLA compound type OK + Old VLA array arbitrary placement OK + Old VLA u32 OK + Simple array OK + + ============================================================================== + JVPP Core Test Case + ============================================================================== + JVPP Acl Callback Api Test Case OK + JVPP Acl Future Api Test Case OK + JVPP Core Callback Api Test Case OK + JVPP Core Future Api Test Case OK + JVPP Ioamexport Callback Api Test Case OK + JVPP Ioamexport Future Api Test Case OK + JVPP Ioampot Callback Api Test Case OK + JVPP Ioampot Future Api Test Case OK + JVPP Ioamtrace Callback Api Test Case OK + JVPP Ioamtrace Future Api Test Case OK + JVPP Snat Callback Api Test Case OK + JVPP Snat Future Api Test Case OK + +ARP +``` +:: + + ============================================================================== + ARP Test Case + ============================================================================== + ARP OK + ARP Duplicates OK + ARP Static OK + ARP reply with VRRP virtual src hw addr OK + MPLS OK + Proxy ARP OK + Interface Mirror Proxy ARP OK + + ============================================================================== + L2BD arp termination Test Case + ============================================================================== + L2BD arp term - add 5 hosts, verify arp responses OK + L2BD arp term - delete 3 hosts, verify arp responses OK + L2BD arp term - recreate BD1, readd 3 hosts, verify arp responses OK + L2BD arp term - 2 IP4 addrs per host OK + L2BD arp term - create and update 10 IP4-mac pairs OK + L2BD arp/ND term - hosts with both ip4/ip6 OK + L2BD ND term - Add and Del hosts, verify ND replies OK + L2BD ND term - Add and update IP+mac, verify ND replies OK + L2BD arp term - send garps, verify arp event reports OK + L2BD arp term - send duplicate garps, verify suppression OK + L2BD arp term - disable ip4 arp events,send garps, verify no events OK + L2BD ND term - send NS packets verify reports OK + L2BD ND term - send duplicate ns, verify suppression OK + L2BD ND term - disable ip4 arp events,send ns, verify no events OK + +BFD API +```````` +:: + + ============================================================================== + Bidirectional Forwarding Detection (BFD) - API + ============================================================================== + activate SHA1 authentication SKIP + create BFD session using non-existent SHA1 (negative case) SKIP + create a BFD session SKIP + create IPv6 BFD session SKIP + create a BFD session (SHA1) SKIP + add SHA1 keys SKIP + change SHA1 key SKIP + deactivate SHA1 authentication SKIP + create the same BFD session twice (negative case) SKIP + create the same BFD session twice (negative case) (SHA1) SKIP + modify BFD session parameters SKIP + share single SHA1 key between multiple BFD sessions SKIP + +BFD Authentication +`````````````````` +:: + + ============================================================================== + Bidirectional Forwarding Detection (BFD) (SHA1 auth) + ============================================================================== + hold BFD session up SKIP + hold BFD session up - meticulous auth SKIP + session is not brought down by unauthenticated msg SKIP + session is not brought down by msg with non-existent key-id SKIP + session is not brought down by msg with wrong auth type SKIP + simulate remote peer restart and resynchronization SKIP + session is not kept alive by msgs with bad sequence numbers SKIP + bring BFD session up SKIP + +BFD Authentication Change +````````````````````````` +:: + + ============================================================================== + Bidirectional Forwarding Detection (BFD) (changing auth) + ============================================================================== + change auth key without disturbing session state (delayed) SKIP + change auth key without disturbing session state (immediate) SKIP + turn auth off without disturbing session state (delayed) SKIP + turn auth off without disturbing session state (immediate) SKIP + turn auth on without disturbing session state (delayed) SKIP + turn auth on without disturbing session state (immediate) SKIP + +BFD CLI +```````` +:: + + ============================================================================== + Bidirectional Forwarding Detection (BFD) (CLI) + ============================================================================== + create/modify/delete IPv4 BFD UDP session SKIP + create/modify/delete IPv6 BFD UDP session SKIP + create/modify/delete IPv6 BFD UDP session (authenticated) SKIP + create/modify/delete IPv4 BFD UDP session (authenticated) SKIP + put session admin-up and admin-down SKIP + turn authentication on and off SKIP + turn authentication on and off (delayed) SKIP + set/delete meticulous SHA1 auth key SKIP + set/delete SHA1 auth key SKIP + set/del udp echo source SKIP + show commands SKIP + +BFD IPv4 +```````` +:: + + ============================================================================== + Bidirectional Forwarding Detection (BFD) + ============================================================================== + put session admin-up and admin-down SKIP + configuration change while peer in demand mode SKIP + verify session goes down after inactivity SKIP + echo function SKIP + session goes down if echo function fails SKIP + echo packets looped back SKIP + echo function stops if echo source is removed SKIP + echo function stops if peer sets required min echo rx zero SKIP + hold BFD session up SKIP + immediately honor remote required min rx reduction SKIP + interface with bfd session deleted SKIP + echo packets with invalid checksum don't keep a session up SKIP + large remote required min rx interval SKIP + modify detect multiplier SKIP + modify session - double required min rx SKIP + modify session - halve required min rx SKIP + no periodic frames outside poll sequence if remote demand set SKIP + test correct response to control frame with poll bit set SKIP + test poll sequence queueing SKIP + bring BFD session down SKIP + bring BFD session up SKIP + bring BFD session up - first frame looked up by address pair SKIP + verify slow periodic control frames while session down SKIP + stale echo packets don't keep a session up SKIP + no packets when zero remote required min rx interval SKIP + +BFD IPv6 +```````` +:: + + ============================================================================== + Bidirectional Forwarding Detection (BFD) (IPv6) + ============================================================================== + echo function used SKIP + echo packets looped back SKIP + hold BFD session up SKIP + interface with bfd session deleted SKIP + bring BFD session up SKIP + bring BFD session up - first frame looked up by address pair SKIP + + ============================================================================== + BFD-FIB interactions (IPv6) + ============================================================================== + BFD-FIB interactions SKIP + +BIER - Bit Indexed Explicit Replication +``````````````````````````````````````` +:: + + ============================================================================== + BIER Test Case + ============================================================================== + BIER end-to-end OK + BIER head OK + BIER head over UDP OK + BIER midpoint OK + BIER Tail OK + BIER Tail over UDP OK + + ============================================================================== + BIER FIB Test Case + ============================================================================== + BFIB Unit Tests OK + +Classifier +`````````` +:: + + ============================================================================== + Classifier Test Case + ============================================================================== + IP ACL test OK + MAC ACL test OK + IP PBR test OK + +Container Integration +````````````````````` +:: + + ============================================================================== + Container integration extended testcases + ============================================================================== + IPv4 basic connectivity test SKIP + IPv6 basic connectivity test SKIP + Create loopbacks overlapping with remote addresses SKIP + IPv4 local-spoof connectivity test SKIP + IPv6 local-spoof connectivity test SKIP + Configure container commands SKIP + IPv4 test after configuring container SKIP + IPv6 test after configuring container SKIP + Unconfigure container commands SKIP + IPv4 local-spoof after unconfig test SKIP + IPv6 local-spoof after unconfig test SKIP + +CRUD Loopback +````````````` +:: + + ============================================================================== + CRUD Loopback + ============================================================================== + test_crud (test_interface_crud.TestLoopbackInterfaceCRUD) OK + test_down (test_interface_crud.TestLoopbackInterfaceCRUD) OK + +DHCP +```` +:: + + ============================================================================== + DHCP Test Case + ============================================================================== + DHCPv6 Proxy OK + DHCP Client OK + DHCPv4 Proxy OK + +Distributed Virtual Router +`````````````````````````` +:: + + ============================================================================== + Distributed Virtual Router + ============================================================================== + Distributed Virtual Router OK + L2 Emulation OK + +DS-Lite Softwire +```````````````` +:: + + ============================================================================== + DS-Lite Test Cases + ============================================================================== + Test DS-Lite OK + +FIB +``` +:: + + ============================================================================== + FIB Test Case + ============================================================================== + FIB Unit Tests OK + +Flowprobe +````````` +:: + + ============================================================================== + Re-enable Flowprobe feature + ============================================================================== + disable flowprobe feature after first packets and re-enable SKIP + + ============================================================================== + collect information on Ethernet, IP4 and IP6 datapath (no timers) + ============================================================================== + no timers, one CFLOW packet, 9 Flows inside OK + no timers, two CFLOW packets (mtu=256), 3 Flows in each OK + L2 data on IP4 datapath OK + L2 data on IP6 datapath OK + L2 data on L2 datapath OK + L3 data on IP4 datapath OK + L3 data on IP6 datapath OK + L3 data on L2 datapath OK + L4 data on IP4 datapath OK + L4 data on IP6 datapath OK + L4 data on L2 datapath OK + verify templates on IP6 datapath OK + verify templates on IP4 datapath OK + verify template on L2 datapath OK + + ============================================================================== + Disable Flowprobe feature + ============================================================================== + disable flowprobe feature after first packets SKIP + + ============================================================================== + Re-enable IPFIX + ============================================================================== + disable IPFIX after first packets and re-enable after few packets SKIP + + ============================================================================== + Disable IPFIX + ============================================================================== + disable IPFIX after first packets SKIP + +Geneve Tunnels +`````````````` +:: + + ============================================================================== + GENEVE Test Case + ============================================================================== + Decapsulation test OK + Encapsulation test OK + Multicast flood test OK + Multicast receive test OK + Unicast flood test OK + +GRE Tunnels +``````````` +:: + + ============================================================================== + GRE Test Case + ============================================================================== + GRE IPv4 tunnel Tests OK + GRE IPv6 tunnel Tests OK + GRE tunnel L2 Tests OK + GRE tunnel VRF Tests OK + +GTPU Tunnels +```````````` +:: + + ============================================================================== + GTPU Test Case + ============================================================================== + Decapsulation test OK + Encapsulation test OK + Multicast flood test OK + Multicast receive test OK + Unicast flood test OK + +IP Multicast Routing +```````````````````` +:: + + ============================================================================== + IP Multicast Test Case + ============================================================================== + IP Multicast Bi-directional OK + IPv6 Multicast Replication OK + IPv6 Multicast Replication in non-default table OK + IP Multicast Replication OK + IP Multicast Connected Source check OK + IP Multicast Signal OK + IP Multicast Replication in non-default table OK + +IPSec +````` +:: + + ============================================================================== + Basic test for IPSEC using AH transport and Tunnel mode + ============================================================================== + ipsec ah v4 transport basic test OK + ipsec ah v4 transport burst test OK + ipsec ah 4o4 tunnel basic test OK + ipsec ah 4o4 tunnel burst test OK + + ============================================================================== + Basic test for ipsec esp sanity - tunnel and transport modes. + ============================================================================== + ipsec esp v4 transport basic test OK + ipsec esp v4 transport burst test OK + ipsec esp 4o4 tunnel basic test OK + ipsec esp 4o4 tunnel burst test OK + +IPv4 FIB CRUD +````````````` +:: + + ============================================================================== + FIB - add/update/delete - ip4 routes + ============================================================================== + Add 1k routes OK + Delete 100 routes OK + Add 1k routes OK + Delete 1.5k routes OK + +IPv4 Routing +```````````` +:: + + ============================================================================== + IPv4 Test Case + ============================================================================== + IPv4 FIB test OK + + ============================================================================== + IPv4 routes via NULL + ============================================================================== + IP NULL route OK + + ============================================================================== + IPv4 disabled + ============================================================================== + IP Disabled OK + + ============================================================================== + IPv4 Subnets + ============================================================================== + IP Sub Nets OK + + ============================================================================== + IPv4 VLAN-0 + ============================================================================== + IP VLAN-0 OK + + ============================================================================== + IPv4 Load-Balancing + ============================================================================== + IP Load-Balancing OK + + ============================================================================== + IPv4 Deaggregate Routes + ============================================================================== + IP Deag Routes OK + + ============================================================================== + IPv4 Input Exceptions + ============================================================================== + IP Input Exceptions OK + + ============================================================================== + IPv4 Punt Police/Redirect + ============================================================================== + IP punt police and redirect OK + +IPv4 VRF Multi-instance +``````````````````````` +:: + + ============================================================================== + IP4 VRF Multi-instance Test Case + ============================================================================== + IP4 VRF Multi-instance test 1 - create 5 BDs OK + IP4 VRF Multi-instance test 2 - delete 2 VRFs OK + IP4 VRF Multi-instance 3 - add 2 VRFs OK + IP4 VRF Multi-instance test 4 - delete 4 VRFs OK + +IPv6 Routing +```````````` +:: + + ============================================================================== + IPv6 Test Case + ============================================================================== + IPv6 FIB test OK + IPv6 Neighbour Solicitation Exceptions OK + ND Duplicates OK + IPv6 Router Solicitation Exceptions OK + + ============================================================================== + IPv6 Punt Police/Redirect + ============================================================================== + IP6 punt police and redirect OK + + ============================================================================== + IPv6 disabled + ============================================================================== + IP Disabled OK + + ============================================================================== + IPv6 ND ProxyTest Case + ============================================================================== + IPv6 Proxy ND OK + + ============================================================================== + IPv6 Load-Balancing + ============================================================================== + IPv6 Load-Balancing OK + + ============================================================================== + IPv6 routes via NULL + ============================================================================== + IP NULL route OK + + ============================================================================== + IPv6 Input Exceptions + ============================================================================== + IP6 Input Exceptions OK + +IPv6 VRF Multi-instance +``````````````````````` +:: + + ============================================================================== + IP6 VRF Multi-instance Test Case + ============================================================================== + IP6 VRF Multi-instance test 1 - create 4 VRFs OK + IP6 VRF Multi-instance test 2 - reset 2 VRFs OK + IP6 VRF Multi-instance 3 - add 2 VRFs OK + IP6 VRF Multi-instance test 4 - reset 4 VRFs OK + +IRB Integrated Routing-Bridging +``````````````````````````````` +:: + + ============================================================================== + IRB Test Case + ============================================================================== + IPv4 IRB test 1 OK + IPv4 IRB test 2 OK + +Kube-proxy +`````````` +:: + + ============================================================================== + Kube-proxy Test Case + ============================================================================== + Kube-proxy NAT44 OK + Kube-proxy NAT46 SKIP + Kube-proxy NAT64 SKIP + Kube-proxy NAT66 SKIP + +L2 FIB CRUD +``````````` +:: + + ============================================================================== + L2 FIB Test Case + ============================================================================== + L2 FIB - program 100 + 100 MACs OK + L2 FIB - program 100 + delete 12 MACs OK + L2 FIB - flush all OK + L2 FIB - flush BD OK + L2 FIB - flush interface OK + L2 FIB - mac learning events OK + L2 FIB - mac learning max macs in event OK + L2 FIB - program 100 MACs OK + L2 FIB - Program 10 MACs, learn 10 OK + +L2BD Multi-instance +``````````````````` +:: + + ============================================================================== + L2BD Multi-instance Test Case + ============================================================================== + L2BD Multi-instance test 1 - create 5 BDs OK + L2BD Multi-instance test 2 - update data of 5 BDs OK + L2BD Multi-instance test 3 - delete 2 BDs OK + L2BD Multi-instance test 4 - add 2 BDs OK + L2BD Multi-instance test 5 - delete 5 BDs SKIP + +L2BD Switching +`````````````` +:: + + ============================================================================== + L2BD Test Case + ============================================================================== + L2BD MAC learning dual-loop test OK + L2BD MAC learning single-loop test OK + +L2XC Multi-instance +``````````````````` +:: + + ============================================================================== + L2XC Multi-instance Test Case + ============================================================================== + L2XC Multi-instance test 1 - create 10 cross-connects OK + L2XC Multi-instance test 2 - delete 4 cross-connects OK + L2BD Multi-instance 3 - add new 4 cross-connects OK + L2XC Multi-instance test 4 - delete 10 cross-connects OK + +L2XC Switching +`````````````` +:: + + ============================================================================== + L2XC Test Case + ============================================================================== + L2XC dual-loop test OK + L2XC single-loop test OK + +LISP Tunnels +```````````` +:: + + ============================================================================== + Basic LISP test + ============================================================================== + Test case for basic encapsulation OK + +Load Balancer +````````````` +:: + + ============================================================================== + Load Balancer Test Case + ============================================================================== + Load Balancer IP4 GRE4 OK + Load Balancer IP4 GRE6 OK + Load Balancer IP6 GRE4 OK + Load Balancer IP6 GRE6 OK + +MACIP Access Control +```````````````````` +:: + + ============================================================================== + MACIP Tests + ============================================================================== + MACIP 10 ACLs each with 100+ entries OK + MACIP 10 ACLs each with 100+ entries with IP4 traffic OK + MACIP 10 ACLs each with 100+ entries with IP6 traffic OK + MACIP ACL with 10 entries OK + MACIP ACL with 100 entries OK + MACIP ACL with 2 entries OK + MACIP ACL with 20 entries OK + MACIP ACL with 5 entries OK + MACIP ACL with 50 entries OK + MACIP 2 ACLs each with 100+ entries OK + MACIP replace ACL OK + MACIP ACL delete intf with acl OK + + ============================================================================== + MACIP with IP6 traffic + ============================================================================== + IP6 MACIP exactMAC|exactIP ACL bridged traffic OK + IP6 MACIP exactMAC|subnetIP ACL bridged traffic OK + IP6 MACIP exactMAC|wildIP ACL bridged traffic OK + IP6 MACIP oui_MAC|exactIP ACL bridged traffic OK + IP6 MACIP ouiMAC|subnetIP ACL bridged traffic OK + IP6 MACIP ouiMAC|wildIP ACL bridged traffic OK + IP6 MACIP wildcardMAC|exactIP ACL bridged traffic OK + IP6 MACIP wildcardMAC|subnetIP ACL bridged traffic OK + IP6 MACIP wildcardMAC|wildIP ACL bridged traffic OK + MACIP replace ACL with IP6 traffic OK + IP6 MACIP exactMAC|exactIP ACL routed traffic OK + IP6 MACIP exactMAC|subnetIP ACL routed traffic OK + IP6 MACIP exactMAC|wildIP ACL routed traffic OK + IP6 MACIP ouiMAC|exactIP ACL routed traffic OK + IP6 MACIP ouiMAC|subnetIP ACL routed traffic OK + IP6 MACIP ouiMAC|wildIP ACL routed traffic OK + IP6 MACIP wildcardMAC|exactIP ACL routed traffic OK + IP6 MACIP wildcardMAC|subnetIP ACL routed traffic OK + IP6 MACIP wildcardMAC|wildIP ACL OK + + ============================================================================== + MACIP with IP4 traffic + ============================================================================== + IP4 MACIP wildcardMAC|exactIP ACL bridged traffic OK + IP4 MACIP exactMAC|exactIP ACL bridged traffic OK + IP4 MACIP exactMAC|subnetIP ACL bridged traffic OK + IP4 MACIP exactMAC|wildIP ACL bridged traffic OK + IP4 MACIP ouiMAC|exactIP ACL bridged traffic OK + IP4 MACIP ouiMAC|subnetIP ACL bridged traffic OK + IP4 MACIP ouiMAC|wildIP ACL bridged traffic OK + IP4 MACIP wildcardMAC|subnetIP ACL bridged traffic OK + IP4 MACIP wildcardMAC|wildIP ACL bridged traffic OK + MACIP replace ACL with IP4 traffic OK + IP4 MACIP exactMAC|exactIP ACL routed traffic OK + IP4 MACIP exactMAC|subnetIP ACL routed traffic OK + IP4 MACIP exactMAC|wildIP ACL routed traffic OK + IP4 MACIP ouiMAC|exactIP ACL routed traffic OK + IP4 MACIP ouiMAC|subnetIP ACL routed traffic OK + IP4 MACIP ouiMAC|wildIP ACL routed traffic OK + IP4 MACIP wildcardMAC|exactIP ACL routed traffic OK + IP4 MACIP wildcardMAC|subnetIP ACL routed traffic OK + IP4 MACIP wildcardMAC|wildIP ACL OK + +MAP Softwires +````````````` +:: + + ============================================================================== + MAP Test Case + ============================================================================== + MAP-E OK + +MFIB Multicast FIB +`````````````````` +:: + + ============================================================================== + MFIB Test Case + ============================================================================== + MFIB Unit Tests OK + +MPLS Switching +`````````````` +:: + + ============================================================================== + MPLS-L2 + ============================================================================== + Virtual Private LAN Service OK + Virtual Private Wire Service OK + + ============================================================================== + MPLS Test Case + ============================================================================== + MPLS Local Label Binding test OK + MPLS Deagg OK + MPLS label imposition test OK + MPLS Interface Receive OK + MPLS Multicast Head-end OK + MPLS IPv4 Multicast Tail OK + MPLS IPv6 Multicast Tail OK + MPLS Multicast Mid Point OK + MPLS label swap tests OK + MPLS Tunnel Tests OK + MPLS V4 Explicit NULL test OK + MPLS V6 Explicit NULL test OK + + ============================================================================== + MPLS PIC edge convergence + ============================================================================== + MPLS eBGP PIC edge convergence OK + MPLS iBGP PIC edge convergence OK + MPLSv6 eBGP PIC edge convergence OK + + ============================================================================== + MPLS disabled + ============================================================================== + MPLS Disabled OK + +NAT44 +````` +:: + + ============================================================================== + NAT44 Test Cases + ============================================================================== + Delete NAT44 session OK + NAT44 dynamic translation test OK + NAT44 handling of client packets with TTL=1 OK + NAT44 handling of error responses to client packets with TTL=2 OK + NAT44 handling of server packets with TTL=1 OK + NAT44 handling of error responses to server packets with TTL=2 OK + NAT44 interfaces without configured IP address OK + NAT44 forwarding test OK + NAT44 translate fragments arriving in order OK + NAT44 translate fragments arriving out of order OK + NAT44 hairpinning - 1:1 NAPT OK + NAT44 hairpinning - 1:1 NAT OK + 1:1 NAT translate packet with unknown protocol - hairpinning OK + NAT44 translate packet with unknown protocol - hairpinning OK + Identity NAT OK + NAT44 multiple inside interfaces with overlapping address space OK + Acquire NAT44 addresses from interface OK + Identity NAT with addresses from interface OK + Static mapping with addresses from interface OK + IPFIX logging NAT addresses exhausted OK + IPFIX logging NAT44 session created/delted OK + MAX translations per user - recycle the least recently used OK + NAT44 multiple non-overlapping address space inside interfaces OK + One armed NAT44 OK + NAT44 interface output feature (in2out postrouting) OK + NAT44 interface output feature hairpinning (in2out postrouting) OK + NAT44 interface output feature VRF aware (in2out postrouting) OK + Ping internal host from outside network OK + Ping NAT44 out interface from outside network OK + NAT44 add pool addresses to FIB OK + Port restricted NAT44 (MAP-E CE) OK + NAT44 fragments hairpinning OK + NAT44 set/get virtual fragmentation reassembly OK + 1:1 NAT initialized from inside network OK + NAT44 interfaces without configured IP address - 1:1 NAT OK + NAT44 local service load balancing OK + 1:1 NAT initialized from outside network OK + 1:1 NAT translate packet with unknown protocol OK + 1:1 NAT VRF awareness OK + 1:1 NAPT initialized from inside network OK + NAT44 interfaces without configured IP address - 1:1 NAPT OK + 1:1 NAPT initialized from outside network OK + Twice NAT44 OK + Acquire twice NAT44 addresses from interface OK + Twice NAT44 local service load balancing OK + NAT44 translate packet with unknown protocol OK + NAT44 tenant VRF independent address pool mode OK + NAT44 tenant VRF aware address pool mode OK + + ============================================================================== + Deterministic NAT Test Cases + ============================================================================== + Deterministic NAT translation test (TCP, UDP, ICMP) OK + NAT plugin run deterministic mode OK + Deterministic NAT multiple users OK + Deterministic NAT maximum sessions per user limit SKIP + Deterministic NAT session timeouts SKIP + Set deterministic NAT timeouts OK + Deterministic NAT TCP session close from inside network OK + Deterministic NAT TCP session close from outside network OK + +NAT64 +````` +:: + + ============================================================================== + NAT64 Test Cases + ============================================================================== + NAT64 dynamic translation test OK + NAT64 translate fragments arriving in order OK + NAT64 translate fragments arriving out of order OK + NAT64 hairpinning OK + NAT64 translate packet with unknown protocol - hairpinning OK + NAT64 ICMP Error message translation OK + Enable/disable NAT64 feature on the interface OK + Acquire NAT64 pool addresses from interface OK + One armed NAT64 OK + Add/delete address to NAT64 pool OK + NAT64 Network-Specific Prefix OK + NAT64 fragments hairpinning OK + NAT64 session timeout SKIP + Set NAT64 timeouts OK + NAT64 static translation test OK + Add/delete static BIB entry OK + NAT64 translate packet with unknown protocol OK + +P2P Ethernet Subinterface +````````````````````````` +:: + + ============================================================================== + P2P Ethernet tests + ============================================================================== + delete/create p2p subif OK + create 1k of p2p subifs OK + + ============================================================================== + P2P Ethernet IPv4 tests + ============================================================================== + receive ipv4 packet via p2p subinterface OK + route rx packet not matching p2p subinterface OK + send ip4 packet via p2p subinterface OK + drop tx ip4 packet not matching p2p subinterface OK + + ============================================================================== + P2P Ethernet IPv6 tests + ============================================================================== + receive ipv6 packet via p2p subinterface OK + drop rx packet not matching p2p subinterface OK + route rx ip6 packet not matching p2p subinterface OK + send packet via p2p subinterface OK + drop tx ip6 packet not matching p2p subinterface OK + standard routing without p2p subinterfaces OK + +PPPoE Encapsulation +``````````````````` +:: + + ============================================================================== + PPPoE Test Case + ============================================================================== + PPPoE Add Same Session Twice Test OK + PPPoE Decap Test OK + PPPoE Decap Multiple Sessions Test OK + PPPoE Delete Same Session Twice Test OK + PPPoE Encap Test OK + PPPoE Encap Multiple Sessions Test OK + +SPAN Switch Port Analyzer +````````````````````````` +:: + + ============================================================================== + SPAN Test Case + ============================================================================== + SPAN device rx mirror OK + SPAN l2 broadcast mirror OK + SPAN l2 rx tx mirror OK + SPAN l2 tx mirror OK + SPAN l2 rx mirror OK + SPAN l2 rx mirror into 1ad subif+vtr OK + SPAN l2 rx mirror into 1q subif+vtr OK + SPAN l2 rx mirror into gre-subif+vtr OK + SPAN l2 rx mirror into vxlan OK + +SRv6 Routing +```````````` +:: + + ============================================================================== + SRv6 Test Case + ============================================================================== + Test SRv6 End (without PSP) behavior. OK + Test SRv6 End.DT4 behavior. OK + Test SRv6 End.DT6 behavior. OK + Test SRv6 End.DX2 behavior. OK + Test SRv6 End.DX4 behavior. OK + Test SRv6 End.DX6 behavior. OK + Test SRv6 End.X (without PSP) behavior. OK + Test SRv6 End.X with PSP behavior. OK + Test SRv6 End with PSP behavior. OK + Test SRv6 Transit.Encaps behavior for IPv6. OK + Test SRv6 Transit.Encaps behavior for IPv4. OK + Test SRv6 Transit.Encaps behavior for L2. SKIP + Test SRv6 Transit.Insert behavior (IPv6 only). OK + Test SRv6 Transit.Insert behavior (IPv6 only). OK + +TCP/IP Stack +```````````` +:: + + ============================================================================== + TCP Test Case + ============================================================================== + TCP builtin client/server transfer OK + TCP Unit Tests OK + +UDP Stack +````````` +:: + + ============================================================================== + UDP Encap Test Case + ============================================================================== + UDP Encap test OK + +VTR VLAN Tag Rewrites +````````````````````` +:: + + ============================================================================== + VTR Test Case + ============================================================================== + 1AD VTR pop 1 test OK + 1AD VTR pop 2 test OK + 1AD VTR push 1 1AD test OK + 1AD VTR push 1 1Q test OK + 1AD VTR push 2 1AD test OK + 1AD VTR push 2 1Q test OK + 1AD VTR translate 1 -> 1 1AD test OK + 1AD VTR translate 1 -> 1 1Q test OK + 1AD VTR translate 1 -> 2 1AD test OK + 1AD VTR translate 1 -> 2 1Q test OK + 1AD VTR translate 2 -> 1 1AD test OK + 1AD VTR translate 2 -> 1 1Q test OK + 1AD VTR translate 2 -> 2 1AD test OK + 1AD VTR translate 2 -> 2 1Q test OK + 1Q VTR pop 1 test OK + 1Q VTR push 1 test OK + 1Q VTR push 2 test OK + 1Q VTR translate 1 -> 1 test OK + 1Q VTR translate 1 -> 2 test OK + +VXLAN Tunnels +````````````` +:: + + ============================================================================== + VXLAN Test Case + ============================================================================== + Decapsulation test OK + Encapsulation test OK + Multicast flood test OK + Multicast receive test OK + Unicast flood test OK + +VXLAN-GPE Tunnels +````````````````` +:: + + ============================================================================== + VXLAN-GPE Test Case + ============================================================================== + Decapsulation test SKIP + Encapsulation test SKIP + Multicast flood test SKIP + Multicast receive test SKIP + Unicast flood test SKIP + +Other Tests +``````````` +:: + + ============================================================================== + Ping Test Case + ============================================================================== + basic ping test OK + burst ping test OK + + ============================================================================== + Session Test Case + ============================================================================== + Session Unit Tests OK + + ============================================================================== + Template verification, timer tests + ============================================================================== + timer less than template timeout OK + timer greater than template timeout OK + verify cflow packet fields OK + diff --git a/docs/report/dpdk_performance_tests/documentation.rst b/docs/report/dpdk_performance_tests/documentation.rst index 47ec7cbe2a..254569dc66 100644 --- a/docs/report/dpdk_performance_tests/documentation.rst +++ b/docs/report/dpdk_performance_tests/documentation.rst @@ -1,5 +1,5 @@ -Documentation -============= - -`CSIT DPDK Performance Tests Documentation`_ contains detailed -functional description and input parameters for each test case. +Documentation +============= + +`CSIT DPDK Performance Tests Documentation`_ contains detailed +functional description and input parameters for each test case. diff --git a/docs/report/dpdk_performance_tests/test_environment.rst b/docs/report/dpdk_performance_tests/test_environment.rst index eb6617b11c..204792015f 100644 --- a/docs/report/dpdk_performance_tests/test_environment.rst +++ b/docs/report/dpdk_performance_tests/test_environment.rst @@ -1,68 +1,68 @@ -.. include:: ../vpp_performance_tests/test_environment_intro.rst - -.. include:: ../vpp_performance_tests/test_environment_sut_conf_1.rst - -.. include:: ../vpp_performance_tests/test_environment_sut_conf_3.rst - - -DUT Configuration - DPDK ------------------------- - -**DPDK Version** - -|dpdk-release| - -**DPDK Compile Parameters** - -.. code-block:: bash - - make install T=x86_64-native-linuxapp-gcc -j - -**Testpmd Startup Configuration** - -Testpmd startup configuration changes per test case with different settings for CPU -cores, rx-queues. Startup config is aligned with applied test case tag: - -Tagged by **1T1C** - -.. code-block:: bash - - testpmd -c 0x3 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=1 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=1 --txq=1 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start - -Tagged by **2T2C** - -.. code-block:: bash - - testpmd -c 0x403 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=2 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=1 --txq=1 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start - -Tagged by **4T4C** - -.. code-block:: bash - - testpmd -c 0xc07 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=4 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=2 --txq=2 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start - -**L3FWD Startup Configuration** - -L3FWD startup configuration changes per test case with different settings for CPU -cores, rx-queues. Startup config is aligned with applied test case tag: - -Tagged by **1T1C** - -.. code-block:: bash - - l3fwd -l 1 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype - -Tagged by **2T2C** - -.. code-block:: bash - - l3fwd -l 1,2 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype - -Tagged by **4T4C** - -.. code-block:: bash - - l3fwd -l 1,2,3,4 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype - - -.. include:: ../vpp_performance_tests/test_environment_tg.rst +.. include:: ../vpp_performance_tests/test_environment_intro.rst + +.. include:: ../vpp_performance_tests/test_environment_sut_conf_1.rst + +.. include:: ../vpp_performance_tests/test_environment_sut_conf_3.rst + + +DUT Configuration - DPDK +------------------------ + +**DPDK Version** + +|dpdk-release| + +**DPDK Compile Parameters** + +.. code-block:: bash + + make install T=x86_64-native-linuxapp-gcc -j + +**Testpmd Startup Configuration** + +Testpmd startup configuration changes per test case with different settings for CPU +cores, rx-queues. Startup config is aligned with applied test case tag: + +Tagged by **1T1C** + +.. code-block:: bash + + testpmd -c 0x3 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=1 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=1 --txq=1 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start + +Tagged by **2T2C** + +.. code-block:: bash + + testpmd -c 0x403 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=2 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=1 --txq=1 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start + +Tagged by **4T4C** + +.. code-block:: bash + + testpmd -c 0xc07 -n 4 -- --numa --nb-ports=2 --portmask=0x3 --nb-cores=4 --max-pkt-len=9000 --txqflags=0 --forward-mode=io --rxq=2 --txq=2 --burst=64 --rxd=1024 --txd=1024 --disable-link-check --auto-start + +**L3FWD Startup Configuration** + +L3FWD startup configuration changes per test case with different settings for CPU +cores, rx-queues. Startup config is aligned with applied test case tag: + +Tagged by **1T1C** + +.. code-block:: bash + + l3fwd -l 1 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype + +Tagged by **2T2C** + +.. code-block:: bash + + l3fwd -l 1,2 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype + +Tagged by **4T4C** + +.. code-block:: bash + + l3fwd -l 1,2,3,4 -n 4 -- -P -L -p 0x3 --config='${port_config}' --enable-jumbo --max-pkt-len=9000 --eth-dest=0,${adj_mac0} --eth-dest=1,${adj_mac1} --parse-ptype + + +.. include:: ../vpp_performance_tests/test_environment_tg.rst diff --git a/docs/report/honeycomb_functional_tests/csit_release_notes.rst b/docs/report/honeycomb_functional_tests/csit_release_notes.rst index f47a15af5a..c1c46ff9e7 100644 --- a/docs/report/honeycomb_functional_tests/csit_release_notes.rst +++ b/docs/report/honeycomb_functional_tests/csit_release_notes.rst @@ -1,31 +1,31 @@ -CSIT Release Notes -================== - -Changes in CSIT |release| -------------------------- - -No changes. - -Known Issues ------------- - -Here is the list of known issues in CSIT |release| for Honeycomb functional -tests in VIRL: - -+---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ -| # | Issue | Jira ID | Description | -+---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ -| 1 | IPv6 BGP route configuration | HONEYCOMB-403 | Configuring Ipv6 route results in missing writer | -| | | | for IPv6Route and IPv6NextHop exception. | -+---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ -| 2 | IP address subnet validation | VPP-649 | When configuring two IP addresses from the same subnet on an interface, | -| | | | VPP refuses the configuration but returns code 200:OK. This can cause | -| | | | desync between Honeycomb's config and operational data. | -+---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ -| 3 | VxLAN GPE configuration crashes VPP | VPP-875 | Specific VxLAN GPE configurations cause VPP to crash and restart. | -+---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ -| 4 | Operational data for IPv6 special routes | HC2VPP-254 | Special hop routes are misidentified as regular routes | -| | | | in operational data. | -+---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ -| 5 | LISP PITR feature configuration | HC2VPP-263 | Locator set reference in operational data is incorrect. | -+---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ +CSIT Release Notes +================== + +Changes in CSIT |release| +------------------------- + +No changes. + +Known Issues +------------ + +Here is the list of known issues in CSIT |release| for Honeycomb functional +tests in VIRL: + ++---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ +| # | Issue | Jira ID | Description | ++---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ +| 1 | IPv6 BGP route configuration | HONEYCOMB-403 | Configuring Ipv6 route results in missing writer | +| | | | for IPv6Route and IPv6NextHop exception. | ++---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ +| 2 | IP address subnet validation | VPP-649 | When configuring two IP addresses from the same subnet on an interface, | +| | | | VPP refuses the configuration but returns code 200:OK. This can cause | +| | | | desync between Honeycomb's config and operational data. | ++---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ +| 3 | VxLAN GPE configuration crashes VPP | VPP-875 | Specific VxLAN GPE configurations cause VPP to crash and restart. | ++---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ +| 4 | Operational data for IPv6 special routes | HC2VPP-254 | Special hop routes are misidentified as regular routes | +| | | | in operational data. | ++---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ +| 5 | LISP PITR feature configuration | HC2VPP-263 | Locator set reference in operational data is incorrect. | ++---+--------------------------------------------+---------------+-------------------------------------------------------------------------+ diff --git a/docs/report/honeycomb_functional_tests/documentation.rst b/docs/report/honeycomb_functional_tests/documentation.rst index 7ded3d4181..354c4062f6 100644 --- a/docs/report/honeycomb_functional_tests/documentation.rst +++ b/docs/report/honeycomb_functional_tests/documentation.rst @@ -1,5 +1,5 @@ -Documentation -============= - -`CSIT Honeycomb Functional Tests Documentation`_ contains detailed -functional description and input parameters for each test case. +Documentation +============= + +`CSIT Honeycomb Functional Tests Documentation`_ contains detailed +functional description and input parameters for each test case. diff --git a/docs/report/honeycomb_functional_tests/test_environment.rst b/docs/report/honeycomb_functional_tests/test_environment.rst index 8f7e7efa51..8d8d832e92 100644 --- a/docs/report/honeycomb_functional_tests/test_environment.rst +++ b/docs/report/honeycomb_functional_tests/test_environment.rst @@ -1,5 +1,5 @@ -Test Environment -================ - -CSIT functional tests are currently executed in FD.IO VIRL testbed. -See `VPP Functional Tests Environment <../vpp_functional_tests/test_environment.html>`_ for more details. +Test Environment +================ + +CSIT functional tests are currently executed in FD.IO VIRL testbed. +See `VPP Functional Tests Environment <../vpp_functional_tests/test_environment.html>`_ for more details. diff --git a/docs/report/nsh_sfc_functional_tests/documentation.rst b/docs/report/nsh_sfc_functional_tests/documentation.rst index 709f2a8a84..9e3298a295 100644 --- a/docs/report/nsh_sfc_functional_tests/documentation.rst +++ b/docs/report/nsh_sfc_functional_tests/documentation.rst @@ -1,6 +1,6 @@ -Documentation -============= - -`CSIT NSH_SFC Functional Tests Documentation`_ contains detailed -functional description and input parameters for each test case. - +Documentation +============= + +`CSIT NSH_SFC Functional Tests Documentation`_ contains detailed +functional description and input parameters for each test case. + diff --git a/docs/report/nsh_sfc_functional_tests/overview.rst b/docs/report/nsh_sfc_functional_tests/overview.rst index 6420d6a660..754fd9242d 100644 --- a/docs/report/nsh_sfc_functional_tests/overview.rst +++ b/docs/report/nsh_sfc_functional_tests/overview.rst @@ -1,118 +1,118 @@ -Overview -======== - -Tested Virtual Topologies -------------------------- - -CSIT NSH_SFC functional tests are executed on virtualized topologies created using -:abbr:`VIRL (Virtual Internet Routing Lab)` simulation platform contributed by -Cisco. VIRL runs on physical baremetal servers hosted by LF FD.io project. -Majority of the tests are executed in the three node logical test topology - -Traffic Generator (TG) node and two Systems Under Test (SUT) nodes connected in -a loop. Some tests use two node logical test topology - TG node and SUT1 node. -Both logical test topologies are shown in the figures below.:: - - +------------------------+ +------------------------+ - | | | | - | +------------------+ | | +------------------+ | - | | <-----------------> | | - | | | | | | | | - | | DUT1 <-----------------> DUT2 | | - | +--^--^------------+ | | +------------^--^--+ | - | | | | | | | | - | | | SUT1 | | SUT2 | | | - +------------------------+ +------------------------+ - | | | | - | | | | - | | +-----------+ | | - | +---------------> <---------------+ | - | | TG | | - +------------------> <------------------+ - +-----------+ - - +------------------------+ - | | - | +------------------+ | - +---------------> <--------------+ - | | | | | | - | |------------> DUT1 <-----------+ | - | | | +------------------+ | | | - | | | | | | - | | | SUT1 | | | - | | +------------------------+ | | - | | | | - | | | | - | | +-----------+ | | - | +---------------> <---------------+ | - | | TG | | - +------------------> <------------------+ - +-----------+ - -SUT1 and SUT2 are two VMs (Ubuntu or Centos, depending on the test suite), TG -is a Traffic Generator (TG, another Ubuntu VM). SUTs run VPP SW application in -Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy SW -application as a packet Traffic Generator. Logical connectivity between SUTs -and to TG is provided using virtual NICs using VMs' virtio driver. - -Virtual testbeds are created on-demand whenever a verification job is started -(e.g. triggered by the gerrit patch submission) and destroyed upon completion -of all functional tests. Each node is a Virtual Machine and each connection -that is drawn on the diagram is available for use in any test case. During the -test execution, all nodes are reachable thru the Management network connected -to every node via dedicated virtual NICs and virtual links (not shown above -for clarity). - -For the test cases that require DUT (VPP) to communicate with VM over the -vhost-user interfaces, a nested VM is created on SUT1 and/or SUT2 for the -duration of these particular test cases only. DUT (VPP) test topology with VM -is shown in the figure below including the applicable packet flow thru the VM -(marked in the figure with ``***``).:: - - +------------------------+ +------------------------+ - | +----------+ | | +----------+ | - | | VM | | | | VM | | - | | ****** | | | | ****** | | - | +--^----^--+ | | +--^----^--+ | - | *| |* | | *| |* | - | +------v----v------+ | | +------v----v------+ | - | | * * |**|***********|**| * * | | - | | ***** *******<----------------->******* ***** | | - | | * DUT1 | | | | DUT2 * | | - | +--^---------------+ | | +---------------^--+ | - | *| | | |* | - | *| SUT1 | | SUT2 |* | - +------------------------+ +------------------^-----+ - *| |* - *| |* - *| +-----------+ |* - *| | | |* - *+------------------> TG <------------------+* - ******************* | |******************** - +-----------+ - -NSH_SFC Functional Tests Coverage ---------------------------------- - -Following NSH_SFC functional test areas are covered in the CSIT |release| with -results listed in this report: - -- **NSH SFC Classifier** - TG sends some TCP packets to test NSH SFC - Classifier functional. DUT1 will receive these packets from one NIC and loopback - the VXLAN-GPE-NSH encapsulated packets to the TG from other NIC. - - Test case count: 7 -- **NSH SFC Proxy Inbound** - TG sends some VXLAN-GPE-NSH encapsulated packets - to test NSH SFC Proxy Inbound functional. DUT1 will receive these packets from one - NIC and loopback the VXLAN encapsulated packets to the TG from other NIC. - - Test case count: 6 -- **NSH SFC Proxy Outbound** - TG sends some VXLAN encapsulated packets to test - NSH SFC Proxy Outbound functional. DUT1 will receive these packets from one NIC - and loopback the VXLAN-GPE-NSH encapsulated packets to the TG from other NIC. - - Test case count: 6 -- **NSH SFC Service Function Forward** - TG sends some VXLAN-GPE-NSH encapsulated - packets to test NSH SFC Service Function Forward functional. DUT1 will receive these - packets from one NIC and swap the VXLAN-GPE-NSH header, after that DUT1 loopback the - VXLAN-GPE-NSH encapsulated packtes to the TG from other NIC. - - Test case count: 6 - -Total 25 NSH SFC functional tests in the CSIT |release|. - +Overview +======== + +Tested Virtual Topologies +------------------------- + +CSIT NSH_SFC functional tests are executed on virtualized topologies created using +:abbr:`VIRL (Virtual Internet Routing Lab)` simulation platform contributed by +Cisco. VIRL runs on physical baremetal servers hosted by LF FD.io project. +Majority of the tests are executed in the three node logical test topology - +Traffic Generator (TG) node and two Systems Under Test (SUT) nodes connected in +a loop. Some tests use two node logical test topology - TG node and SUT1 node. +Both logical test topologies are shown in the figures below.:: + + +------------------------+ +------------------------+ + | | | | + | +------------------+ | | +------------------+ | + | | <-----------------> | | + | | | | | | | | + | | DUT1 <-----------------> DUT2 | | + | +--^--^------------+ | | +------------^--^--+ | + | | | | | | | | + | | | SUT1 | | SUT2 | | | + +------------------------+ +------------------------+ + | | | | + | | | | + | | +-----------+ | | + | +---------------> <---------------+ | + | | TG | | + +------------------> <------------------+ + +-----------+ + + +------------------------+ + | | + | +------------------+ | + +---------------> <--------------+ + | | | | | | + | |------------> DUT1 <-----------+ | + | | | +------------------+ | | | + | | | | | | + | | | SUT1 | | | + | | +------------------------+ | | + | | | | + | | | | + | | +-----------+ | | + | +---------------> <---------------+ | + | | TG | | + +------------------> <------------------+ + +-----------+ + +SUT1 and SUT2 are two VMs (Ubuntu or Centos, depending on the test suite), TG +is a Traffic Generator (TG, another Ubuntu VM). SUTs run VPP SW application in +Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy SW +application as a packet Traffic Generator. Logical connectivity between SUTs +and to TG is provided using virtual NICs using VMs' virtio driver. + +Virtual testbeds are created on-demand whenever a verification job is started +(e.g. triggered by the gerrit patch submission) and destroyed upon completion +of all functional tests. Each node is a Virtual Machine and each connection +that is drawn on the diagram is available for use in any test case. During the +test execution, all nodes are reachable thru the Management network connected +to every node via dedicated virtual NICs and virtual links (not shown above +for clarity). + +For the test cases that require DUT (VPP) to communicate with VM over the +vhost-user interfaces, a nested VM is created on SUT1 and/or SUT2 for the +duration of these particular test cases only. DUT (VPP) test topology with VM +is shown in the figure below including the applicable packet flow thru the VM +(marked in the figure with ``***``).:: + + +------------------------+ +------------------------+ + | +----------+ | | +----------+ | + | | VM | | | | VM | | + | | ****** | | | | ****** | | + | +--^----^--+ | | +--^----^--+ | + | *| |* | | *| |* | + | +------v----v------+ | | +------v----v------+ | + | | * * |**|***********|**| * * | | + | | ***** *******<----------------->******* ***** | | + | | * DUT1 | | | | DUT2 * | | + | +--^---------------+ | | +---------------^--+ | + | *| | | |* | + | *| SUT1 | | SUT2 |* | + +------------------------+ +------------------^-----+ + *| |* + *| |* + *| +-----------+ |* + *| | | |* + *+------------------> TG <------------------+* + ******************* | |******************** + +-----------+ + +NSH_SFC Functional Tests Coverage +--------------------------------- + +Following NSH_SFC functional test areas are covered in the CSIT |release| with +results listed in this report: + +- **NSH SFC Classifier** - TG sends some TCP packets to test NSH SFC + Classifier functional. DUT1 will receive these packets from one NIC and loopback + the VXLAN-GPE-NSH encapsulated packets to the TG from other NIC. + - Test case count: 7 +- **NSH SFC Proxy Inbound** - TG sends some VXLAN-GPE-NSH encapsulated packets + to test NSH SFC Proxy Inbound functional. DUT1 will receive these packets from one + NIC and loopback the VXLAN encapsulated packets to the TG from other NIC. + - Test case count: 6 +- **NSH SFC Proxy Outbound** - TG sends some VXLAN encapsulated packets to test + NSH SFC Proxy Outbound functional. DUT1 will receive these packets from one NIC + and loopback the VXLAN-GPE-NSH encapsulated packets to the TG from other NIC. + - Test case count: 6 +- **NSH SFC Service Function Forward** - TG sends some VXLAN-GPE-NSH encapsulated + packets to test NSH SFC Service Function Forward functional. DUT1 will receive these + packets from one NIC and swap the VXLAN-GPE-NSH header, after that DUT1 loopback the + VXLAN-GPE-NSH encapsulated packtes to the TG from other NIC. + - Test case count: 6 + +Total 25 NSH SFC functional tests in the CSIT |release|. + diff --git a/docs/report/nsh_sfc_functional_tests/test_environment.rst b/docs/report/nsh_sfc_functional_tests/test_environment.rst index ca7285d5e7..3b35fd432f 100644 --- a/docs/report/nsh_sfc_functional_tests/test_environment.rst +++ b/docs/report/nsh_sfc_functional_tests/test_environment.rst @@ -1 +1 @@ -.. include:: ../vpp_functional_tests/test_environment.rst +.. include:: ../vpp_functional_tests/test_environment.rst diff --git a/docs/report/test_operational_data/index.rst b/docs/report/test_operational_data/index.rst index d5eca487c0..a85e8eaea2 100644 --- a/docs/report/test_operational_data/index.rst +++ b/docs/report/test_operational_data/index.rst @@ -1,8 +1,8 @@ -.. only:: html - - Test Operational Data - ===================== - - .. toctree:: - - vpp_performance_operational_data/index +.. only:: html + + Test Operational Data + ===================== + + .. toctree:: + + vpp_performance_operational_data/index diff --git a/docs/report/vpp_functional_tests/documentation.rst b/docs/report/vpp_functional_tests/documentation.rst index e0f180c3ec..6d99185892 100644 --- a/docs/report/vpp_functional_tests/documentation.rst +++ b/docs/report/vpp_functional_tests/documentation.rst @@ -1,6 +1,6 @@ -Documentation -============= - -`CSIT VPP Functional Tests Documentation`_ contains detailed -functional description and input parameters for each test case. - +Documentation +============= + +`CSIT VPP Functional Tests Documentation`_ contains detailed +functional description and input parameters for each test case. + diff --git a/docs/report/vpp_functional_tests/overview.rst b/docs/report/vpp_functional_tests/overview.rst index 1cd5def87d..32d70f102b 100644 --- a/docs/report/vpp_functional_tests/overview.rst +++ b/docs/report/vpp_functional_tests/overview.rst @@ -1,149 +1,149 @@ -Overview -======== - -Tested Virtual Topologies -------------------------- - -CSIT VPP functional tests are executed on virtualized topologies created using -:abbr:`VIRL (Virtual Internet Routing Lab)` simulation platform contributed by -Cisco. VIRL runs on physical baremetal servers hosted by LF FD.io project. -Majority of the tests are executed in the three node logical test topology - -Traffic Generator (TG) node and two Systems Under Test (SUT) nodes connected in -a loop. Some tests use two node logical test topology - TG node and SUT1 node. -Both logical test topologies are shown in the figures below.:: - - +------------------------+ +------------------------+ - | | | | - | +------------------+ | | +------------------+ | - | | <-----------------> | | - | | | | | | | | - | | DUT1 <-----------------> DUT2 | | - | +--^--^------------+ | | +------------^--^--+ | - | | | | | | | | - | | | SUT1 | | SUT2 | | | - +------------------------+ +------------------------+ - | | | | - | | | | - | | +-----------+ | | - | +---------------> <---------------+ | - | | TG | | - +------------------> <------------------+ - +-----------+ - - +------------------------+ - | | - | +------------------+ | - +---------------> <--------------+ - | | | | | | - | |------------> DUT1 <-----------+ | - | | | +------------------+ | | | - | | | | | | - | | | SUT1 | | | - | | +------------------------+ | | - | | | | - | | | | - | | +-----------+ | | - | +---------------> <---------------+ | - | | TG | | - +------------------> <------------------+ - +-----------+ - -SUT1 and SUT2 are two VMs (Ubuntu or Centos, depending on the test suite), TG -is a Traffic Generator (TG, another Ubuntu VM). SUTs run VPP SW application in -Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy SW -application as a packet Traffic Generator. Logical connectivity between SUTs -and to TG is provided using virtual NICs using VMs' virtio driver. - -Virtual testbeds are created on-demand whenever a verification job is started -(e.g. triggered by the gerrit patch submission) and destroyed upon completion -of all functional tests. Each node is a Virtual Machine and each connection -that is drawn on the diagram is available for use in any test case. During the -test execution, all nodes are reachable thru the Management network connected -to every node via dedicated virtual NICs and virtual links (not shown above -for clarity). - -For the test cases that require DUT (VPP) to communicate with VM over the -vhost-user interfaces, a nested VM is created on SUT1 and/or SUT2 for the -duration of these particular test cases only. DUT (VPP) test topology with VM -is shown in the figure below including the applicable packet flow thru the VM -(marked in the figure with ``***``).:: - - +------------------------+ +------------------------+ - | +----------+ | | +----------+ | - | | VM | | | | VM | | - | | ****** | | | | ****** | | - | +--^----^--+ | | +--^----^--+ | - | *| |* | | *| |* | - | +------v----v------+ | | +------v----v------+ | - | | * * |**|***********|**| * * | | - | | ***** *******<----------------->******* ***** | | - | | * DUT1 | | | | DUT2 * | | - | +--^---------------+ | | +---------------^--+ | - | *| | | |* | - | *| SUT1 | | SUT2 |* | - +------------------------+ +------------------^-----+ - *| |* - *| |* - *| +-----------+ |* - *| | | |* - *+------------------> TG <------------------+* - ******************* | |******************** - +-----------+ - -Functional Tests Coverage -------------------------- - -Following VPP functional test areas are covered in the CSIT |release| with -results listed in this report: - -- **DHCP - Client and Proxy** - Dynamic Host Control Protocol Client and Proxy - for IPv4, IPv6. -- **GRE Overlay Tunnels** - Generic Routing Encapsulation for IPv4. -- **L2BD Ethernet Switching** - L2 Bridge-Domain switched-forwarding for - untagged Ethernet, dot1q and dot1ad tagged. -- **L2XC Ethernet Switching** - L2 Cross-Connect switched-forwarding for - untagged Ethernet, dot1q and dot1ad tagged. -- **LISP Overlay Tunnels** - Locator/ID Separation Protocol overlay tunnels and - locator/id mapping control. -- **Softwire Tunnels** - IPv4-in-IPv6 softwire tunnels. -- **Cop Address Security** - address white-list and black-list filtering for - IPv4, IPv6. -- **IPSec - Tunnels and Transport** - IPSec tunnel and transport modes. -- **IPv6 Routed-Forwarding** - IPv6 routed-forwarding, NS/ND, RA, ICMPv6. -- **uRPF Source Security** - unicast Reverse Path Forwarding security. -- **Tap Interface** - baseline Linux tap interface tests. -- **Telemetry - IPFIX and SPAN** - IPFIX netflow statistics and SPAN port - mirroring. -- **VRF Routed-Forwarding** - multi-context IPVPN routed-forwarding for IPv4, - IPv6. -- **iACL Security** - ingress Access Control List security for IPv4, IPv6, MAC. -- **IPv4 Routed-Forwarding** - IPv4 routed-forwarding, RPF, ARP, Proxy ARP, - ICMPv4. -- **QoS Policer Metering** - ingress packet rate measuring and marking for IPv4, - IPv6. -- **VLAN Tag Translation** - L2 VLAN tag translation 2to2, 2to1, 1to2, 1to1. -- **VXLAN Overlay Tunnels** - VXLAN tunneling for L2-over-IP, for IPv4, IPv6. - -Functional Tests Naming ------------------------ - -CSIT |release| follows a common structured naming convention for all performance -and system functional tests, introduced in CSIT rls1701. - -The naming should be intuitive for majority of the tests. Complete description -of CSIT test naming convention is provided on :ref:`csit_test_naming`.. - -Here few illustrative examples of the new naming usage for functional test -suites: - -#. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P** - - - *eth2p-ethip4-ip4base-func.robot* => 2 ports of Ethernet, IPv4 baseline - routed forwarding, functional tests. - -#. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC, - P2V2P, NIC2VMchain2NIC, P2V2V2P** - - - *eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot* => 2 ports of - Ethernet, IPv4 VXLAN Ethernet, L2 bridge-domain switching to/from two vhost - interfaces and one VM, functional tests. +Overview +======== + +Tested Virtual Topologies +------------------------- + +CSIT VPP functional tests are executed on virtualized topologies created using +:abbr:`VIRL (Virtual Internet Routing Lab)` simulation platform contributed by +Cisco. VIRL runs on physical baremetal servers hosted by LF FD.io project. +Majority of the tests are executed in the three node logical test topology - +Traffic Generator (TG) node and two Systems Under Test (SUT) nodes connected in +a loop. Some tests use two node logical test topology - TG node and SUT1 node. +Both logical test topologies are shown in the figures below.:: + + +------------------------+ +------------------------+ + | | | | + | +------------------+ | | +------------------+ | + | | <-----------------> | | + | | | | | | | | + | | DUT1 <-----------------> DUT2 | | + | +--^--^------------+ | | +------------^--^--+ | + | | | | | | | | + | | | SUT1 | | SUT2 | | | + +------------------------+ +------------------------+ + | | | | + | | | | + | | +-----------+ | | + | +---------------> <---------------+ | + | | TG | | + +------------------> <------------------+ + +-----------+ + + +------------------------+ + | | + | +------------------+ | + +---------------> <--------------+ + | | | | | | + | |------------> DUT1 <-----------+ | + | | | +------------------+ | | | + | | | | | | + | | | SUT1 | | | + | | +------------------------+ | | + | | | | + | | | | + | | +-----------+ | | + | +---------------> <---------------+ | + | | TG | | + +------------------> <------------------+ + +-----------+ + +SUT1 and SUT2 are two VMs (Ubuntu or Centos, depending on the test suite), TG +is a Traffic Generator (TG, another Ubuntu VM). SUTs run VPP SW application in +Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy SW +application as a packet Traffic Generator. Logical connectivity between SUTs +and to TG is provided using virtual NICs using VMs' virtio driver. + +Virtual testbeds are created on-demand whenever a verification job is started +(e.g. triggered by the gerrit patch submission) and destroyed upon completion +of all functional tests. Each node is a Virtual Machine and each connection +that is drawn on the diagram is available for use in any test case. During the +test execution, all nodes are reachable thru the Management network connected +to every node via dedicated virtual NICs and virtual links (not shown above +for clarity). + +For the test cases that require DUT (VPP) to communicate with VM over the +vhost-user interfaces, a nested VM is created on SUT1 and/or SUT2 for the +duration of these particular test cases only. DUT (VPP) test topology with VM +is shown in the figure below including the applicable packet flow thru the VM +(marked in the figure with ``***``).:: + + +------------------------+ +------------------------+ + | +----------+ | | +----------+ | + | | VM | | | | VM | | + | | ****** | | | | ****** | | + | +--^----^--+ | | +--^----^--+ | + | *| |* | | *| |* | + | +------v----v------+ | | +------v----v------+ | + | | * * |**|***********|**| * * | | + | | ***** *******<----------------->******* ***** | | + | | * DUT1 | | | | DUT2 * | | + | +--^---------------+ | | +---------------^--+ | + | *| | | |* | + | *| SUT1 | | SUT2 |* | + +------------------------+ +------------------^-----+ + *| |* + *| |* + *| +-----------+ |* + *| | | |* + *+------------------> TG <------------------+* + ******************* | |******************** + +-----------+ + +Functional Tests Coverage +------------------------- + +Following VPP functional test areas are covered in the CSIT |release| with +results listed in this report: + +- **DHCP - Client and Proxy** - Dynamic Host Control Protocol Client and Proxy + for IPv4, IPv6. +- **GRE Overlay Tunnels** - Generic Routing Encapsulation for IPv4. +- **L2BD Ethernet Switching** - L2 Bridge-Domain switched-forwarding for + untagged Ethernet, dot1q and dot1ad tagged. +- **L2XC Ethernet Switching** - L2 Cross-Connect switched-forwarding for + untagged Ethernet, dot1q and dot1ad tagged. +- **LISP Overlay Tunnels** - Locator/ID Separation Protocol overlay tunnels and + locator/id mapping control. +- **Softwire Tunnels** - IPv4-in-IPv6 softwire tunnels. +- **Cop Address Security** - address white-list and black-list filtering for + IPv4, IPv6. +- **IPSec - Tunnels and Transport** - IPSec tunnel and transport modes. +- **IPv6 Routed-Forwarding** - IPv6 routed-forwarding, NS/ND, RA, ICMPv6. +- **uRPF Source Security** - unicast Reverse Path Forwarding security. +- **Tap Interface** - baseline Linux tap interface tests. +- **Telemetry - IPFIX and SPAN** - IPFIX netflow statistics and SPAN port + mirroring. +- **VRF Routed-Forwarding** - multi-context IPVPN routed-forwarding for IPv4, + IPv6. +- **iACL Security** - ingress Access Control List security for IPv4, IPv6, MAC. +- **IPv4 Routed-Forwarding** - IPv4 routed-forwarding, RPF, ARP, Proxy ARP, + ICMPv4. +- **QoS Policer Metering** - ingress packet rate measuring and marking for IPv4, + IPv6. +- **VLAN Tag Translation** - L2 VLAN tag translation 2to2, 2to1, 1to2, 1to1. +- **VXLAN Overlay Tunnels** - VXLAN tunneling for L2-over-IP, for IPv4, IPv6. + +Functional Tests Naming +----------------------- + +CSIT |release| follows a common structured naming convention for all performance +and system functional tests, introduced in CSIT rls1701. + +The naming should be intuitive for majority of the tests. Complete description +of CSIT test naming convention is provided on :ref:`csit_test_naming`.. + +Here few illustrative examples of the new naming usage for functional test +suites: + +#. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P** + + - *eth2p-ethip4-ip4base-func.robot* => 2 ports of Ethernet, IPv4 baseline + routed forwarding, functional tests. + +#. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC, + P2V2P, NIC2VMchain2NIC, P2V2V2P** + + - *eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot* => 2 ports of + Ethernet, IPv4 VXLAN Ethernet, L2 bridge-domain switching to/from two vhost + interfaces and one VM, functional tests. diff --git a/docs/report/vpp_functional_tests/test_environment.rst b/docs/report/vpp_functional_tests/test_environment.rst index 175e5181b0..dfc40b0040 100644 --- a/docs/report/vpp_functional_tests/test_environment.rst +++ b/docs/report/vpp_functional_tests/test_environment.rst @@ -1,622 +1,622 @@ -Test Environment -================ - -CSIT functional tests are currently executed in FD.IO VIRL testbed. The physical -VIRL testbed infrastructure consists of three VIRL hosts: - -- All hosts are Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @2.30GHz, - 18c, 512GB RAM) - -- tb4-virl1: - - - Status: Production - - OS: Ubuntu 16.04.2 - - STD server version 0.10.32.16 - - UWM server version 0.10.32.16 - -- tb4-virl2: - - - Status: Production - - OS: Ubuntu 16.04.2 - - STD server version 0.10.32.16 - - UWM server version 0.10.32.16 - -- tb4-virl3: - - - Status: Testing - - OS: Ubuntu 16.04.2 - - STD server version 0.10.32.19 - - UWM server version 0.10.32.19 - -Whenever a patch is submitted to gerrit for review, parallel VIRL simulations -are started to reduce the time of execution of all functional tests. The number -of parallel VIRL simulations is equal to number of test groups defined by -TEST_GROUPS variable in :file:`csit/bootstrap.sh` file. The VIRL host to run -VIRL simulation is selected based on least load algorithm per VIRL simulation. - -Every VIRL simulation uses the same three-node - Traffic Generator (TG node) and -two Systems Under Test (SUT1 and SUT2) - "double-ring" topology. The appropriate -pre-built VPP packages built by Jenkins for the patch under review are then -installed on the two SUTs, along with their :file:`/etc/vpp/startup.conf` file, -in all VIRL simulations. - -SUT Configuration - VIRL Guest VM ---------------------------------- - -Configurations of the SUT VMs is defined in `VIRL topologies directory`_ - -- List of SUT VM interfaces::: - - - - - - -- Number of 2MB hugepages: 1024 - -- Maximum number of memory map areas: 20000 - -- Kernel Shared Memory Max: 2147483648 (vm.nr_hugepages * 2 * 1024 * 1024) - -SUT Configuration - VIRL Guest OS Linux ---------------------------------------- - -In CSIT terminology, the VM operating system for both SUTs that |vpp-release| has -been tested with, is the following: - -#. **Ubuntu VIRL image** - - This image implies Ubuntu 16.04.1 LTS, current as of yyyy-mm-dd (that is, - package versions are those that would have been installed by a - :command:`apt-get update`, :command:`apt-get upgrade` on that day), produced - by CSIT disk image build scripts. - - The exact list of installed packages and their versions (including the Linux - kernel package version) are included in `VIRL ubuntu images lists`_. - - A replica of this VM image can be built by running the :command:`build.sh` - script in CSIT repository. - -#. **CentOS VIRL image** - - This image implies Centos 7.4-1711, current as of yyyy-mm-dd (that is, - package versions are those that would have been installed by a - :command:`yum update`, :command:`yum upgrade` on that day), produced - by CSIT disk image build scripts. - - The exact list of installed packages and their versions (including the Linux - kernel package version) are included in `VIRL centos images lists`_. - - A replica of this VM image can be built by running the :command:`build.sh` - script in CSIT repository. - -#. **Nested VM image** - - In addition to the "main" VM image, tests which require VPP to communicate to - a VM over a vhost-user interface, utilize a "nested" VM image. - - This "nested" VM is dynamically created and destroyed as part of a test case, - and therefore the "nested" VM image is optimized to be small, lightweight and - have a short boot time. The "nested" VM image is not built around any - established Linux distribution, but is based on `BuildRoot - `_, a tool for building embedded Linux systems. Just - as for the "main" image, scripts to produce an identical replica of the - "nested" image are included in CSIT GIT repository, and the image can be - rebuilt using the "build.sh" script at `VIRL nested`_. - -DUT Configuration - VPP ------------------------ - -Every System Under Test runs VPP SW application in Linux user-mode as a Device -Under Test (DUT) node. - -**DUT port configuration** - -Port configuration of DUTs is defined in topology file that is generated per -VIRL simulation based on the definition stored in `VIRL topologies directory`_. - -Example of DUT nodes configuration::: - - DUT1: - type: DUT - host: "10.30.51.157" - port: 22 - username: cisco - honeycomb: - user: admin - passwd: admin - port: 8183 - netconf_port: 2831 - priv_key: | - -----BEGIN RSA PRIVATE KEY----- - MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb - v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd - vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2 - 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc - 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs - 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7 - 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN - EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN - 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU - MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD - p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW - 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG - OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD - GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef - bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg - 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu - f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u - UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/ - +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT - D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/ - sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn - g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY - qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8 - EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD - BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN - -----END RSA PRIVATE KEY----- - - interfaces: - port1: - mac_address: "fa:16:3e:9b:89:52" - pci_address: "0000:00:04.0" - link: link1 - port2: - mac_address: "fa:16:3e:7a:33:60" - pci_address: "0000:00:05.0" - link: link4 - port3: - mac_address: "fa:16:3e:29:b7:ae" - pci_address: "0000:00:06.0" - link: link3 - port4: - mac_address: "fa:16:3e:76:8d:ff" - pci_address: "0000:00:07.0" - link: link6 - DUT2: - type: DUT - host: "10.30.51.156" - port: 22 - username: cisco - honeycomb: - user: admin - passwd: admin - port: 8183 - netconf_port: 2831 - priv_key: | - -----BEGIN RSA PRIVATE KEY----- - MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb - v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd - vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2 - 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc - 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs - 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7 - 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN - EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN - 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU - MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD - p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW - 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG - OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD - GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef - bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg - 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu - f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u - UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/ - +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT - D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/ - sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn - g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY - qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8 - EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD - BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN - -----END RSA PRIVATE KEY----- - - interfaces: - port1: - mac_address: "fa:16:3e:ad:6c:7d" - pci_address: "0000:00:04.0" - link: link2 - port2: - mac_address: "fa:16:3e:94:a4:99" - pci_address: "0000:00:05.0" - link: link5 - port3: - mac_address: "fa:16:3e:75:92:da" - pci_address: "0000:00:06.0" - link: link3 - port4: - mac_address: "fa:16:3e:2c:b1:2a" - pci_address: "0000:00:07.0" - link: link6 - -**VPP Version** - -|vpp-release| - -**VPP Installed Packages - Ubuntu** -:: - - $ dpkg -l vpp\* - Desired=Unknown/Install/Remove/Purge/Hold - | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend - |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) - ||/ Name Version Architecture Description - +++-==============-=============-============-================================================= - ii vpp 18.04-release amd64 Vector Packet Processing--executables - ii vpp-dbg 18.04-release amd64 Vector Packet Processing--debug symbols - ii vpp-dev 18.04-release amd64 Vector Packet Processing--development support - ii vpp-dpdk-dkms 18.02-vpp1 amd64 DPDK Development Package for VPP - Kernel Modules - ii vpp-lib 18.04-release amd64 Vector Packet Processing--runtime libraries - ii vpp-plugins 18.04-release amd64 Vector Packet Processing--runtime plugins - -**VPP Installed Packages - Centos** -:: - - $ rpm -qai vpp* - Name : vpp - Version : 18.04 - Release : release - Architecture: x86_64 - Install Date: Fri 04 May 2018 02:47:36 AM EDT - Group : Unspecified - Size : 2311324 - License : ASL 2.0 - Signature : (none) - Source RPM : vpp-18.04-release.src.rpm - Build Date : Wed 25 Apr 2018 01:00:24 PM EDT - Build Host : c29a51c12e88 - Relocations : (not relocatable) - Summary : Vector Packet Processing - Description : - This package provides VPP executables: vpp, vpp_api_test, vpp_json_test - vpp - the vector packet engine - vpp_api_test - vector packet engine API test tool - vpp_json_test - vector packet engine JSON test tool - Name : vpp-lib - Version : 18.04 - Release : release - Architecture: x86_64 - Install Date: Fri 04 May 2018 02:47:36 AM EDT - Group : System Environment/Libraries - Size : 24949034 - License : ASL 2.0 - Signature : (none) - Source RPM : vpp-18.04-release.src.rpm - Build Date : Wed 25 Apr 2018 01:00:24 PM EDT - Build Host : c29a51c12e88 - Relocations : (not relocatable) - Summary : VPP libraries - Description : - This package contains the VPP shared libraries, including: - vppinfra - foundation library supporting vectors, hashes, bitmaps, pools, and string formatting. - svm - vm library - vlib - vector processing library - vlib-api - binary API library - vnet - network stack library - Name : vpp-selinux-policy - Version : 18.04 - Release : release - Architecture: x86_64 - Install Date: Fri 04 May 2018 02:47:35 AM EDT - Group : System Environment/Base - Size : 86170 - License : ASL 2.0 - Signature : (none) - Source RPM : vpp-18.04-release.src.rpm - Build Date : Wed 25 Apr 2018 01:00:24 PM EDT - Build Host : c29a51c12e88 - Relocations : (not relocatable) - Summary : VPP Security-Enhanced Linux (SELinux) policy - Description : - This package contains a tailored VPP SELinux policy - Name : vpp-devel - Version : 18.04 - Release : release - Architecture: x86_64 - Install Date: Fri 04 May 2018 02:47:38 AM EDT - Group : Development/Libraries - Size : 10632218 - License : ASL 2.0 - Signature : (none) - Source RPM : vpp-18.04-release.src.rpm - Build Date : Wed 25 Apr 2018 01:00:24 PM EDT - Build Host : c29a51c12e88 - Relocations : (not relocatable) - Summary : VPP header files, static libraries - Description : - This package contains the header files for VPP. - Install this package if you want to write a - program for compilation and linking with vpp lib. - vlib - vlibmemory - vnet - devices, classify, dhcp, ethernet flow, gre, ip, etc. - vpp-api - vppinfra - Name : vpp-plugins - Version : 18.04 - Release : release - Architecture: x86_64 - Install Date: Fri 04 May 2018 02:47:37 AM EDT - Group : System Environment/Libraries - Size : 44145968 - License : ASL 2.0 - Signature : (none) - Source RPM : vpp-18.04-release.src.rpm - Build Date : Wed 25 Apr 2018 01:00:24 PM EDT - Build Host : c29a51c12e88 - Relocations : (not relocatable) - Summary : Vector Packet Processing--runtime plugins - Description : - This package contains VPP plugins - -**VPP Startup Configuration** - -VPP startup configuration is common for all test cases except test cases related -to SW Crypto device. - -**Default** - -:: - - $ cat /etc/vpp/startup.conf - unix { - nodaemon - log /var/log/vpp/vpp.log - full-coredump - cli-listen /run/vpp/cli.sock - gid vpp - } - - api-trace { - ## This stanza controls binary API tracing. Unless there is a very strong reason, - ## please leave this feature enabled. - on - ## Additional parameters: - ## - ## To set the number of binary API trace records in the circular buffer, configure nitems - ## - ## nitems - ## - ## To save the api message table decode tables, configure a filename. Results in /tmp/ - ## Very handy for understanding api message changes between versions, identifying missing - ## plugins, and so forth. - ## - ## save-api-table - } - - api-segment { - gid vpp - } - - cpu { - ## In the VPP there is one main thread and optionally the user can create worker(s) - ## The main thread and worker thread(s) can be pinned to CPU core(s) manually or automatically - - ## Manual pinning of thread(s) to CPU core(s) - - ## Set logical CPU core where main thread runs - # main-core 1 - - ## Set logical CPU core(s) where worker threads are running - # corelist-workers 2-3,18-19 - - ## Automatic pinning of thread(s) to CPU core(s) - - ## Sets number of CPU core(s) to be skipped (1 ... N-1) - ## Skipped CPU core(s) are not used for pinning main thread and working thread(s). - ## The main thread is automatically pinned to the first available CPU core and worker(s) - ## are pinned to next free CPU core(s) after core assigned to main thread - # skip-cores 4 - - ## Specify a number of workers to be created - ## Workers are pinned to N consecutive CPU cores while skipping "skip-cores" CPU core(s) - ## and main thread's CPU core - # workers 2 - - ## Set scheduling policy and priority of main and worker threads - - ## Scheduling policy options are: other (SCHED_OTHER), batch (SCHED_BATCH) - ## idle (SCHED_IDLE), fifo (SCHED_FIFO), rr (SCHED_RR) - # scheduler-policy fifo - - ## Scheduling priority is used only for "real-time policies (fifo and rr), - ## and has to be in the range of priorities supported for a particular policy - # scheduler-priority 50 - } - - # dpdk { - ## Change default settings for all intefaces - # dev default { - ## Number of receive queues, enables RSS - ## Default is 1 - # num-rx-queues 3 - - ## Number of transmit queues, Default is equal - ## to number of worker threads or 1 if no workers treads - # num-tx-queues 3 - - ## Number of descriptors in transmit and receive rings - ## increasing or reducing number can impact performance - ## Default is 1024 for both rx and tx - # num-rx-desc 512 - # num-tx-desc 512 - - ## VLAN strip offload mode for interface - ## Default is off - # vlan-strip-offload on - # } - - ## Whitelist specific interface by specifying PCI address - # dev 0000:02:00.0 - - ## Whitelist specific interface by specifying PCI address and in - ## addition specify custom parameters for this interface - # dev 0000:02:00.1 { - # num-rx-queues 2 - # } - - ## Specify bonded interface and its slaves via PCI addresses - ## - ## Bonded interface in XOR load balance mode (mode 2) with L3 and L4 headers - # vdev eth_bond0,mode=2,slave=0000:02:00.0,slave=0000:03:00.0,xmit_policy=l34 - # vdev eth_bond1,mode=2,slave=0000:02:00.1,slave=0000:03:00.1,xmit_policy=l34 - ## - ## Bonded interface in Active-Back up mode (mode 1) - # vdev eth_bond0,mode=1,slave=0000:02:00.0,slave=0000:03:00.0 - # vdev eth_bond1,mode=1,slave=0000:02:00.1,slave=0000:03:00.1 - - ## Change UIO driver used by VPP, Options are: igb_uio, vfio-pci, - ## uio_pci_generic or auto (default) - # uio-driver vfio-pci - - ## Disable mutli-segment buffers, improves performance but - ## disables Jumbo MTU support - # no-multi-seg - - ## Increase number of buffers allocated, needed only in scenarios with - ## large number of interfaces and worker threads. Value is per CPU socket. - ## Default is 16384 - # num-mbufs 128000 - - ## Change hugepages allocation per-socket, needed only if there is need for - ## larger number of mbufs. Default is 256M on each detected CPU socket - # socket-mem 2048,2048 - - ## Disables UDP / TCP TX checksum offload. Typically needed for use - ## faster vector PMDs (together with no-multi-seg) - # no-tx-checksum-offload - # } - - - # plugins { - ## Adjusting the plugin path depending on where the VPP plugins are - # path /home/bms/vpp/build-root/install-vpp-native/vpp/lib64/vpp_plugins - - ## Disable all plugins by default and then selectively enable specific plugins - # plugin default { disable } - # plugin dpdk_plugin.so { enable } - # plugin acl_plugin.so { enable } - - ## Enable all plugins by default and then selectively disable specific plugins - # plugin dpdk_plugin.so { disable } - # plugin acl_plugin.so { disable } - # } - - ## Alternate syntax to choose plugin path - # plugin_path /home/bms/vpp/build-root/install-vpp-native/vpp/lib64/vpp_plugins - -**SW Crypto Device** - -:: - - $ cat /etc/vpp/startup.conf - unix - { - cli-listen /run/vpp/cli.sock - gid vpp - nodaemon - full-coredump - log /tmp/vpp.log - } - api-segment - { - gid vpp - } - dpdk - { - vdev cryptodev_aesni_gcm_pmd,socket_id=0 - vdev cryptodev_aesni_mb_pmd,socket_id=0 - } - -TG Configuration ----------------- - -Traffic Generator node is VM running the same OS Linux as SUTs. Ports of this -VM are used as source (Tx) and destination (Rx) ports for the traffic. - -Traffic scripts of test cases are executed on this VM. - -**TG VM configuration** - -Configuration of the TG VMs is defined in `VIRL topologies directory`_. - - /csit/resources/tools/virl/topologies/double-ring-nested.xenial.virl - -- List of TG VM interfaces::: - - - - - - - - -**TG node port configuration** - -Port configuration of TG is defined in topology file that is generated per VIRL -simulation based on the definition stored in `VIRL topologies directory`_. - -Example of TG node configuration::: - - TG: - type: TG - host: "10.30.51.155" - port: 22 - username: cisco - priv_key: | - -----BEGIN RSA PRIVATE KEY----- - MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb - v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd - vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2 - 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc - 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs - 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7 - 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN - EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN - 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU - MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD - p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW - 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG - OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD - GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef - bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg - 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu - f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u - UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/ - +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT - D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/ - sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn - g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY - qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8 - EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD - BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN - -----END RSA PRIVATE KEY----- - - interfaces: - port3: - mac_address: "fa:16:3e:b9:e1:27" - pci_address: "0000:00:06.0" - link: link1 - driver: virtio-pci - port4: - mac_address: "fa:16:3e:e9:c8:68" - pci_address: "0000:00:07.0" - link: link4 - driver: virtio-pci - port5: - mac_address: "fa:16:3e:e8:d3:47" - pci_address: "0000:00:08.0" - link: link2 - driver: virtio-pci - port6: - mac_address: "fa:16:3e:cf:ca:58" - pci_address: "0000:00:09.0" - link: link5 - driver: virtio-pci - -**Traffic generator** - -Functional tests utilize Scapy as a traffic generator. There was used Scapy -v2.3.1 for |vpp-release| tests. - +Test Environment +================ + +CSIT functional tests are currently executed in FD.IO VIRL testbed. The physical +VIRL testbed infrastructure consists of three VIRL hosts: + +- All hosts are Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @2.30GHz, + 18c, 512GB RAM) + +- tb4-virl1: + + - Status: Production + - OS: Ubuntu 16.04.2 + - STD server version 0.10.32.16 + - UWM server version 0.10.32.16 + +- tb4-virl2: + + - Status: Production + - OS: Ubuntu 16.04.2 + - STD server version 0.10.32.16 + - UWM server version 0.10.32.16 + +- tb4-virl3: + + - Status: Testing + - OS: Ubuntu 16.04.2 + - STD server version 0.10.32.19 + - UWM server version 0.10.32.19 + +Whenever a patch is submitted to gerrit for review, parallel VIRL simulations +are started to reduce the time of execution of all functional tests. The number +of parallel VIRL simulations is equal to number of test groups defined by +TEST_GROUPS variable in :file:`csit/bootstrap.sh` file. The VIRL host to run +VIRL simulation is selected based on least load algorithm per VIRL simulation. + +Every VIRL simulation uses the same three-node - Traffic Generator (TG node) and +two Systems Under Test (SUT1 and SUT2) - "double-ring" topology. The appropriate +pre-built VPP packages built by Jenkins for the patch under review are then +installed on the two SUTs, along with their :file:`/etc/vpp/startup.conf` file, +in all VIRL simulations. + +SUT Configuration - VIRL Guest VM +--------------------------------- + +Configurations of the SUT VMs is defined in `VIRL topologies directory`_ + +- List of SUT VM interfaces::: + + + + + + +- Number of 2MB hugepages: 1024 + +- Maximum number of memory map areas: 20000 + +- Kernel Shared Memory Max: 2147483648 (vm.nr_hugepages * 2 * 1024 * 1024) + +SUT Configuration - VIRL Guest OS Linux +--------------------------------------- + +In CSIT terminology, the VM operating system for both SUTs that |vpp-release| has +been tested with, is the following: + +#. **Ubuntu VIRL image** + + This image implies Ubuntu 16.04.1 LTS, current as of yyyy-mm-dd (that is, + package versions are those that would have been installed by a + :command:`apt-get update`, :command:`apt-get upgrade` on that day), produced + by CSIT disk image build scripts. + + The exact list of installed packages and their versions (including the Linux + kernel package version) are included in `VIRL ubuntu images lists`_. + + A replica of this VM image can be built by running the :command:`build.sh` + script in CSIT repository. + +#. **CentOS VIRL image** + + This image implies Centos 7.4-1711, current as of yyyy-mm-dd (that is, + package versions are those that would have been installed by a + :command:`yum update`, :command:`yum upgrade` on that day), produced + by CSIT disk image build scripts. + + The exact list of installed packages and their versions (including the Linux + kernel package version) are included in `VIRL centos images lists`_. + + A replica of this VM image can be built by running the :command:`build.sh` + script in CSIT repository. + +#. **Nested VM image** + + In addition to the "main" VM image, tests which require VPP to communicate to + a VM over a vhost-user interface, utilize a "nested" VM image. + + This "nested" VM is dynamically created and destroyed as part of a test case, + and therefore the "nested" VM image is optimized to be small, lightweight and + have a short boot time. The "nested" VM image is not built around any + established Linux distribution, but is based on `BuildRoot + `_, a tool for building embedded Linux systems. Just + as for the "main" image, scripts to produce an identical replica of the + "nested" image are included in CSIT GIT repository, and the image can be + rebuilt using the "build.sh" script at `VIRL nested`_. + +DUT Configuration - VPP +----------------------- + +Every System Under Test runs VPP SW application in Linux user-mode as a Device +Under Test (DUT) node. + +**DUT port configuration** + +Port configuration of DUTs is defined in topology file that is generated per +VIRL simulation based on the definition stored in `VIRL topologies directory`_. + +Example of DUT nodes configuration::: + + DUT1: + type: DUT + host: "10.30.51.157" + port: 22 + username: cisco + honeycomb: + user: admin + passwd: admin + port: 8183 + netconf_port: 2831 + priv_key: | + -----BEGIN RSA PRIVATE KEY----- + MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb + v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd + vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2 + 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc + 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs + 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7 + 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN + EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN + 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU + MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD + p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW + 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG + OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD + GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef + bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg + 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu + f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u + UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/ + +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT + D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/ + sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn + g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY + qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8 + EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD + BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN + -----END RSA PRIVATE KEY----- + + interfaces: + port1: + mac_address: "fa:16:3e:9b:89:52" + pci_address: "0000:00:04.0" + link: link1 + port2: + mac_address: "fa:16:3e:7a:33:60" + pci_address: "0000:00:05.0" + link: link4 + port3: + mac_address: "fa:16:3e:29:b7:ae" + pci_address: "0000:00:06.0" + link: link3 + port4: + mac_address: "fa:16:3e:76:8d:ff" + pci_address: "0000:00:07.0" + link: link6 + DUT2: + type: DUT + host: "10.30.51.156" + port: 22 + username: cisco + honeycomb: + user: admin + passwd: admin + port: 8183 + netconf_port: 2831 + priv_key: | + -----BEGIN RSA PRIVATE KEY----- + MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb + v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd + vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2 + 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc + 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs + 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7 + 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN + EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN + 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU + MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD + p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW + 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG + OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD + GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef + bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg + 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu + f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u + UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/ + +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT + D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/ + sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn + g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY + qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8 + EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD + BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN + -----END RSA PRIVATE KEY----- + + interfaces: + port1: + mac_address: "fa:16:3e:ad:6c:7d" + pci_address: "0000:00:04.0" + link: link2 + port2: + mac_address: "fa:16:3e:94:a4:99" + pci_address: "0000:00:05.0" + link: link5 + port3: + mac_address: "fa:16:3e:75:92:da" + pci_address: "0000:00:06.0" + link: link3 + port4: + mac_address: "fa:16:3e:2c:b1:2a" + pci_address: "0000:00:07.0" + link: link6 + +**VPP Version** + +|vpp-release| + +**VPP Installed Packages - Ubuntu** +:: + + $ dpkg -l vpp\* + Desired=Unknown/Install/Remove/Purge/Hold + | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend + |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) + ||/ Name Version Architecture Description + +++-==============-=============-============-================================================= + ii vpp 18.04-release amd64 Vector Packet Processing--executables + ii vpp-dbg 18.04-release amd64 Vector Packet Processing--debug symbols + ii vpp-dev 18.04-release amd64 Vector Packet Processing--development support + ii vpp-dpdk-dkms 18.02-vpp1 amd64 DPDK Development Package for VPP - Kernel Modules + ii vpp-lib 18.04-release amd64 Vector Packet Processing--runtime libraries + ii vpp-plugins 18.04-release amd64 Vector Packet Processing--runtime plugins + +**VPP Installed Packages - Centos** +:: + + $ rpm -qai vpp* + Name : vpp + Version : 18.04 + Release : release + Architecture: x86_64 + Install Date: Fri 04 May 2018 02:47:36 AM EDT + Group : Unspecified + Size : 2311324 + License : ASL 2.0 + Signature : (none) + Source RPM : vpp-18.04-release.src.rpm + Build Date : Wed 25 Apr 2018 01:00:24 PM EDT + Build Host : c29a51c12e88 + Relocations : (not relocatable) + Summary : Vector Packet Processing + Description : + This package provides VPP executables: vpp, vpp_api_test, vpp_json_test + vpp - the vector packet engine + vpp_api_test - vector packet engine API test tool + vpp_json_test - vector packet engine JSON test tool + Name : vpp-lib + Version : 18.04 + Release : release + Architecture: x86_64 + Install Date: Fri 04 May 2018 02:47:36 AM EDT + Group : System Environment/Libraries + Size : 24949034 + License : ASL 2.0 + Signature : (none) + Source RPM : vpp-18.04-release.src.rpm + Build Date : Wed 25 Apr 2018 01:00:24 PM EDT + Build Host : c29a51c12e88 + Relocations : (not relocatable) + Summary : VPP libraries + Description : + This package contains the VPP shared libraries, including: + vppinfra - foundation library supporting vectors, hashes, bitmaps, pools, and string formatting. + svm - vm library + vlib - vector processing library + vlib-api - binary API library + vnet - network stack library + Name : vpp-selinux-policy + Version : 18.04 + Release : release + Architecture: x86_64 + Install Date: Fri 04 May 2018 02:47:35 AM EDT + Group : System Environment/Base + Size : 86170 + License : ASL 2.0 + Signature : (none) + Source RPM : vpp-18.04-release.src.rpm + Build Date : Wed 25 Apr 2018 01:00:24 PM EDT + Build Host : c29a51c12e88 + Relocations : (not relocatable) + Summary : VPP Security-Enhanced Linux (SELinux) policy + Description : + This package contains a tailored VPP SELinux policy + Name : vpp-devel + Version : 18.04 + Release : release + Architecture: x86_64 + Install Date: Fri 04 May 2018 02:47:38 AM EDT + Group : Development/Libraries + Size : 10632218 + License : ASL 2.0 + Signature : (none) + Source RPM : vpp-18.04-release.src.rpm + Build Date : Wed 25 Apr 2018 01:00:24 PM EDT + Build Host : c29a51c12e88 + Relocations : (not relocatable) + Summary : VPP header files, static libraries + Description : + This package contains the header files for VPP. + Install this package if you want to write a + program for compilation and linking with vpp lib. + vlib + vlibmemory + vnet - devices, classify, dhcp, ethernet flow, gre, ip, etc. + vpp-api + vppinfra + Name : vpp-plugins + Version : 18.04 + Release : release + Architecture: x86_64 + Install Date: Fri 04 May 2018 02:47:37 AM EDT + Group : System Environment/Libraries + Size : 44145968 + License : ASL 2.0 + Signature : (none) + Source RPM : vpp-18.04-release.src.rpm + Build Date : Wed 25 Apr 2018 01:00:24 PM EDT + Build Host : c29a51c12e88 + Relocations : (not relocatable) + Summary : Vector Packet Processing--runtime plugins + Description : + This package contains VPP plugins + +**VPP Startup Configuration** + +VPP startup configuration is common for all test cases except test cases related +to SW Crypto device. + +**Default** + +:: + + $ cat /etc/vpp/startup.conf + unix { + nodaemon + log /var/log/vpp/vpp.log + full-coredump + cli-listen /run/vpp/cli.sock + gid vpp + } + + api-trace { + ## This stanza controls binary API tracing. Unless there is a very strong reason, + ## please leave this feature enabled. + on + ## Additional parameters: + ## + ## To set the number of binary API trace records in the circular buffer, configure nitems + ## + ## nitems + ## + ## To save the api message table decode tables, configure a filename. Results in /tmp/ + ## Very handy for understanding api message changes between versions, identifying missing + ## plugins, and so forth. + ## + ## save-api-table + } + + api-segment { + gid vpp + } + + cpu { + ## In the VPP there is one main thread and optionally the user can create worker(s) + ## The main thread and worker thread(s) can be pinned to CPU core(s) manually or automatically + + ## Manual pinning of thread(s) to CPU core(s) + + ## Set logical CPU core where main thread runs + # main-core 1 + + ## Set logical CPU core(s) where worker threads are running + # corelist-workers 2-3,18-19 + + ## Automatic pinning of thread(s) to CPU core(s) + + ## Sets number of CPU core(s) to be skipped (1 ... N-1) + ## Skipped CPU core(s) are not used for pinning main thread and working thread(s). + ## The main thread is automatically pinned to the first available CPU core and worker(s) + ## are pinned to next free CPU core(s) after core assigned to main thread + # skip-cores 4 + + ## Specify a number of workers to be created + ## Workers are pinned to N consecutive CPU cores while skipping "skip-cores" CPU core(s) + ## and main thread's CPU core + # workers 2 + + ## Set scheduling policy and priority of main and worker threads + + ## Scheduling policy options are: other (SCHED_OTHER), batch (SCHED_BATCH) + ## idle (SCHED_IDLE), fifo (SCHED_FIFO), rr (SCHED_RR) + # scheduler-policy fifo + + ## Scheduling priority is used only for "real-time policies (fifo and rr), + ## and has to be in the range of priorities supported for a particular policy + # scheduler-priority 50 + } + + # dpdk { + ## Change default settings for all intefaces + # dev default { + ## Number of receive queues, enables RSS + ## Default is 1 + # num-rx-queues 3 + + ## Number of transmit queues, Default is equal + ## to number of worker threads or 1 if no workers treads + # num-tx-queues 3 + + ## Number of descriptors in transmit and receive rings + ## increasing or reducing number can impact performance + ## Default is 1024 for both rx and tx + # num-rx-desc 512 + # num-tx-desc 512 + + ## VLAN strip offload mode for interface + ## Default is off + # vlan-strip-offload on + # } + + ## Whitelist specific interface by specifying PCI address + # dev 0000:02:00.0 + + ## Whitelist specific interface by specifying PCI address and in + ## addition specify custom parameters for this interface + # dev 0000:02:00.1 { + # num-rx-queues 2 + # } + + ## Specify bonded interface and its slaves via PCI addresses + ## + ## Bonded interface in XOR load balance mode (mode 2) with L3 and L4 headers + # vdev eth_bond0,mode=2,slave=0000:02:00.0,slave=0000:03:00.0,xmit_policy=l34 + # vdev eth_bond1,mode=2,slave=0000:02:00.1,slave=0000:03:00.1,xmit_policy=l34 + ## + ## Bonded interface in Active-Back up mode (mode 1) + # vdev eth_bond0,mode=1,slave=0000:02:00.0,slave=0000:03:00.0 + # vdev eth_bond1,mode=1,slave=0000:02:00.1,slave=0000:03:00.1 + + ## Change UIO driver used by VPP, Options are: igb_uio, vfio-pci, + ## uio_pci_generic or auto (default) + # uio-driver vfio-pci + + ## Disable mutli-segment buffers, improves performance but + ## disables Jumbo MTU support + # no-multi-seg + + ## Increase number of buffers allocated, needed only in scenarios with + ## large number of interfaces and worker threads. Value is per CPU socket. + ## Default is 16384 + # num-mbufs 128000 + + ## Change hugepages allocation per-socket, needed only if there is need for + ## larger number of mbufs. Default is 256M on each detected CPU socket + # socket-mem 2048,2048 + + ## Disables UDP / TCP TX checksum offload. Typically needed for use + ## faster vector PMDs (together with no-multi-seg) + # no-tx-checksum-offload + # } + + + # plugins { + ## Adjusting the plugin path depending on where the VPP plugins are + # path /home/bms/vpp/build-root/install-vpp-native/vpp/lib64/vpp_plugins + + ## Disable all plugins by default and then selectively enable specific plugins + # plugin default { disable } + # plugin dpdk_plugin.so { enable } + # plugin acl_plugin.so { enable } + + ## Enable all plugins by default and then selectively disable specific plugins + # plugin dpdk_plugin.so { disable } + # plugin acl_plugin.so { disable } + # } + + ## Alternate syntax to choose plugin path + # plugin_path /home/bms/vpp/build-root/install-vpp-native/vpp/lib64/vpp_plugins + +**SW Crypto Device** + +:: + + $ cat /etc/vpp/startup.conf + unix + { + cli-listen /run/vpp/cli.sock + gid vpp + nodaemon + full-coredump + log /tmp/vpp.log + } + api-segment + { + gid vpp + } + dpdk + { + vdev cryptodev_aesni_gcm_pmd,socket_id=0 + vdev cryptodev_aesni_mb_pmd,socket_id=0 + } + +TG Configuration +---------------- + +Traffic Generator node is VM running the same OS Linux as SUTs. Ports of this +VM are used as source (Tx) and destination (Rx) ports for the traffic. + +Traffic scripts of test cases are executed on this VM. + +**TG VM configuration** + +Configuration of the TG VMs is defined in `VIRL topologies directory`_. + + /csit/resources/tools/virl/topologies/double-ring-nested.xenial.virl + +- List of TG VM interfaces::: + + + + + + + + +**TG node port configuration** + +Port configuration of TG is defined in topology file that is generated per VIRL +simulation based on the definition stored in `VIRL topologies directory`_. + +Example of TG node configuration::: + + TG: + type: TG + host: "10.30.51.155" + port: 22 + username: cisco + priv_key: | + -----BEGIN RSA PRIVATE KEY----- + MIIEpgIBAAKCAQEAwUDlTpzSHpwLQotZOFS4AgcPNEWCnP1AB2hWFmvI+8Kah/gb + v8ruZU9RqhPs56tyKzxbhvNkY4VbH5F1GilHZu3mLqzM4KfghMmaeMEjO1T7BYYd + vuBfTvIluljfQ2vAlnYrDwn+ClxJk81m0pDgvrLEX4qVVh2sGh7UEkYy5r82DNa2 + 4VjzPB1J/c8a9zP8FoZUhYIzF4FLvRMjUADpbMXgJMsGpaZLmz95ap0Eot7vb1Cc + 1LvF97iyBCrtIOSKRKA50ZhLGjMKmOwnYU+cP5718tbproDVi6VJOo7zeuXyetMs + 8YBl9kWblWG9BqP9jctFvsmi5G7hXgq1Y8u+DwIDAQABAoIBAQC/W4E0DHjLMny7 + 0bvw2YKzD0Zw3fttdB94tkm4PdZv5MybooPnsAvLaXVV0hEdfVi5kzSWNl/LY/tN + EP1BgGphc2QgB59/PPxGwFIjDCvUzlsZpynBHe+B/qh5ExNQcVvsIOqWI7DXlXaN + 0i/khOzmJ6HncRRah1spKimYRsaUUDskyg7q3QqMWVaqBbbMvLs/w7ZWd/zoDqCU + MY/pCI6hkB3QbRo0OdiZLohphBl2ShABTwjvVyyKL5UA4jAEneJrhH5gWVLXnfgD + p62W5CollKEYblC8mUkPxpP7Qo277zw3xaq+oktIZhc5SUEUd7nJZtNqVAHqkItW + 79VmpKyxAoGBAPfU+kqNPaTSvp+x1n5sn2SgipzDtgi9QqNmC4cjtrQQaaqI57SG + OHw1jX8i7L2G1WvVtkHg060nlEVo5n65ffFOqeVBezLVJ7ghWI8U+oBiJJyQ4boD + GJVNsoOSUQ0rtuGd9eVwfDk3ol9aCN0KK53oPfIYli29pyu4l095kg11AoGBAMef + bPEMBI/2XmCPshLSwhGFl+dW8d+Klluj3CUQ/0vUlvma3dfBOYNsIwAgTP0iIUTg + 8DYE6KBCdPtxAUEI0YAEAKB9ry1tKR2NQEIPfslYytKErtwjAiqSi0heM6+zwEzu + f54Z4oBhsMSL0jXoOMnu+NZzEc6EUdQeY4O+jhjzAoGBAIogC3dtjMPGKTP7+93u + UE/XIioI8fWg9fj3sMka4IMu+pVvRCRbAjRH7JrFLkjbUyuMqs3Arnk9K+gbdQt/ + +m95Njtt6WoFXuPCwgbM3GidSmZwYT4454SfDzVBYScEDCNm1FuR+8ov9bFLDtGT + D4gsngnGJj1MDFXTxZEn4nzZAoGBAKCg4WmpUPaCuXibyB+rZavxwsTNSn2lJ83/ + sYJGBhf/raiV/FLDUcM1vYg5dZnu37RsB/5/vqxOLZGyYd7x+Jo5HkQGPnKgNwhn + g8BkdZIRF8uEJqxOo0ycdOU7n/2O93swIpKWo5LIiRPuqqzj+uZKnAL7vuVdxfaY + qVz2daMPAoGBALgaaKa3voU/HO1PYLWIhFrBThyJ+BQSQ8OqrEzC8AnegWFxRAM8 + EqrzZXl7ACUuo1dH0Eipm41j2+BZWlQjiUgq5uj8+yzy+EU1ZRRyJcOKzbDACeuD + BpWWSXGBI5G4CppeYLjMUHZpJYeX1USULJQd2c4crLJKb76E8gz3Z9kN + -----END RSA PRIVATE KEY----- + + interfaces: + port3: + mac_address: "fa:16:3e:b9:e1:27" + pci_address: "0000:00:06.0" + link: link1 + driver: virtio-pci + port4: + mac_address: "fa:16:3e:e9:c8:68" + pci_address: "0000:00:07.0" + link: link4 + driver: virtio-pci + port5: + mac_address: "fa:16:3e:e8:d3:47" + pci_address: "0000:00:08.0" + link: link2 + driver: virtio-pci + port6: + mac_address: "fa:16:3e:cf:ca:58" + pci_address: "0000:00:09.0" + link: link5 + driver: virtio-pci + +**Traffic generator** + +Functional tests utilize Scapy as a traffic generator. There was used Scapy +v2.3.1 for |vpp-release| tests. + diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 377bd30d3f..0309885c0d 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -1,150 +1,150 @@ -CSIT Release Notes -================== - -Changes in CSIT |release| -------------------------- - -#. **VPP performance tests** - - - *MRR tests* - New Maximum Receive Rate tests measure the packet - forwarding rate under the maximum load offered by traffic - generator over a set trial duration, regardless of packet loss. - MRR tests are used for continuous performance trending and for - comparison between releases. - - - *Service Chaining with SRv6* - New SRv6 (Segment Routing IPv6) proxy - tests measure performance of SRv6 Endpoint fronting SR-unaware - appliance via masquerading (End.AM), dynamic proxy (End.AD) or - static proxy (End.AS) SR functions. - -#. **Presentation and Analytics Layer** - - - *Performance trending* - Added continuous performance trending and - analysis. New Performance Trending and Performance Analysis jobs - executed regular throughput tests, with results being subsequently - analysed and trend and anomalies summarized and presented in VPP - Performance Dashboard and trendline graphs. - -#. **Test Framework Optimizations** - - - *Performance tests efficiency* - Qemu build/install optimizations, - warmup phase handling, vpp restart handling. Resulted in improved - stability and reduced total execution time by 30% for single pkt - size e.g. 64B/78B. - - - *General code housekeeping* - ongoing RF keywords optimizations, - removal of redundant RF keywords. - -Performance Changes -------------------- - -Relative performance changes in measured packet throughput in CSIT -|release| are calculated against the results from CSIT |release-1| -report. Listed mean and standard deviation values are computed based on -a series of the same tests executed against respective VPP releases to -verify test results repeatibility, with percentage change calculated for -mean values. Note that the standard deviation is quite high for a small -number of packet throughput tests, what indicates poor test results -repeatability and makes the relative change of mean throughput value not -fully representative for these tests. The root causes behind poor -results repeatibility vary between the test cases. - -NDR Changes -~~~~~~~~~~~ - -NDR small packet throughput changes between releases are available in a -CSV and pretty ASCII formats: - - - `csv format for 1t1c <../_static/vpp/performance-changes-ndr-1t1c-full.csv>`_, - - `csv format for 2t2c <../_static/vpp/performance-changes-ndr-2t2c-full.csv>`_, - - `pretty ASCII format for 1t1c <../_static/vpp/performance-changes-ndr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../_static/vpp/performance-changes-ndr-2t2c-full.txt>`_. - -.. note:: - - Test results have been generated by - `FD.io test executor vpp performance jobs`_ with Robot Framework result - files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. - -PDR Changes -~~~~~~~~~~~ - -NDR small packet throughput changes between releases are available in a -CSV and pretty ASCII formats: - - - `csv format for 1t1c <../_static/vpp/performance-changes-pdr-1t1c-full.csv>`_, - - `csv format for 2t2c <../_static/vpp/performance-changes-pdr-2t2c-full.csv>`_, - - `pretty ASCII format for 1t1c <../_static/vpp/performance-changes-pdr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../_static/vpp/performance-changes-pdr-2t2c-full.txt>`_. - -.. note:: - - Test results have been generated by - `FD.io test executor vpp performance jobs`_ with Robot Framework result - files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. - -MRR Changes -~~~~~~~~~~~ - -MRR small packet throughput changes between releases are available in a -CSV and pretty ASCII formats: - - - `csv format for 1t1c <../_static/vpp/performance-changes-mrr-1t1c-full.csv>`_, - - `csv format for 2t2c <../_static/vpp/performance-changes-mrr-2t2c-full.csv>`_, - - `csv format for 4t4c <../_static/vpp/performance-changes-mrr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../_static/vpp/performance-changes-mrr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../_static/vpp/performance-changes-mrr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../_static/vpp/performance-changes-mrr-4t4c-full.txt>`_. - -.. note:: - - Test results have been generated by - `FD.io test executor vpp mrr jobs `_ - with Robot Framework result files csit-vpp-perf-mrr-daily-master__*__output.xml.gz - `archived here `_. - -Throughput Trending -------------------- - -In addition to reporting throughput changes between VPP releases, CSIT -provides continuous performance trending for VPP master branch: - -#. `VPP Performance Dashboard `_ - - per VPP test case throughput trend, trend compliance and summary of - detected anomalies. - -#. `Trending Methodology `_ - - throughput test metrics, trend calculations and anomaly - classification (progression, regression, outlier). - -#. `Trendline Graphs `_ - - per VPP build MRR throughput measurements against the trendline - with anomaly highlights, with associated CSIT test jobs. - -Known Issues ------------- - -List of known issues in CSIT |release| for VPP performance tests: - -+---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| # | Issue | Jira ID | Description | -+===+=================================================+============+=================================================================+ -| 1 | Sporadic (1 in 200) NDR discovery test failures | CSIT-570 | DPDK reporting rx-errors, indicating L1 issue. Suspected issue | -| | on x520. | | with HW combination of X710-X520 in LF testbeds. Not observed | -| | | | outside of LF testbeds. | -+---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| 2 | Lower than expected NDR throughput of DPDK | CSIT-571 | Suspected NIC firmware or DPDK driver issue affecting NDR and | -| | testpmd and VPP L2 path NDR throughput with | | PDR throughput on XL710 and X710 NICs. | -| | xl710 and x710 NICs, compared to x520 NICs. | | | -+---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| 3 | Tagged Ethernet dot1q and dot1ad L2 path | CSIT-1066 | Tagged Ethernet dot1q and dot1ad L2 path throughput regression: | -| | throughput regression. | | NDR -2%..-5%, PDR -2%..-6%, MRR. Affects l2xc and l2bd | -| | | | performance tests. | -+---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| 4 | IPSec (software, no QAT HW) throughput | CSIT-1064 | IPSec throughput regression: NDR -3%..-8%, PDR -2%..-8%, MRR | -| | regression. | | -3%..-7%. Affects IPSec SW tests, QAT HW tests not affected. | -+---+-------------------------------------------------+------------+-----------------------------------------------------------------+ -| 5 | High failure rate of creating working container | CSIT-1065 | About 20% of orchestrated container topology tests failing data | -| | topologies with K8s/Ligato orchestration. | | plane verification indicating configuration issue. Suspected | -| | | | issue with Ligato vpp-agent. | -+---+-------------------------------------------------+------------+-----------------------------------------------------------------+ +CSIT Release Notes +================== + +Changes in CSIT |release| +------------------------- + +#. **VPP performance tests** + + - *MRR tests* - New Maximum Receive Rate tests measure the packet + forwarding rate under the maximum load offered by traffic + generator over a set trial duration, regardless of packet loss. + MRR tests are used for continuous performance trending and for + comparison between releases. + + - *Service Chaining with SRv6* - New SRv6 (Segment Routing IPv6) proxy + tests measure performance of SRv6 Endpoint fronting SR-unaware + appliance via masquerading (End.AM), dynamic proxy (End.AD) or + static proxy (End.AS) SR functions. + +#. **Presentation and Analytics Layer** + + - *Performance trending* - Added continuous performance trending and + analysis. New Performance Trending and Performance Analysis jobs + executed regular throughput tests, with results being subsequently + analysed and trend and anomalies summarized and presented in VPP + Performance Dashboard and trendline graphs. + +#. **Test Framework Optimizations** + + - *Performance tests efficiency* - Qemu build/install optimizations, + warmup phase handling, vpp restart handling. Resulted in improved + stability and reduced total execution time by 30% for single pkt + size e.g. 64B/78B. + + - *General code housekeeping* - ongoing RF keywords optimizations, + removal of redundant RF keywords. + +Performance Changes +------------------- + +Relative performance changes in measured packet throughput in CSIT +|release| are calculated against the results from CSIT |release-1| +report. Listed mean and standard deviation values are computed based on +a series of the same tests executed against respective VPP releases to +verify test results repeatibility, with percentage change calculated for +mean values. Note that the standard deviation is quite high for a small +number of packet throughput tests, what indicates poor test results +repeatability and makes the relative change of mean throughput value not +fully representative for these tests. The root causes behind poor +results repeatibility vary between the test cases. + +NDR Changes +~~~~~~~~~~~ + +NDR small packet throughput changes between releases are available in a +CSV and pretty ASCII formats: + + - `csv format for 1t1c <../_static/vpp/performance-changes-ndr-1t1c-full.csv>`_, + - `csv format for 2t2c <../_static/vpp/performance-changes-ndr-2t2c-full.csv>`_, + - `pretty ASCII format for 1t1c <../_static/vpp/performance-changes-ndr-1t1c-full.txt>`_, + - `pretty ASCII format for 2t2c <../_static/vpp/performance-changes-ndr-2t2c-full.txt>`_. + +.. note:: + + Test results have been generated by + `FD.io test executor vpp performance jobs`_ with Robot Framework result + files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. + +PDR Changes +~~~~~~~~~~~ + +NDR small packet throughput changes between releases are available in a +CSV and pretty ASCII formats: + + - `csv format for 1t1c <../_static/vpp/performance-changes-pdr-1t1c-full.csv>`_, + - `csv format for 2t2c <../_static/vpp/performance-changes-pdr-2t2c-full.csv>`_, + - `pretty ASCII format for 1t1c <../_static/vpp/performance-changes-pdr-1t1c-full.txt>`_, + - `pretty ASCII format for 2t2c <../_static/vpp/performance-changes-pdr-2t2c-full.txt>`_. + +.. note:: + + Test results have been generated by + `FD.io test executor vpp performance jobs`_ with Robot Framework result + files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. + +MRR Changes +~~~~~~~~~~~ + +MRR small packet throughput changes between releases are available in a +CSV and pretty ASCII formats: + + - `csv format for 1t1c <../_static/vpp/performance-changes-mrr-1t1c-full.csv>`_, + - `csv format for 2t2c <../_static/vpp/performance-changes-mrr-2t2c-full.csv>`_, + - `csv format for 4t4c <../_static/vpp/performance-changes-mrr-4t4c-full.csv>`_, + - `pretty ASCII format for 1t1c <../_static/vpp/performance-changes-mrr-1t1c-full.txt>`_, + - `pretty ASCII format for 2t2c <../_static/vpp/performance-changes-mrr-2t2c-full.txt>`_, + - `pretty ASCII format for 4t4c <../_static/vpp/performance-changes-mrr-4t4c-full.txt>`_. + +.. note:: + + Test results have been generated by + `FD.io test executor vpp mrr jobs `_ + with Robot Framework result files csit-vpp-perf-mrr-daily-master__*__output.xml.gz + `archived here `_. + +Throughput Trending +------------------- + +In addition to reporting throughput changes between VPP releases, CSIT +provides continuous performance trending for VPP master branch: + +#. `VPP Performance Dashboard `_ + - per VPP test case throughput trend, trend compliance and summary of + detected anomalies. + +#. `Trending Methodology `_ + - throughput test metrics, trend calculations and anomaly + classification (progression, regression, outlier). + +#. `Trendline Graphs `_ + - per VPP build MRR throughput measurements against the trendline + with anomaly highlights, with associated CSIT test jobs. + +Known Issues +------------ + +List of known issues in CSIT |release| for VPP performance tests: + ++---+-------------------------------------------------+------------+-----------------------------------------------------------------+ +| # | Issue | Jira ID | Description | ++===+=================================================+============+=================================================================+ +| 1 | Sporadic (1 in 200) NDR discovery test failures | CSIT-570 | DPDK reporting rx-errors, indicating L1 issue. Suspected issue | +| | on x520. | | with HW combination of X710-X520 in LF testbeds. Not observed | +| | | | outside of LF testbeds. | ++---+-------------------------------------------------+------------+-----------------------------------------------------------------+ +| 2 | Lower than expected NDR throughput of DPDK | CSIT-571 | Suspected NIC firmware or DPDK driver issue affecting NDR and | +| | testpmd and VPP L2 path NDR throughput with | | PDR throughput on XL710 and X710 NICs. | +| | xl710 and x710 NICs, compared to x520 NICs. | | | ++---+-------------------------------------------------+------------+-----------------------------------------------------------------+ +| 3 | Tagged Ethernet dot1q and dot1ad L2 path | CSIT-1066 | Tagged Ethernet dot1q and dot1ad L2 path throughput regression: | +| | throughput regression. | | NDR -2%..-5%, PDR -2%..-6%, MRR. Affects l2xc and l2bd | +| | | | performance tests. | ++---+-------------------------------------------------+------------+-----------------------------------------------------------------+ +| 4 | IPSec (software, no QAT HW) throughput | CSIT-1064 | IPSec throughput regression: NDR -3%..-8%, PDR -2%..-8%, MRR | +| | regression. | | -3%..-7%. Affects IPSec SW tests, QAT HW tests not affected. | ++---+-------------------------------------------------+------------+-----------------------------------------------------------------+ +| 5 | High failure rate of creating working container | CSIT-1065 | About 20% of orchestrated container topology tests failing data | +| | topologies with K8s/Ligato orchestration. | | plane verification indicating configuration issue. Suspected | +| | | | issue with Ligato vpp-agent. | ++---+-------------------------------------------------+------------+-----------------------------------------------------------------+ diff --git a/docs/report/vpp_performance_tests/test_environment.rst b/docs/report/vpp_performance_tests/test_environment.rst index 77fe3216e4..91ca3bcc07 100644 --- a/docs/report/vpp_performance_tests/test_environment.rst +++ b/docs/report/vpp_performance_tests/test_environment.rst @@ -1,186 +1,186 @@ -.. include:: test_environment_intro.rst - -.. include:: test_environment_sut_conf_1.rst - -.. include:: test_environment_sut_conf_2.rst - -.. include:: test_environment_sut_conf_3.rst - - -DUT Configuration - VPP ------------------------ - -**VPP Version** - -|vpp-release| - -**VPP Compile Parameters** - -`FD.io VPP compile job`_ - -**VPP Install Parameters** - -:: - - $ dpkg -i --force-all vpp* - -**VPP Startup Configuration** - -VPP startup configuration changes per test case with different settings for CPU -cores, rx-queues and no-multi-seg parameter. Startup config is aligned with -applied test case tag: - -Tagged by **1T1C** - -:: - - ip - { - heap-size 4G - } - unix - { - cli-listen localhost:5002 - log /tmp/vpe.log - nodaemon - } - ip6 - { - heap-size 4G - hash-buckets 2000000 - } - heapsize 4G - plugins - { - plugin default - { - disable - } - plugin dpdk_plugin.so - { - enable - } - } - cpu - { - corelist-workers 2 - main-core 1 - } - dpdk - { - dev 0000:0a:00.0 - dev 0000:0a:00.1 - no-multi-seg - uio-driver uio_pci_generic - log-level debug - dev default - { - num-rx-queues 1 - } - socket-mem 1024,1024 - no-tx-checksum-offload - } - -Tagged by **2T2C** - -:: - - ip - { - heap-size 4G - } - unix - { - cli-listen localhost:5002 - log /tmp/vpe.log - nodaemon - } - ip6 - { - heap-size 4G - hash-buckets 2000000 - } - heapsize 4G - plugins - { - plugin default - { - disable - } - plugin dpdk_plugin.so - { - enable - } - } - cpu - { - corelist-workers 2,3 - main-core 1 - } - dpdk - { - dev 0000:0a:00.0 - dev 0000:0a:00.1 - no-multi-seg - uio-driver uio_pci_generic - log-level debug - dev default - { - num-rx-queues 1 - } - socket-mem 1024,1024 - no-tx-checksum-offload - } - -Tagged by **4T4C** - -:: - - ip - { - heap-size 4G - } - unix - { - cli-listen localhost:5002 - log /tmp/vpe.log - nodaemon - } - ip6 - { - heap-size 4G - hash-buckets 2000000 - } - heapsize 4G - plugins - { - plugin default - { - disable - } - plugin dpdk_plugin.so - { - enable - } - } - cpu - { - corelist-workers 2,3,4,5 - main-core 1 - } - dpdk - { - dev 0000:0a:00.0 - dev 0000:0a:00.1 - no-multi-seg - uio-driver uio_pci_generic - log-level debug - dev default - { - num-rx-queues 1 - } - socket-mem 1024,1024 - no-tx-checksum-offload - } - -.. include:: test_environment_tg.rst +.. include:: test_environment_intro.rst + +.. include:: test_environment_sut_conf_1.rst + +.. include:: test_environment_sut_conf_2.rst + +.. include:: test_environment_sut_conf_3.rst + + +DUT Configuration - VPP +----------------------- + +**VPP Version** + +|vpp-release| + +**VPP Compile Parameters** + +`FD.io VPP compile job`_ + +**VPP Install Parameters** + +:: + + $ dpkg -i --force-all vpp* + +**VPP Startup Configuration** + +VPP startup configuration changes per test case with different settings for CPU +cores, rx-queues and no-multi-seg parameter. Startup config is aligned with +applied test case tag: + +Tagged by **1T1C** + +:: + + ip + { + heap-size 4G + } + unix + { + cli-listen localhost:5002 + log /tmp/vpe.log + nodaemon + } + ip6 + { + heap-size 4G + hash-buckets 2000000 + } + heapsize 4G + plugins + { + plugin default + { + disable + } + plugin dpdk_plugin.so + { + enable + } + } + cpu + { + corelist-workers 2 + main-core 1 + } + dpdk + { + dev 0000:0a:00.0 + dev 0000:0a:00.1 + no-multi-seg + uio-driver uio_pci_generic + log-level debug + dev default + { + num-rx-queues 1 + } + socket-mem 1024,1024 + no-tx-checksum-offload + } + +Tagged by **2T2C** + +:: + + ip + { + heap-size 4G + } + unix + { + cli-listen localhost:5002 + log /tmp/vpe.log + nodaemon + } + ip6 + { + heap-size 4G + hash-buckets 2000000 + } + heapsize 4G + plugins + { + plugin default + { + disable + } + plugin dpdk_plugin.so + { + enable + } + } + cpu + { + corelist-workers 2,3 + main-core 1 + } + dpdk + { + dev 0000:0a:00.0 + dev 0000:0a:00.1 + no-multi-seg + uio-driver uio_pci_generic + log-level debug + dev default + { + num-rx-queues 1 + } + socket-mem 1024,1024 + no-tx-checksum-offload + } + +Tagged by **4T4C** + +:: + + ip + { + heap-size 4G + } + unix + { + cli-listen localhost:5002 + log /tmp/vpe.log + nodaemon + } + ip6 + { + heap-size 4G + hash-buckets 2000000 + } + heapsize 4G + plugins + { + plugin default + { + disable + } + plugin dpdk_plugin.so + { + enable + } + } + cpu + { + corelist-workers 2,3,4,5 + main-core 1 + } + dpdk + { + dev 0000:0a:00.0 + dev 0000:0a:00.1 + no-multi-seg + uio-driver uio_pci_generic + log-level debug + dev default + { + num-rx-queues 1 + } + socket-mem 1024,1024 + no-tx-checksum-offload + } + +.. include:: test_environment_tg.rst diff --git a/docs/report/vpp_unit_tests/documentation.rst b/docs/report/vpp_unit_tests/documentation.rst index 304db1121b..b6bc0272a8 100644 --- a/docs/report/vpp_unit_tests/documentation.rst +++ b/docs/report/vpp_unit_tests/documentation.rst @@ -1,6 +1,6 @@ -Documentation -============= - -For complete description of the VPP Test Framework including anatomy of a test -case and detailed documentation of existing VPP unit test cases please refer -to the `VPP test framework documentation`_. +Documentation +============= + +For complete description of the VPP Test Framework including anatomy of a test +case and detailed documentation of existing VPP unit test cases please refer +to the `VPP test framework documentation`_. diff --git a/docs/report/vpp_unit_tests/overview.rst b/docs/report/vpp_unit_tests/overview.rst index 9472680aa5..6165c71c50 100644 --- a/docs/report/vpp_unit_tests/overview.rst +++ b/docs/report/vpp_unit_tests/overview.rst @@ -1,87 +1,87 @@ -Overview -======== - -.. note:: - - This section includes an abbreviated version of the VPP Test Framework - overview maintained within the VPP project. Complete overview can be found - in `VPP test framework documentation`_. - -VPP Unit Test Framework ------------------------ - -VPP Test Framework is used to ease writing, running and debugging unit tests -for the VPP. It is based on python as a high level language to allow rapid -test development. scapy\_ is used as a tool for creating and dissecting -packets. - -VPP Test Framework does not send any packets to VPP directly. Traffic is -instead injected using VPP packet-generator interfaces. Packets are written -into a temporary .pcap file, which is then read by the VPP code with packets -getting injected into the VPP processing nodes. - -Similarly, VPP does not send any packets to VPP Test Framework directly. -Instead, VPP packet capture feature is used to capture and write packets to a -temporary .pcap file, which is then read and analyzed by the VPP Test -Framework. - -For complete description of the VPP Test Framework including anatomy of a test -case and detailed documentation of existing VPP unit test cases please refer -to the `VPP test framework documentation`_ - -Unit Tests Coverage -------------------- - -Following VPP functional test areas are covered in VPP unit test code included -in VPP rls1710 with results listed in this report: - -- ACL Security - stateful and stateless security-groups access-control-lists. -- APIs - VAPI, VOM, PAPI, JVPP. -- ARP - ARP, proxy ARP, static arp. -- BFD - API, Authentication, Authentication Change, CLI. -- BFD IPv4 - sessions operation. -- BFD IPv6 - sessions operation. -- BIER - Bit Indexed Explicit Replication. -- Classifier - classification with IP ACL, MAC ACL, IP PBR. -- Container Integration - IPv4, IPv6 local-spoof connectivity tests. -- CRUD Loopback - create, read, update, delete Loopback interfaces. -- DHCP - DHCPv4/v6 Client and Proxy. -- Distributed Virtual Router. -- DS-Lite Softwire - softwire termination. -- FIB - baseline and scale tests. -- Flowprobe. -- Geneve Tunnels. -- GRE Tunnels - GRE IPv4/IPv6 tunnel, L2, VRF tests. -- GTPU Tunnels - baseline GTPU tests. -- IP Multicast Routing - IPv4/IPv6 multicast replication, connected source check. -- IPSec - baseline IPSec sanity tests. -- IPv4 FIB CRUD - add/update/delete IPv4 routes. -- IPv4 Routing. -- IP4 VRF Multi-instance - create, read, update, delete and verify IPv4 VRFs. -- IPv6 Routing - baseline FIB operations, NS/RS exception handling. -- IP6 VRF Multi-instance - create, read, update, delete and verify IPv6 VRFs. -- IRB Integrated Routing-Bridging. -- Kube-proxy - data plane NAT tests. -- L2 FIB CRUD - add/update/delete L2 MAC entries. -- L2BD Multi-instance. -- L2BD Switching - L2 Bridge-Domain baseline tests incl. single- and dual-loop. -- L2XC Multi-instance - L2 cross-connect multi-instance tests. -- L2XC Switching - L2 cross-connect baseline tests incl. single- and dual-loop. -- LISP Tunnels - basic LISP tests. -- Load Balancer - IP4 GRE4, IP4 GRE6, IP6 GRE4, IP6 GRE6. -- MACIP Access Control - ingress access control for IPv4, IPv6 with L2BDP and IP routing. -- MAP Softwires - softwire termination. -- MFIB Multicast FIB. -- MPLS Switching - MPLS baseline, prefix independent convergence for MPLS PE. -- NAT44 - NAT44 tests, IPFIX logging, VRF awareness, deterministic CGNAT. -- NAT64 - NAT64 static and dynamic translation tests. -- P2P Ethernet Subinterface. -- PPPoE Encapsulation. -- SPAN Switch Port Analyzer - packet mirroring. -- SRv6 Routing - Segment Routing IPv6 tests. -- TCP/IP Stack - unit tests, builtin client/server transfers. -- UDP Stack - unit tests. -- VTR VLAN Tag Rewrites - VLAN tag rewrite tests. -- VXLAN Tunnels - baseline VXLAN tests including multicast. -- VXLAN-GPE Tunnels - baseline VXLAN-GPE tunneling including multicast. -- Other Tests - ping, session, template verification, timer tests. +Overview +======== + +.. note:: + + This section includes an abbreviated version of the VPP Test Framework + overview maintained within the VPP project. Complete overview can be found + in `VPP test framework documentation`_. + +VPP Unit Test Framework +----------------------- + +VPP Test Framework is used to ease writing, running and debugging unit tests +for the VPP. It is based on python as a high level language to allow rapid +test development. scapy\_ is used as a tool for creating and dissecting +packets. + +VPP Test Framework does not send any packets to VPP directly. Traffic is +instead injected using VPP packet-generator interfaces. Packets are written +into a temporary .pcap file, which is then read by the VPP code with packets +getting injected into the VPP processing nodes. + +Similarly, VPP does not send any packets to VPP Test Framework directly. +Instead, VPP packet capture feature is used to capture and write packets to a +temporary .pcap file, which is then read and analyzed by the VPP Test +Framework. + +For complete description of the VPP Test Framework including anatomy of a test +case and detailed documentation of existing VPP unit test cases please refer +to the `VPP test framework documentation`_ + +Unit Tests Coverage +------------------- + +Following VPP functional test areas are covered in VPP unit test code included +in VPP rls1710 with results listed in this report: + +- ACL Security - stateful and stateless security-groups access-control-lists. +- APIs - VAPI, VOM, PAPI, JVPP. +- ARP - ARP, proxy ARP, static arp. +- BFD - API, Authentication, Authentication Change, CLI. +- BFD IPv4 - sessions operation. +- BFD IPv6 - sessions operation. +- BIER - Bit Indexed Explicit Replication. +- Classifier - classification with IP ACL, MAC ACL, IP PBR. +- Container Integration - IPv4, IPv6 local-spoof connectivity tests. +- CRUD Loopback - create, read, update, delete Loopback interfaces. +- DHCP - DHCPv4/v6 Client and Proxy. +- Distributed Virtual Router. +- DS-Lite Softwire - softwire termination. +- FIB - baseline and scale tests. +- Flowprobe. +- Geneve Tunnels. +- GRE Tunnels - GRE IPv4/IPv6 tunnel, L2, VRF tests. +- GTPU Tunnels - baseline GTPU tests. +- IP Multicast Routing - IPv4/IPv6 multicast replication, connected source check. +- IPSec - baseline IPSec sanity tests. +- IPv4 FIB CRUD - add/update/delete IPv4 routes. +- IPv4 Routing. +- IP4 VRF Multi-instance - create, read, update, delete and verify IPv4 VRFs. +- IPv6 Routing - baseline FIB operations, NS/RS exception handling. +- IP6 VRF Multi-instance - create, read, update, delete and verify IPv6 VRFs. +- IRB Integrated Routing-Bridging. +- Kube-proxy - data plane NAT tests. +- L2 FIB CRUD - add/update/delete L2 MAC entries. +- L2BD Multi-instance. +- L2BD Switching - L2 Bridge-Domain baseline tests incl. single- and dual-loop. +- L2XC Multi-instance - L2 cross-connect multi-instance tests. +- L2XC Switching - L2 cross-connect baseline tests incl. single- and dual-loop. +- LISP Tunnels - basic LISP tests. +- Load Balancer - IP4 GRE4, IP4 GRE6, IP6 GRE4, IP6 GRE6. +- MACIP Access Control - ingress access control for IPv4, IPv6 with L2BDP and IP routing. +- MAP Softwires - softwire termination. +- MFIB Multicast FIB. +- MPLS Switching - MPLS baseline, prefix independent convergence for MPLS PE. +- NAT44 - NAT44 tests, IPFIX logging, VRF awareness, deterministic CGNAT. +- NAT64 - NAT64 static and dynamic translation tests. +- P2P Ethernet Subinterface. +- PPPoE Encapsulation. +- SPAN Switch Port Analyzer - packet mirroring. +- SRv6 Routing - Segment Routing IPv6 tests. +- TCP/IP Stack - unit tests, builtin client/server transfers. +- UDP Stack - unit tests. +- VTR VLAN Tag Rewrites - VLAN tag rewrite tests. +- VXLAN Tunnels - baseline VXLAN tests including multicast. +- VXLAN-GPE Tunnels - baseline VXLAN-GPE tunneling including multicast. +- Other Tests - ping, session, template verification, timer tests. diff --git a/resources/templates/vat/memif_create.vat b/resources/templates/vat/memif_create.vat index 1e9d0e8a6a..b20b3d7c52 100644 --- a/resources/templates/vat/memif_create.vat +++ b/resources/templates/vat/memif_create.vat @@ -1 +1 @@ -memif_create id {id} socket-id {socket} {role} rx-queues {rxq} tx-queues {txq} +memif_create id {id} socket-id {socket} {role} rx-queues {rxq} tx-queues {txq} diff --git a/resources/templates/vat/nat/nat44_deterministic_forward.vat b/resources/templates/vat/nat/nat44_deterministic_forward.vat index a901398598..bcfcce961a 100644 --- a/resources/templates/vat/nat/nat44_deterministic_forward.vat +++ b/resources/templates/vat/nat/nat44_deterministic_forward.vat @@ -1 +1 @@ -exec nat44 deterministic forward {ip} +exec nat44 deterministic forward {ip} diff --git a/resources/templates/vat/nat/nat44_deterministic_reverse.vat b/resources/templates/vat/nat/nat44_deterministic_reverse.vat index 614a2c0dfb..04ba8ba6dc 100644 --- a/resources/templates/vat/nat/nat44_deterministic_reverse.vat +++ b/resources/templates/vat/nat/nat44_deterministic_reverse.vat @@ -1 +1 @@ -exec nat44 deterministic reverse {ip}:{port} +exec nat44 deterministic reverse {ip}:{port} diff --git a/resources/templates/vat/nat/nat44_set_deterministic.vat b/resources/templates/vat/nat/nat44_set_deterministic.vat index 39f3deeeb2..6141cce07c 100644 --- a/resources/templates/vat/nat/nat44_set_deterministic.vat +++ b/resources/templates/vat/nat/nat44_set_deterministic.vat @@ -1 +1 @@ -exec nat44 deterministic add in {ip_in}/{subnet_in} out {ip_out}/{subnet_out} +exec nat44 deterministic add in {ip_in}/{subnet_in} out {ip_out}/{subnet_out} diff --git a/resources/templates/vat/nat/nat44_set_interfaces.vat b/resources/templates/vat/nat/nat44_set_interfaces.vat index f25961dbc2..3dab50b79e 100644 --- a/resources/templates/vat/nat/nat44_set_interfaces.vat +++ b/resources/templates/vat/nat/nat44_set_interfaces.vat @@ -1 +1 @@ -exec set interface nat44 in {int_in} out {int_out} +exec set interface nat44 in {int_in} out {int_out} diff --git a/resources/templates/vat/nat/nat_set_workers.vat b/resources/templates/vat/nat/nat_set_workers.vat index 1a4d48979d..99fe68e4a2 100644 --- a/resources/templates/vat/nat/nat_set_workers.vat +++ b/resources/templates/vat/nat/nat_set_workers.vat @@ -1 +1 @@ -exec set nat workers {lcores} +exec set nat workers {lcores} diff --git a/resources/templates/vat/nat/nat_show_nat.vat b/resources/templates/vat/nat/nat_show_nat.vat index 52f723280d..4fe01cb211 100644 --- a/resources/templates/vat/nat/nat_show_nat.vat +++ b/resources/templates/vat/nat/nat_show_nat.vat @@ -1 +1 @@ -exec show nat verbose +exec show nat verbose diff --git a/resources/templates/vat/nat/snat_interface_dump.vat b/resources/templates/vat/nat/snat_interface_dump.vat index b80db63d82..37bc9d1853 100644 --- a/resources/templates/vat/nat/snat_interface_dump.vat +++ b/resources/templates/vat/nat/snat_interface_dump.vat @@ -1 +1 @@ -snat_interface_dump +snat_interface_dump diff --git a/resources/templates/vat/show_memif.vat b/resources/templates/vat/show_memif.vat index 9c64260b11..4b6c14a8df 100644 --- a/resources/templates/vat/show_memif.vat +++ b/resources/templates/vat/show_memif.vat @@ -1 +1 @@ -exec show memif +exec show memif diff --git a/resources/tools/testbed-setup/ansible/production b/resources/tools/testbed-setup/ansible/production index e1703afbcc..8e5ff62564 100644 --- a/resources/tools/testbed-setup/ansible/production +++ b/resources/tools/testbed-setup/ansible/production @@ -1,53 +1,53 @@ -all: - children: -# haswell: -# children: -# tg: -# hosts: -# 10.30.51.16: null #t1-tg1 -# 10.30.51.20: null #t2-tg1 -# 10.30.51.24: null #t3-tg1 -# sut: -# hosts: -# 10.30.51.17: null #t1-sut1 -# 10.30.51.18: null #t1-sut2 -# 10.30.51.21: null #t2-sut1 -# 10.30.51.22: null #t2-sut2 -# 10.30.51.25: null #t3-sut1 -# 10.30.51.26: null #t3-sut2 -# virl: # WARNING, DO NOT ENABLE VIRL UNLESS YOU KNOW WHAT YOU ARE DOING -# hosts: -# 10.30.51.28: null #t4-virl1 -# 10.30.51.29: null #t4-virl2 -# 10.30.51.30: null #t4-virl3 - skylake: - children: - tg: - hosts: - 10.30.51.45: null #s4-t21-tg1 - 10.30.51.48: null #s13-t31-tg1 - 10.30.51.49: null #s17-t33-tg1 - 10.30.51.53: null #s6-t22-tg1 - 10.30.51.55: null #s8-t23-tg1 - 10.30.51.57: null #s10-t24-tg1 - 10.30.51.60: null #s16-t32-tg1 - sut: - hosts: - 10.30.51.44: null #s3-t21-sut1 - 10.30.51.46: null #s11-t31-sut1 - 10.30.51.47: null #s12-t31-sut2 - 10.30.51.52: null #s5-t22-sut1 - 10.30.51.54: null #s7-t23-sut1 - 10.30.51.56: null #s9-t24-sut1 - 10.30.51.58: null #s14-t32-sut1 - 10.30.51.59: null #s15-t32-sut2 - vppdevice: - hosts: - 10.30.51.50: null #s1-t11-sut1 - 10.30.51.51: null #s2-t12-sut1 -# arm: -# children: -# tg: -# hosts: -# sut: -# hosts: +all: + children: +# haswell: +# children: +# tg: +# hosts: +# 10.30.51.16: null #t1-tg1 +# 10.30.51.20: null #t2-tg1 +# 10.30.51.24: null #t3-tg1 +# sut: +# hosts: +# 10.30.51.17: null #t1-sut1 +# 10.30.51.18: null #t1-sut2 +# 10.30.51.21: null #t2-sut1 +# 10.30.51.22: null #t2-sut2 +# 10.30.51.25: null #t3-sut1 +# 10.30.51.26: null #t3-sut2 +# virl: # WARNING, DO NOT ENABLE VIRL UNLESS YOU KNOW WHAT YOU ARE DOING +# hosts: +# 10.30.51.28: null #t4-virl1 +# 10.30.51.29: null #t4-virl2 +# 10.30.51.30: null #t4-virl3 + skylake: + children: + tg: + hosts: + 10.30.51.45: null #s4-t21-tg1 + 10.30.51.48: null #s13-t31-tg1 + 10.30.51.49: null #s17-t33-tg1 + 10.30.51.53: null #s6-t22-tg1 + 10.30.51.55: null #s8-t23-tg1 + 10.30.51.57: null #s10-t24-tg1 + 10.30.51.60: null #s16-t32-tg1 + sut: + hosts: + 10.30.51.44: null #s3-t21-sut1 + 10.30.51.46: null #s11-t31-sut1 + 10.30.51.47: null #s12-t31-sut2 + 10.30.51.52: null #s5-t22-sut1 + 10.30.51.54: null #s7-t23-sut1 + 10.30.51.56: null #s9-t24-sut1 + 10.30.51.58: null #s14-t32-sut1 + 10.30.51.59: null #s15-t32-sut2 + vppdevice: + hosts: + 10.30.51.50: null #s1-t11-sut1 + 10.30.51.51: null #s2-t12-sut1 +# arm: +# children: +# tg: +# hosts: +# sut: +# hosts: diff --git a/resources/tools/testbed-setup/ansible/roles/common/tasks/main.yaml b/resources/tools/testbed-setup/ansible/roles/common/tasks/main.yaml index e16a6a7356..06058a427b 100644 --- a/resources/tools/testbed-setup/ansible/roles/common/tasks/main.yaml +++ b/resources/tools/testbed-setup/ansible/roles/common/tasks/main.yaml @@ -1,46 +1,46 @@ ---- -# file: roles/common/tasks/main.yaml - -- name: Ubuntu specific - import_tasks: ubuntu_x86_64.yaml - when: ansible_distribution|lower == 'ubuntu' and ansible_machine == 'x86_64' - -- name: Set hostname - hostname: - name: '{{ hostname }}' - tags: set-hostname - -- name: Ensure hostname is in /etc/hosts - lineinfile: - path: '/etc/hosts' - regexp: '^{{ ansible_default_ipv4.address }}.+$' - line: '{{ ansible_default_ipv4.address }} {{ hostname }}.linuxfoundation.org' - tags: set-hostname - -- name: Set sudoers - lineinfile: - path: '/etc/sudoers' - state: 'present' - regexp: '^%admin ALL=' - line: '%admin ALL=(ALL) ALL' - validate: '/usr/sbin/visudo -cf %s' - tags: set-sudoers - -- name: Set sudoers - lineinfile: - path: '/etc/sudoers' - state: 'present' - regexp: '^%sudo' - line: '%sudo ALL=(ALL:ALL) NOPASSWD: ALL' - validate: '/usr/sbin/visudo -cf %s' - tags: set-sudoers - -- name: Copy grub file - template: - src: 'files/grub' - dest: '/etc/default/grub' - owner: 'root' - group: 'root' - mode: '644' - notify: ['Update GRUB'] - tags: copy-grub +--- +# file: roles/common/tasks/main.yaml + +- name: Ubuntu specific + import_tasks: ubuntu_x86_64.yaml + when: ansible_distribution|lower == 'ubuntu' and ansible_machine == 'x86_64' + +- name: Set hostname + hostname: + name: '{{ hostname }}' + tags: set-hostname + +- name: Ensure hostname is in /etc/hosts + lineinfile: + path: '/etc/hosts' + regexp: '^{{ ansible_default_ipv4.address }}.+$' + line: '{{ ansible_default_ipv4.address }} {{ hostname }}.linuxfoundation.org' + tags: set-hostname + +- name: Set sudoers + lineinfile: + path: '/etc/sudoers' + state: 'present' + regexp: '^%admin ALL=' + line: '%admin ALL=(ALL) ALL' + validate: '/usr/sbin/visudo -cf %s' + tags: set-sudoers + +- name: Set sudoers + lineinfile: + path: '/etc/sudoers' + state: 'present' + regexp: '^%sudo' + line: '%sudo ALL=(ALL:ALL) NOPASSWD: ALL' + validate: '/usr/sbin/visudo -cf %s' + tags: set-sudoers + +- name: Copy grub file + template: + src: 'files/grub' + dest: '/etc/default/grub' + owner: 'root' + group: 'root' + mode: '644' + notify: ['Update GRUB'] + tags: copy-grub diff --git a/resources/tools/testbed-setup/ansible/roles/common/tasks/ubuntu_x86_64.yaml b/resources/tools/testbed-setup/ansible/roles/common/tasks/ubuntu_x86_64.yaml index d0f32a31cc..b6116c3690 100644 --- a/resources/tools/testbed-setup/ansible/roles/common/tasks/ubuntu_x86_64.yaml +++ b/resources/tools/testbed-setup/ansible/roles/common/tasks/ubuntu_x86_64.yaml @@ -1,50 +1,50 @@ ---- -# file: roles/common/tasks/ubuntu_x86_64.yaml - -- name: Copy apt sources file - template: - src: 'files/apt-sources.list' - dest: '/etc/apt/sources.list' - tags: copy-apt-sources - -- name: Install python-apt - apt: - name: 'python-apt' - state: 'present' - update_cache: True - tags: install-python-apt - -- name: Install git - apt: - name: 'git' - state: 'present' - update_cache: True - tags: install-git - -- name: Install crudini - apt: - name: 'crudini' - state: 'present' - update_cache: True - tags: install-crudini - -- name: Install expect - apt: - name: 'expect' - state: 'present' - update_cache: True - tags: install-expect - -- name: Install socat - apt: - name: 'socat' - state: 'present' - update_cache: True - tags: install-socat - -- name: Install qemu - apt: - name: 'qemu-system-x86' - state: 'present' - update_cache: True - tags: install-qemu +--- +# file: roles/common/tasks/ubuntu_x86_64.yaml + +- name: Copy apt sources file + template: + src: 'files/apt-sources.list' + dest: '/etc/apt/sources.list' + tags: copy-apt-sources + +- name: Install python-apt + apt: + name: 'python-apt' + state: 'present' + update_cache: True + tags: install-python-apt + +- name: Install git + apt: + name: 'git' + state: 'present' + update_cache: True + tags: install-git + +- name: Install crudini + apt: + name: 'crudini' + state: 'present' + update_cache: True + tags: install-crudini + +- name: Install expect + apt: + name: 'expect' + state: 'present' + update_cache: True + tags: install-expect + +- name: Install socat + apt: + name: 'socat' + state: 'present' + update_cache: True + tags: install-socat + +- name: Install qemu + apt: + name: 'qemu-system-x86' + state: 'present' + update_cache: True + tags: install-qemu diff --git a/resources/tools/testbed-setup/ansible/roles/sut/tasks/main.yaml b/resources/tools/testbed-setup/ansible/roles/sut/tasks/main.yaml index ffdbbcc860..0ecdcdba3b 100644 --- a/resources/tools/testbed-setup/ansible/roles/sut/tasks/main.yaml +++ b/resources/tools/testbed-setup/ansible/roles/sut/tasks/main.yaml @@ -1,14 +1,14 @@ ---- -# file: roles/sut/tasks/main.yaml - -- name: Ubuntu specific - import_tasks: ubuntu_x86_64.yaml - when: ansible_distribution|lower == 'ubuntu' and ansible_machine == 'x86_64' - -- name: Copy 80-vpp.conf - file: - src: '/dev/null' - dest: '/etc/sysctl.d/80-vpp.conf' - state: 'link' - become: yes - tags: copy-80-vpp +--- +# file: roles/sut/tasks/main.yaml + +- name: Ubuntu specific + import_tasks: ubuntu_x86_64.yaml + when: ansible_distribution|lower == 'ubuntu' and ansible_machine == 'x86_64' + +- name: Copy 80-vpp.conf + file: + src: '/dev/null' + dest: '/etc/sysctl.d/80-vpp.conf' + state: 'link' + become: yes + tags: copy-80-vpp diff --git a/resources/tools/testbed-setup/ansible/roles/sut/tasks/ubuntu_x86_64.yaml b/resources/tools/testbed-setup/ansible/roles/sut/tasks/ubuntu_x86_64.yaml index f9ab901ded..6c9a839730 100644 --- a/resources/tools/testbed-setup/ansible/roles/sut/tasks/ubuntu_x86_64.yaml +++ b/resources/tools/testbed-setup/ansible/roles/sut/tasks/ubuntu_x86_64.yaml @@ -1,90 +1,90 @@ ---- -# file: roles/sut/tasks/ubuntu_x86_64.yaml - -- name: Install DKMS - apt: - name: 'dkms' - state: 'present' - update_cache: True - become: yes - tags: install-dkms - -- name: Install pkg-config - apt: - name: 'pkg-config' - state: 'present' - update_cache: True - become: yes - tags: install-pkg-config - -- name: Install libglib2.0-dev - apt: - name: 'libglib2.0-dev' - state: 'present' - update_cache: True - become: yes - tags: install-libglib2.0-dev - -- name: Install autoconf - apt: - name: 'autoconf' - state: 'present' - update_cache: True - become: yes - tags: install-autoconf - -- name: Install libtool - apt: - name: 'libtool' - state: 'present' - update_cache: True - become: yes - tags: install-libtool - -- name: Install screen - apt: - name: 'screen' - state: 'present' - update_cache: True - become: yes - tags: install-screen - -- name: Install libmbedcrypto1 - apt: - name: 'libmbedcrypto1' - state: 'present' - update_cache: True - become: yes - tags: install-libmbedcrypto1 - -- name: Install libmbedtls10 - apt: - name: 'libmbedtls10' - state: 'present' - update_cache: True - become: yes - tags: install-libmbedtls10 - -- name: Install libmbedx509-0 - apt: - name: 'libmbedx509-0' - state: 'present' - update_cache: True - become: yes - tags: install-libmbedx509-0 - -- name: Install lxc - apt: - name: 'lxc' - state: 'present' - update_cache: True - become: yes - tags: install-lxc - -- name: Install java - apt: - name: 'openjdk-8-jdk' - state: 'present' - update_cache: True - become: yes - tags: install-java +--- +# file: roles/sut/tasks/ubuntu_x86_64.yaml + +- name: Install DKMS + apt: + name: 'dkms' + state: 'present' + update_cache: True + become: yes + tags: install-dkms + +- name: Install pkg-config + apt: + name: 'pkg-config' + state: 'present' + update_cache: True + become: yes + tags: install-pkg-config + +- name: Install libglib2.0-dev + apt: + name: 'libglib2.0-dev' + state: 'present' + update_cache: True + become: yes + tags: install-libglib2.0-dev + +- name: Install autoconf + apt: + name: 'autoconf' + state: 'present' + update_cache: True + become: yes + tags: install-autoconf + +- name: Install libtool + apt: + name: 'libtool' + state: 'present' + update_cache: True + become: yes + tags: install-libtool + +- name: Install screen + apt: + name: 'screen' + state: 'present' + update_cache: True + become: yes + tags: install-screen + +- name: Install libmbedcrypto1 + apt: + name: 'libmbedcrypto1' + state: 'present' + update_cache: True + become: yes + tags: install-libmbedcrypto1 + +- name: Install libmbedtls10 + apt: + name: 'libmbedtls10' + state: 'present' + update_cache: True + become: yes + tags: install-libmbedtls10 + +- name: Install libmbedx509-0 + apt: + name: 'libmbedx509-0' + state: 'present' + update_cache: True + become: yes + tags: install-libmbedx509-0 + +- name: Install lxc + apt: + name: 'lxc' + state: 'present' + update_cache: True + become: yes + tags: install-lxc + +- name: Install java + apt: + name: 'openjdk-8-jdk' + state: 'present' + update_cache: True + become: yes + tags: install-java diff --git a/resources/tools/testbed-setup/ansible/roles/tg/tasks/main.yaml b/resources/tools/testbed-setup/ansible/roles/tg/tasks/main.yaml index 8e71800bca..54001a7272 100644 --- a/resources/tools/testbed-setup/ansible/roles/tg/tasks/main.yaml +++ b/resources/tools/testbed-setup/ansible/roles/tg/tasks/main.yaml @@ -1,6 +1,6 @@ ---- -# file: roles/tg/tasks/main.yaml - -- name: Ubuntu specific - import_tasks: ubuntu_x86_64.yaml - when: ansible_distribution|lower == 'ubuntu' and ansible_machine == 'x86_64' +--- +# file: roles/tg/tasks/main.yaml + +- name: Ubuntu specific + import_tasks: ubuntu_x86_64.yaml + when: ansible_distribution|lower == 'ubuntu' and ansible_machine == 'x86_64' diff --git a/resources/tools/testbed-setup/ansible/roles/tg/tasks/ubuntu_x86_64.yaml b/resources/tools/testbed-setup/ansible/roles/tg/tasks/ubuntu_x86_64.yaml index 9711b2678d..54a04685e6 100644 --- a/resources/tools/testbed-setup/ansible/roles/tg/tasks/ubuntu_x86_64.yaml +++ b/resources/tools/testbed-setup/ansible/roles/tg/tasks/ubuntu_x86_64.yaml @@ -1,9 +1,9 @@ ---- -# file: roles/tg/tasks/ubuntu_x86_64.yaml - -- name: Install Unzip - apt: - name: 'unzip' - state: 'present' - update_cache: True - tags: install-unzip +--- +# file: roles/tg/tasks/ubuntu_x86_64.yaml + +- name: Install Unzip + apt: + name: 'unzip' + state: 'present' + update_cache: True + tags: install-unzip diff --git a/resources/tools/testbed-setup/ansible/roles/tg_sut/tasks/main.yaml b/resources/tools/testbed-setup/ansible/roles/tg_sut/tasks/main.yaml index 1774134abe..ea779c07da 100644 --- a/resources/tools/testbed-setup/ansible/roles/tg_sut/tasks/main.yaml +++ b/resources/tools/testbed-setup/ansible/roles/tg_sut/tasks/main.yaml @@ -1,101 +1,101 @@ ---- -# file: roles/tg_sut/tasks/main.yaml - -- name: Ubuntu specific - import_tasks: ubuntu_x86_64.yaml - when: ansible_distribution|lower == 'ubuntu' and ansible_machine == 'x86_64' - -- name: Copy interfaces file - template: - src: 'files/interfaces_physical' - dest: '/etc/network/interfaces' - owner: 'root' - group: 'root' - mode: '0644' - tags: copy-interface-file - -- name: Copy sysctl file - template: - src: 'files/90-csit' - dest: '/etc/sysctl.d/90-csit.conf' - owner: 'root' - group: 'root' - mode: '0644' - tags: copy-90-csit - -- name: Copy IRQ load balancing file - copy: - src: 'files/irqbalance' - dest: '/etc/default/irqbalance' - owner: 'root' - group: 'root' - mode: '0644' - tags: copy-irq - -- name: Copy cpufrequtils file - copy: - src: 'files/cpufrequtils' - dest: '/etc/default/cpufrequtils' - owner: 'root' - group: 'root' - mode: '0644' - tags: copy-cpufrequtils - -- name: Copy Python requirements file - copy: - src: 'files/requirements.txt' - dest: '/tmp/requirements.txt' - tags: copy-pip - -- name: Set isolcpus and pstate parameter - lineinfile: - path: '/etc/default/grub' - state: 'present' - regexp: '^GRUB_CMDLINE_LINUX=' - line: 'GRUB_CMDLINE_LINUX="isolcpus={{ isolcpus }} nohz_full={{ isolcpus }} rcu_nocbs={{ isolcpus }} numa_balancing=disable intel_pstate=disable intel_iommu=on iommu=pt nmi_watchdog=0 audit=0 nosoftlockup processor.max_cstate=1 intel_idle.max_cstate=1 hpet=disable tsc=reliable mce=off"' - notify: ['Update GRUB'] - tags: set-grub - -- name: Set ondemand service to disable - service: - name: 'ondemand' - enabled: 'no' - tags: set-ondemand - -- name: Install PIP requirements - pip: - requirements: '/tmp/requirements.txt' - tags: install-pip - -- name: Load vfio-pci by default - lineinfile: - path: '/etc/modules' - state: 'present' - line: 'vfio-pci' - tags: load-vfio-pci - -- name: Load msr by default - lineinfile: - path: '/etc/modules' - state: 'present' - line: 'msr' - tags: disable-turbo-boost - -- name: Disable Turbo-Boost by service - copy: - src: 'files/disable-turbo-boost.service' - dest: '/etc/systemd/system/disable-turbo-boost.service' - owner: 'root' - group: 'root' - mode: '0644' - when: > - inventory_hostname in groups['skylake'] - tags: disable-turbo-boost - -- name: Disable Turbo-Boost by service on startup - service: - name: disable-turbo-boost - enabled: yes - when: > - inventory_hostname in groups['skylake'] - tags: disable-turbo-boost +--- +# file: roles/tg_sut/tasks/main.yaml + +- name: Ubuntu specific + import_tasks: ubuntu_x86_64.yaml + when: ansible_distribution|lower == 'ubuntu' and ansible_machine == 'x86_64' + +- name: Copy interfaces file + template: + src: 'files/interfaces_physical' + dest: '/etc/network/interfaces' + owner: 'root' + group: 'root' + mode: '0644' + tags: copy-interface-file + +- name: Copy sysctl file + template: + src: 'files/90-csit' + dest: '/etc/sysctl.d/90-csit.conf' + owner: 'root' + group: 'root' + mode: '0644' + tags: copy-90-csit + +- name: Copy IRQ load balancing file + copy: + src: 'files/irqbalance' + dest: '/etc/default/irqbalance' + owner: 'root' + group: 'root' + mode: '0644' + tags: copy-irq + +- name: Copy cpufrequtils file + copy: + src: 'files/cpufrequtils' + dest: '/etc/default/cpufrequtils' + owner: 'root' + group: 'root' + mode: '0644' + tags: copy-cpufrequtils + +- name: Copy Python requirements file + copy: + src: 'files/requirements.txt' + dest: '/tmp/requirements.txt' + tags: copy-pip + +- name: Set isolcpus and pstate parameter + lineinfile: + path: '/etc/default/grub' + state: 'present' + regexp: '^GRUB_CMDLINE_LINUX=' + line: 'GRUB_CMDLINE_LINUX="isolcpus={{ isolcpus }} nohz_full={{ isolcpus }} rcu_nocbs={{ isolcpus }} numa_balancing=disable intel_pstate=disable intel_iommu=on iommu=pt nmi_watchdog=0 audit=0 nosoftlockup processor.max_cstate=1 intel_idle.max_cstate=1 hpet=disable tsc=reliable mce=off"' + notify: ['Update GRUB'] + tags: set-grub + +- name: Set ondemand service to disable + service: + name: 'ondemand' + enabled: 'no' + tags: set-ondemand + +- name: Install PIP requirements + pip: + requirements: '/tmp/requirements.txt' + tags: install-pip + +- name: Load vfio-pci by default + lineinfile: + path: '/etc/modules' + state: 'present' + line: 'vfio-pci' + tags: load-vfio-pci + +- name: Load msr by default + lineinfile: + path: '/etc/modules' + state: 'present' + line: 'msr' + tags: disable-turbo-boost + +- name: Disable Turbo-Boost by service + copy: + src: 'files/disable-turbo-boost.service' + dest: '/etc/systemd/system/disable-turbo-boost.service' + owner: 'root' + group: 'root' + mode: '0644' + when: > + inventory_hostname in groups['skylake'] + tags: disable-turbo-boost + +- name: Disable Turbo-Boost by service on startup + service: + name: disable-turbo-boost + enabled: yes + when: > + inventory_hostname in groups['skylake'] + tags: disable-turbo-boost diff --git a/resources/tools/testbed-setup/ansible/roles/tg_sut/tasks/ubuntu_x86_64.yaml b/resources/tools/testbed-setup/ansible/roles/tg_sut/tasks/ubuntu_x86_64.yaml index 3c6eb10d62..f43fad1326 100644 --- a/resources/tools/testbed-setup/ansible/roles/tg_sut/tasks/ubuntu_x86_64.yaml +++ b/resources/tools/testbed-setup/ansible/roles/tg_sut/tasks/ubuntu_x86_64.yaml @@ -1,182 +1,182 @@ ---- -# file: roles/tg_sut/tasks/ubuntu_x86_64.yaml - -- name: Install python-dev - apt: - name: 'python-dev' - state: 'present' - update_cache: True - tags: install-python-dev - -- name: Install python-virtualenv - apt: - name: 'python-virtualenv' - state: 'present' - update_cache: True - tags: install-python-virtualenv - -- name: Install python pip - apt: - name: 'python-pip' - state: 'present' - update_cache: True - tags: install-python-pip - -- name: Install libpcap-dev - apt: - name: 'libpcap-dev' - state: 'present' - update_cache: True - tags: install-libpcap-dev - -- name: Install cpufrequtils - apt: - name: 'cpufrequtils' - state: 'present' - update_cache: True - tags: install-cpufrequtils - -- name: Install cgroup-support - apt: - name: 'cgroup-bin' - state: 'present' - update_cache: True - tags: install-cgroup-support - -- name: Install zlib1g-dev - apt: - name: 'zlib1g-dev' - state: 'present' - update_cache: True - tags: install-zlib1g-dev - -- name: Install libnuma-dev - apt: - name: 'libnuma-dev' - state: 'present' - update_cache: True - tags: install-libnuma-dev - -- name: Install Docker and role dependencies - apt: - name: "{{ item }}" - state: "present" - install_recommends: False - with_items: - - "apt-transport-https" - - "ca-certificates" - - "software-properties-common" - - "cron" - tags: install-docker - -- name: Install upstream APT GPG key - apt_key: - id: "{{ docker_apt_key }}" - keyserver: "{{ ansible_local.core.keyserver - if (ansible_local|d() and ansible_local.core|d() and - ansible_local.core.keyserver) - else 'hkp://pool.sks-keyservers.net' }}" - state: "present" - tags: install-docker - -- name: Install upstream APT repository - apt_repository: - repo: "{{ docker_repository }}" - state: "present" - update_cache: True - tags: install-docker - -- name: Install Docker - apt: - name: "docker-{{ docker_edition }}={{ docker_apt_package_name }}" - state: "present" - update_cache: True - install_recommends: False - cache_valid_time: "{{ apt_cache_time }}" - tags: install-docker - -- name: Remove Upstart config file - file: - path: "/etc/default/docker" - state: "absent" - tags: docker - -- name: Ensure systemd directory exists - file: - path: "/etc/systemd/system" - state: "directory" - owner: "root" - group: "root" - mode: "0755" - tags: ensure-docker - -- name: Copy systemd unit file - template: - src: "templates/docker.service.j2" - dest: "/etc/systemd/system/docker.service" - owner: "root" - group: "root" - mode: "0644" - register: docker_register_systemd_service - tags: copy-docker - -- name: Reload systemd daemon - command: "systemctl daemon-reload" - notify: ["Restart Docker"] - when: (docker_register_systemd_service and - docker_register_systemd_service | changed) - tags: restart-docker - -- name: Set specific users to "docker" group - user: - name: "{{ item }}" - groups: "docker" - append: True - with_items: "{{ docker_users }}" - when: docker_users - tags: set-docker - -- name: Install upstream APT GPG key - apt_key: - id: "{{ kubernetes_apt_key }}" - keyserver: "{{ ansible_local.core.keyserver - if (ansible_local|d() and ansible_local.core|d() and - ansible_local.core.keyserver) - else 'hkp://pool.sks-keyservers.net' }}" - state: "present" - tags: install-kubernetes - -- name: Install upstream APT repository - apt_repository: - repo: "{{ kubernetes_repository }}" - state: "present" - update_cache: True - tags: install-kubernetes - -- name: Install kubeadm - apt: - name: "kubeadm={{ kubernetes_apt_package_name }}" - state: "present" - update_cache: True - install_recommends: False - cache_valid_time: "{{ apt_cache_time }}" - tags: install-kubernetes - -- name: Install kubectl - apt: - name: "kubectl={{ kubernetes_apt_package_name }}" - state: "present" - update_cache: True - install_recommends: False - cache_valid_time: "{{ apt_cache_time }}" - tags: install-kubernetes - -- name: Install kubelet - apt: - name: "kubelet={{ kubernetes_apt_package_name }}" - state: "present" - update_cache: True - install_recommends: False - cache_valid_time: "{{ apt_cache_time }}" - tags: install-kubernetes - +--- +# file: roles/tg_sut/tasks/ubuntu_x86_64.yaml + +- name: Install python-dev + apt: + name: 'python-dev' + state: 'present' + update_cache: True + tags: install-python-dev + +- name: Install python-virtualenv + apt: + name: 'python-virtualenv' + state: 'present' + update_cache: True + tags: install-python-virtualenv + +- name: Install python pip + apt: + name: 'python-pip' + state: 'present' + update_cache: True + tags: install-python-pip + +- name: Install libpcap-dev + apt: + name: 'libpcap-dev' + state: 'present' + update_cache: True + tags: install-libpcap-dev + +- name: Install cpufrequtils + apt: + name: 'cpufrequtils' + state: 'present' + update_cache: True + tags: install-cpufrequtils + +- name: Install cgroup-support + apt: + name: 'cgroup-bin' + state: 'present' + update_cache: True + tags: install-cgroup-support + +- name: Install zlib1g-dev + apt: + name: 'zlib1g-dev' + state: 'present' + update_cache: True + tags: install-zlib1g-dev + +- name: Install libnuma-dev + apt: + name: 'libnuma-dev' + state: 'present' + update_cache: True + tags: install-libnuma-dev + +- name: Install Docker and role dependencies + apt: + name: "{{ item }}" + state: "present" + install_recommends: False + with_items: + - "apt-transport-https" + - "ca-certificates" + - "software-properties-common" + - "cron" + tags: install-docker + +- name: Install upstream APT GPG key + apt_key: + id: "{{ docker_apt_key }}" + keyserver: "{{ ansible_local.core.keyserver + if (ansible_local|d() and ansible_local.core|d() and + ansible_local.core.keyserver) + else 'hkp://pool.sks-keyservers.net' }}" + state: "present" + tags: install-docker + +- name: Install upstream APT repository + apt_repository: + repo: "{{ docker_repository }}" + state: "present" + update_cache: True + tags: install-docker + +- name: Install Docker + apt: + name: "docker-{{ docker_edition }}={{ docker_apt_package_name }}" + state: "present" + update_cache: True + install_recommends: False + cache_valid_time: "{{ apt_cache_time }}" + tags: install-docker + +- name: Remove Upstart config file + file: + path: "/etc/default/docker" + state: "absent" + tags: docker + +- name: Ensure systemd directory exists + file: + path: "/etc/systemd/system" + state: "directory" + owner: "root" + group: "root" + mode: "0755" + tags: ensure-docker + +- name: Copy systemd unit file + template: + src: "templates/docker.service.j2" + dest: "/etc/systemd/system/docker.service" + owner: "root" + group: "root" + mode: "0644" + register: docker_register_systemd_service + tags: copy-docker + +- name: Reload systemd daemon + command: "systemctl daemon-reload" + notify: ["Restart Docker"] + when: (docker_register_systemd_service and + docker_register_systemd_service | changed) + tags: restart-docker + +- name: Set specific users to "docker" group + user: + name: "{{ item }}" + groups: "docker" + append: True + with_items: "{{ docker_users }}" + when: docker_users + tags: set-docker + +- name: Install upstream APT GPG key + apt_key: + id: "{{ kubernetes_apt_key }}" + keyserver: "{{ ansible_local.core.keyserver + if (ansible_local|d() and ansible_local.core|d() and + ansible_local.core.keyserver) + else 'hkp://pool.sks-keyservers.net' }}" + state: "present" + tags: install-kubernetes + +- name: Install upstream APT repository + apt_repository: + repo: "{{ kubernetes_repository }}" + state: "present" + update_cache: True + tags: install-kubernetes + +- name: Install kubeadm + apt: + name: "kubeadm={{ kubernetes_apt_package_name }}" + state: "present" + update_cache: True + install_recommends: False + cache_valid_time: "{{ apt_cache_time }}" + tags: install-kubernetes + +- name: Install kubectl + apt: + name: "kubectl={{ kubernetes_apt_package_name }}" + state: "present" + update_cache: True + install_recommends: False + cache_valid_time: "{{ apt_cache_time }}" + tags: install-kubernetes + +- name: Install kubelet + apt: + name: "kubelet={{ kubernetes_apt_package_name }}" + state: "present" + update_cache: True + install_recommends: False + cache_valid_time: "{{ apt_cache_time }}" + tags: install-kubernetes + diff --git a/resources/tools/testbed-setup/ansible/site.yaml b/resources/tools/testbed-setup/ansible/site.yaml index 8eb0e01710..834ad51787 100644 --- a/resources/tools/testbed-setup/ansible/site.yaml +++ b/resources/tools/testbed-setup/ansible/site.yaml @@ -1,14 +1,14 @@ ---- -# file: site.yaml - -- import_playbook: tg.yaml - tags: tg - -- import_playbook: sut.yaml - tags: sut - -- import_playbook: vppdevice.yaml - tags: vppdevice - -#- import_playbook: virl.yaml -# tags: virl +--- +# file: site.yaml + +- import_playbook: tg.yaml + tags: tg + +- import_playbook: sut.yaml + tags: sut + +- import_playbook: vppdevice.yaml + tags: vppdevice + +#- import_playbook: virl.yaml +# tags: virl diff --git a/resources/tools/testbed-setup/ansible/sut.yaml b/resources/tools/testbed-setup/ansible/sut.yaml index d178a02867..1f9a4cbaab 100644 --- a/resources/tools/testbed-setup/ansible/sut.yaml +++ b/resources/tools/testbed-setup/ansible/sut.yaml @@ -1,11 +1,11 @@ ---- -# file: sut.yaml - -- hosts: sut - remote_user: testuser - become: yes - become_user: root - roles: - - common - - sut - - tg_sut +--- +# file: sut.yaml + +- hosts: sut + remote_user: testuser + become: yes + become_user: root + roles: + - common + - sut + - tg_sut diff --git a/resources/tools/testbed-setup/ansible/tg.yaml b/resources/tools/testbed-setup/ansible/tg.yaml index a524fddb99..a652dc5c80 100644 --- a/resources/tools/testbed-setup/ansible/tg.yaml +++ b/resources/tools/testbed-setup/ansible/tg.yaml @@ -1,11 +1,11 @@ ---- -# file: tg.yaml - -- hosts: tg - remote_user: testuser - become: yes - become_user: root - roles: - - common - - tg - - tg_sut +--- +# file: tg.yaml + +- hosts: tg + remote_user: testuser + become: yes + become_user: root + roles: + - common + - tg + - tg_sut diff --git a/resources/tools/testbed-setup/ansible/virl.yaml b/resources/tools/testbed-setup/ansible/virl.yaml index 61f4bcefb0..5e57d585a4 100644 --- a/resources/tools/testbed-setup/ansible/virl.yaml +++ b/resources/tools/testbed-setup/ansible/virl.yaml @@ -1,10 +1,10 @@ ---- -# file: virl.yaml - -- hosts: virl - remote_user: testuser - become: yes - become_user: root - roles: - - common - - virl +--- +# file: virl.yaml + +- hosts: virl + remote_user: testuser + become: yes + become_user: root + roles: + - common + - virl diff --git a/resources/tools/testbed-setup/ansible/vppdevice.yaml b/resources/tools/testbed-setup/ansible/vppdevice.yaml index 03e61d3cb9..31912d92ff 100644 --- a/resources/tools/testbed-setup/ansible/vppdevice.yaml +++ b/resources/tools/testbed-setup/ansible/vppdevice.yaml @@ -1,10 +1,10 @@ ---- -# file: vppdevice.yaml - -- hosts: vppdevice - remote_user: testuser - become: yes - become_user: root - roles: - - common - - vppdevice +--- +# file: vppdevice.yaml + +- hosts: vppdevice + remote_user: testuser + become: yes + become_user: root + roles: + - common + - vppdevice diff --git a/tests/vpp/perf/container_memif/10ge2p1x520-eth-l2xcbase-eth-1memif-1dcr-mrr.robot b/tests/vpp/perf/container_memif/10ge2p1x520-eth-l2xcbase-eth-1memif-1dcr-mrr.robot index d0c70f0c6d..d9dcce584e 100644 --- a/tests/vpp/perf/container_memif/10ge2p1x520-eth-l2xcbase-eth-1memif-1dcr-mrr.robot +++ b/tests/vpp/perf/container_memif/10ge2p1x520-eth-l2xcbase-eth-1memif-1dcr-mrr.robot @@ -1,220 +1,220 @@ -# Copyright (c) 2018 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR -| ... | NIC_Intel-X520-DA2 | ETH | L2XCFWD | BASE | MEMIF | SINGLE_MEMIF -| ... | DOCKER -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L2 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with MEMIF -| ... | AND | Set up performance topology with containers -| ... -| Suite Teardown | Tear down 3-node performance topology with container -| ... -| Test Setup | Run Keywords -| ... | Set up performance test -| ... | AND | Restart VPP in all 'VNF' containers -| ... -| Test Teardown | Tear down performance mrr test -| ... -| Test Template | Check RR for l2xcbase-eth-1memif-1dcr -| ... -| Documentation | *Raw results L2XC test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for L2 cross connect. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 configured with L2 cross- -| ... | connect. DUT1 and DUT2 tested with 2p10GE NIC X520 Niantic by Intel. -| ... | Container is connected to VPP via Memif interface running same VPP -| ... | version as running on DUT. Resources are limited via cgroup to use 5 -| ... | cores allocated from pool of isolated CPUs. There are no memory -| ... | contraints. Cross Horizontal topology with packets flowing via DUT (VPP) -| ... | to Container, then via horizontal memif to the next Container, and so on -| ... | until the last Container then to NIC (in last Container). Single -| ... | Container is supported as of now. -| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic -| ... | at line rate and reports total received/sent packets over trial period. -| ... | Test packets are generated by TG on links to DUTs. TG traffic profile -| ... | contains two L3 flow-groups (flow-group per direction, 254 flows per -| ... | flow-group) with all packets containing Ethernet header, IPv4 header -| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC -| ... | addresses of the TG node interfaces. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile -| ${traffic_profile} | trex-sl-3n-ethip4-ip4src254 -# Container settings -| ${container_count}= | ${1} -| ${container_engine}= | Docker -| ${container_image}= | ubuntu:xenial-20180412 -| ${container_install_dkms}= | ${TRUE} -| ${container_chain_topology}= | cross_horiz -# CPU settings -| ${system_cpus}= | ${1} -| ${vpp_cpus}= | ${5} -| ${container_cpus}= | ${5} - -*** Keywords *** -| Check RR for l2xcbase-eth-1memif-1dcr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with ${wt} thread(s), ${wt}\ -| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for ${framesize} frames using single\ -| | ... | trial throughput test. -| | ... -| | [Arguments] | ${framesize} | ${wt} | ${rxq} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} -| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology -| | And Add single PCI device to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize L2 xconnect for single memif in 3-node circular topology -| | ... | ${rxq} -| | Then Traffic should pass with maximum rate | ${perf_trial_duration} -| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} - -*** Test Cases *** -| tc01-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 64B | 1C | THIS -| | framesize=${64} | wt=1 | rxq=1 - -| tc02-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 1518B | 1C -| | framesize=${1518} | wt=1 | rxq=1 - -| tc03-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 9000B | 1C -| | framesize=${9000} | wt=1 | rxq=1 - -| tc04-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ -| | ... | throughput test. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 1C -| | framesize=IMIX_v4_1 | wt=1 | rxq=1 - -| tc05-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy cores,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 64B | 2C -| | framesize=${64} | wt=2 | rxq=1 - -| tc06-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy cores,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 1518B | 2C -| | framesize=${1518} | wt=2 | rxq=1 - -| tc07-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy cores,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 9000B | 2C -| | framesize=${9000} | wt=2 | rxq=1 - -| tc08-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy cores,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ -| | ... | throughput test. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 2C -| | framesize=IMIX_v4_1 | wt=2 | rxq=1 - -| tc09-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ -| | ... | 2 receive queues per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 64B | 4C -| | framesize=${64} | wt=4 | rxq=2 - -| tc10-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ -| | ... | 2 receive queues per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 1518B | 4C -| | framesize=${1518} | wt=4 | rxq=2 - -| tc11-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ -| | ... | 2 receive queues per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 9000B | 4C -| | framesize=${9000} | wt=4 | rxq=2 - -| tc12-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ -| | ... | 2 receive queues per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ -| | ... | throughput test. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 4C -| | framesize=IMIX_v4_1 | wt=4 | rxq=2 +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | L2XCFWD | BASE | MEMIF | SINGLE_MEMIF +| ... | DOCKER +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L2 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with MEMIF +| ... | AND | Set up performance topology with containers +| ... +| Suite Teardown | Tear down 3-node performance topology with container +| ... +| Test Setup | Run Keywords +| ... | Set up performance test +| ... | AND | Restart VPP in all 'VNF' containers +| ... +| Test Teardown | Tear down performance mrr test +| ... +| Test Template | Check RR for l2xcbase-eth-1memif-1dcr +| ... +| Documentation | *Raw results L2XC test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for L2 cross connect. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 configured with L2 cross- +| ... | connect. DUT1 and DUT2 tested with 2p10GE NIC X520 Niantic by Intel. +| ... | Container is connected to VPP via Memif interface running same VPP +| ... | version as running on DUT. Resources are limited via cgroup to use 5 +| ... | cores allocated from pool of isolated CPUs. There are no memory +| ... | contraints. Cross Horizontal topology with packets flowing via DUT (VPP) +| ... | to Container, then via horizontal memif to the next Container, and so on +| ... | until the last Container then to NIC (in last Container). Single +| ... | Container is supported as of now. +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 254 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile +| ${traffic_profile} | trex-sl-3n-ethip4-ip4src254 +# Container settings +| ${container_count}= | ${1} +| ${container_engine}= | Docker +| ${container_image}= | ubuntu:xenial-20180412 +| ${container_install_dkms}= | ${TRUE} +| ${container_chain_topology}= | cross_horiz +# CPU settings +| ${system_cpus}= | ${1} +| ${vpp_cpus}= | ${5} +| ${container_cpus}= | ${5} + +*** Keywords *** +| Check RR for l2xcbase-eth-1memif-1dcr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with ${wt} thread(s), ${wt}\ +| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for ${framesize} frames using single\ +| | ... | trial throughput test. +| | ... +| | [Arguments] | ${framesize} | ${wt} | ${rxq} +| | ... +| | Set Test Variable | ${framesize} +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add single PCI device to all DUTs +| | And Run Keyword If | ${get_framesize} < ${1522} +| | ... | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | And Initialize L2 xconnect for single memif in 3-node circular topology +| | ... | ${rxq} +| | Then Traffic should pass with maximum rate | ${perf_trial_duration} +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 64B | 1C +| | framesize=${64} | wt=1 | rxq=1 + +| tc02-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 1518B | 1C +| | framesize=${1518} | wt=1 | rxq=1 + +| tc03-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 9000B | 1C +| | framesize=${9000} | wt=1 | rxq=1 + +| tc04-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ +| | ... | throughput test. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | wt=1 | rxq=1 + +| tc05-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy cores,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 64B | 2C +| | framesize=${64} | wt=2 | rxq=1 + +| tc06-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy cores,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 1518B | 2C +| | framesize=${1518} | wt=2 | rxq=1 + +| tc07-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy cores,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 9000B | 2C +| | framesize=${9000} | wt=2 | rxq=1 + +| tc08-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy cores,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ +| | ... | throughput test. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | wt=2 | rxq=1 + +| tc09-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ +| | ... | 2 receive queues per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 64B | 4C +| | framesize=${64} | wt=4 | rxq=2 + +| tc10-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ +| | ... | 2 receive queues per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 1518B | 4C +| | framesize=${1518} | wt=4 | rxq=2 + +| tc11-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ +| | ... | 2 receive queues per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 9000B | 4C +| | framesize=${9000} | wt=4 | rxq=2 + +| tc12-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ +| | ... | 2 receive queues per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ +| | ... | throughput test. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | wt=4 | rxq=2 diff --git a/tests/vpp/perf/container_memif/10ge2p1x520-eth-l2xcbase-eth-1memif-1dcr-ndrpdrdisc.robot b/tests/vpp/perf/container_memif/10ge2p1x520-eth-l2xcbase-eth-1memif-1dcr-ndrpdrdisc.robot index 6fab3c9397..91350e434f 100644 --- a/tests/vpp/perf/container_memif/10ge2p1x520-eth-l2xcbase-eth-1memif-1dcr-ndrpdrdisc.robot +++ b/tests/vpp/perf/container_memif/10ge2p1x520-eth-l2xcbase-eth-1memif-1dcr-ndrpdrdisc.robot @@ -1,361 +1,361 @@ -# Copyright (c) 2018 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | L2XCFWD | BASE | MEMIF | SINGLE_MEMIF -| ... | DOCKER -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L2 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with MEMIF -| ... | AND | Set up performance topology with containers -| ... -| Suite Teardown | Tear down 3-node performance topology with container -| ... -| Test Setup | Run Keywords -| ... | Set up performance test -| ... | AND | Restart VPP in all 'VNF' containers -| ... -| Test Teardown | Tear down performance discovery test | ${min_rate}pps -| ... | ${framesize} | ${traffic_profile} -| ... -| Test Template | Find NDRPDR for l2xcbase-eth-1memif-1dcr -| ... -| Documentation | *RFC2544: Pkt throughput L2XC test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for L2 cross connect. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 configured with L2 cross- -| ... | connect. DUT1 and DUT2 tested with 2p10GE NIC X520 Niantic by Intel. -| ... | Container is connected to VPP via Memif interface running same VPP -| ... | version as running on DUT. Resources are limited via cgroup to use 5 -| ... | cores allocated from pool of isolated CPUs. There are no memory -| ... | contraints. Cross Horizontal topology with packets flowing via DUT (VPP) -| ... | to Container, then via horizontal memif to the next Container, and so on -| ... | until the last Container then to NIC (in last Container). Single -| ... | Container is supported as of now. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, 254 flows per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with IP protocol=61 and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile -| ${traffic_profile} | trex-sl-3n-ethip4-ip4src254 -# Container settings -| ${container_count}= | ${1} -| ${container_engine}= | Docker -| ${container_image}= | ubuntu:xenial-20180412 -| ${container_install_dkms}= | ${TRUE} -| ${container_chain_topology}= | cross_horiz -# CPU settings -| ${system_cpus}= | ${1} -| ${vpp_cpus}= | ${5} -| ${container_cpus}= | ${5} - -*** Keywords *** -| Find NDRPDR for l2xcbase-eth-1memif-1dcr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with ${wt} thread(s), ${wt}\ -| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. -| | ... | [Ver] Find ${search_type} for ${framesize} frames using binary search\ -| | ... | start at 10GE linerate, step 50kpps. -| | ... -| | [Arguments] | ${framesize} | ${wt} | ${rxq} | ${search_type} -| | ... | ${min_rate}=${50000} -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology -| | And Add single PCI device to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize L2 xconnect for single memif in 3-node circular topology -| | ... | ${rxq} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 64B | 1C | NDRDISC -| | framesize=${64} | wt=1 | rxq=1 | search_type=NDR - -| tc02-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 64B | 1C | PDRDISC -| | framesize=${64} | wt=1 | rxq=1 | search_type=PDR - -| tc03-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 1518B | 1C | NDRDISC -| | framesize=${1518} | wt=1 | rxq=1 | search_type=NDR - -| tc04-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 1518B | 1C | PDRDISC -| | framesize=${1518} | wt=1 | rxq=1 | search_type=PDR - -| tc05-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 9000B | 1C | NDRDISC -| | framesize=${9000} | wt=1 | rxq=1 | search_type=NDR - -| tc06-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 9000B | 1C | PDRDISC -| | framesize=${9000} | wt=1 | rxq=1 | search_type=PDR - -| tc07-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 1C | NDRDISC -| | framesize=IMIX_v4_1 | wt=1 | rxq=1 | search_type=NDR - -| tc08-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 1C | PDRDISC -| | framesize=IMIX_v4_1 | wt=1 | rxq=1 | search_type=PDR - -| tc09-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 64B | 2C | NDRDISC -| | framesize=${64} | wt=2 | rxq=1 | search_type=NDR - -| tc10-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 64B | 1C | PDRDISC -| | framesize=${64} | wt=2 | rxq=1 | search_type=PDR - -| tc11-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 1518B | 2C | NDRDISC -| | framesize=${1518} | wt=2 | rxq=1 | search_type=NDR - -| tc12-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 1518B | 2C | PDRDISC -| | framesize=${1518} | wt=2 | rxq=1 | search_type=PDR - -| tc13-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 9000B | 2C | NDRDISC -| | framesize=${9000} | wt=2 | rxq=1 | search_type=NDR - -| tc14-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 9000B | 2C | PDRDISC -| | framesize=${9000} | wt=2 | rxq=1 | search_type=PDR - -| tc15-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 2C | NDRDISC -| | framesize=IMIX_v4_1 | wt=2 | rxq=1 | search_type=NDR - -| tc16-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 2C | PDRDISC -| | framesize=IMIX_v4_1 | wt=2 | rxq=1 | search_type=PDR - -| tc17-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 64B | 4C | NDRDISC -| | framesize=${64} | wt=4 | rxq=2 | search_type=NDR - -| tc18-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 64B | 4C | PDRDISC -| | framesize=${64} | wt=4 | rxq=2 | search_type=PDR - -| tc19-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 1518B | 4C | NDRDISC -| | framesize=${1518} | wt=4 | rxq=2| search_type=NDR - -| tc20-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 1518B | 4C | PDRDISC -| | framesize=${1518} | wt=4 | rxq=2 | search_type=PDR - -| tc21-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 9000B | 4C | NDRDISC -| | framesize=${9000} | wt=4 | rxq=2 | search_type=NDR - -| tc22-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 9000B | 4C | PDRDISC -| | framesize=${9000} | wt=4 | rxq=2 | search_type=PDR - -| tc23-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 4C | NDRDISC -| | framesize=IMIX_v4_1 | wt=4 | rxq=2 | search_type=NDR - -| tc24-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 4C | PDRDISC -| | framesize=IMIX_v4_1 | wt=4 | rxq=2 | search_type=PDR +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | L2XCFWD | BASE | MEMIF | SINGLE_MEMIF +| ... | DOCKER +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L2 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with MEMIF +| ... | AND | Set up performance topology with containers +| ... +| Suite Teardown | Tear down 3-node performance topology with container +| ... +| Test Setup | Run Keywords +| ... | Set up performance test +| ... | AND | Restart VPP in all 'VNF' containers +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Find NDRPDR for l2xcbase-eth-1memif-1dcr +| ... +| Documentation | *RFC2544: Pkt throughput L2XC test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for L2 cross connect. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 configured with L2 cross- +| ... | connect. DUT1 and DUT2 tested with 2p10GE NIC X520 Niantic by Intel. +| ... | Container is connected to VPP via Memif interface running same VPP +| ... | version as running on DUT. Resources are limited via cgroup to use 5 +| ... | cores allocated from pool of isolated CPUs. There are no memory +| ... | contraints. Cross Horizontal topology with packets flowing via DUT (VPP) +| ... | to Container, then via horizontal memif to the next Container, and so on +| ... | until the last Container then to NIC (in last Container). Single +| ... | Container is supported as of now. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using either binary search or linear search +| ... | algorithms with configured starting rate and final step that determines +| ... | throughput measurement resolution. Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, 254 flows per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with IP protocol=61 and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile +| ${traffic_profile} | trex-sl-3n-ethip4-ip4src254 +# Container settings +| ${container_count}= | ${1} +| ${container_engine}= | Docker +| ${container_image}= | ubuntu:xenial-20180412 +| ${container_install_dkms}= | ${TRUE} +| ${container_chain_topology}= | cross_horiz +# CPU settings +| ${system_cpus}= | ${1} +| ${vpp_cpus}= | ${5} +| ${container_cpus}= | ${5} + +*** Keywords *** +| Find NDRPDR for l2xcbase-eth-1memif-1dcr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with ${wt} thread(s), ${wt}\ +| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. +| | ... | [Ver] Find ${search_type} for ${framesize} frames using binary search\ +| | ... | start at 10GE linerate, step 50kpps. +| | ... +| | [Arguments] | ${framesize} | ${wt} | ${rxq} | ${search_type} +| | ... | ${min_rate}=${50000} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add single PCI device to all DUTs +| | And Run Keyword If | ${get_framesize} < ${1522} +| | ... | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | And Initialize L2 xconnect for single memif in 3-node circular topology +| | ... | ${rxq} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 64B | 1C | NDRDISC +| | framesize=${64} | wt=1 | rxq=1 | search_type=NDR + +| tc02-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1C | PDRDISC +| | framesize=${64} | wt=1 | rxq=1 | search_type=PDR + +| tc03-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 1518B | 1C | NDRDISC +| | framesize=${1518} | wt=1 | rxq=1 | search_type=NDR + +| tc04-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 1518B | 1C | PDRDISC +| | framesize=${1518} | wt=1 | rxq=1 | search_type=PDR + +| tc05-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 9000B | 1C | NDRDISC +| | framesize=${9000} | wt=1 | rxq=1 | search_type=NDR + +| tc06-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 9000B | 1C | PDRDISC +| | framesize=${9000} | wt=1 | rxq=1 | search_type=PDR + +| tc07-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 1C | NDRDISC +| | framesize=IMIX_v4_1 | wt=1 | rxq=1 | search_type=NDR + +| tc08-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 1C | PDRDISC +| | framesize=IMIX_v4_1 | wt=1 | rxq=1 | search_type=PDR + +| tc09-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 64B | 2C | NDRDISC +| | framesize=${64} | wt=2 | rxq=1 | search_type=NDR + +| tc10-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1C | PDRDISC +| | framesize=${64} | wt=2 | rxq=1 | search_type=PDR + +| tc11-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 1518B | 2C | NDRDISC +| | framesize=${1518} | wt=2 | rxq=1 | search_type=NDR + +| tc12-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 1518B | 2C | PDRDISC +| | framesize=${1518} | wt=2 | rxq=1 | search_type=PDR + +| tc13-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 9000B | 2C | NDRDISC +| | framesize=${9000} | wt=2 | rxq=1 | search_type=NDR + +| tc14-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 9000B | 2C | PDRDISC +| | framesize=${9000} | wt=2 | rxq=1 | search_type=PDR + +| tc15-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 2C | NDRDISC +| | framesize=IMIX_v4_1 | wt=2 | rxq=1 | search_type=NDR + +| tc16-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 2C | PDRDISC +| | framesize=IMIX_v4_1 | wt=2 | rxq=1 | search_type=PDR + +| tc17-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 64B | 4C | NDRDISC +| | framesize=${64} | wt=4 | rxq=2 | search_type=NDR + +| tc18-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 4C | PDRDISC +| | framesize=${64} | wt=4 | rxq=2 | search_type=PDR + +| tc19-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 1518B | 4C | NDRDISC +| | framesize=${1518} | wt=4 | rxq=2| search_type=NDR + +| tc20-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 1518B | 4C | PDRDISC +| | framesize=${1518} | wt=4 | rxq=2 | search_type=PDR + +| tc21-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 9000B | 4C | NDRDISC +| | framesize=${9000} | wt=4 | rxq=2 | search_type=NDR + +| tc22-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 9000B | 4C | PDRDISC +| | framesize=${9000} | wt=4 | rxq=2 | search_type=PDR + +| tc23-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 4C | NDRDISC +| | framesize=IMIX_v4_1 | wt=4 | rxq=2 | search_type=NDR + +| tc24-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 4C | PDRDISC +| | framesize=IMIX_v4_1 | wt=4 | rxq=2 | search_type=PDR diff --git a/tests/vpp/perf/container_memif/10ge2p1x710-eth-l2xcbase-eth-1memif-1dcr-mrr.robot b/tests/vpp/perf/container_memif/10ge2p1x710-eth-l2xcbase-eth-1memif-1dcr-mrr.robot index 044d05a45d..eee6a039da 100644 --- a/tests/vpp/perf/container_memif/10ge2p1x710-eth-l2xcbase-eth-1memif-1dcr-mrr.robot +++ b/tests/vpp/perf/container_memif/10ge2p1x710-eth-l2xcbase-eth-1memif-1dcr-mrr.robot @@ -1,219 +1,219 @@ -# Copyright (c) 2018 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR -| ... | NIC_Intel-X710 | ETH | L2XCFWD | BASE | MEMIF | SINGLE_MEMIF | DOCKER -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L2 -| ... | Intel-X710 -| ... | AND | Set up performance test suite with MEMIF -| ... | AND | Set up performance topology with containers -| ... -| Suite Teardown | Tear down 3-node performance topology with container -| ... -| Test Setup | Run Keywords -| ... | Set up performance test -| ... | AND | Restart VPP in all 'VNF' containers -| ... -| Test Teardown | Tear down performance mrr test -| ... -| Test Template | Check RR for l2xcbase-eth-1memif-1dcr -| ... -| Documentation | *Raw results L2XC test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for L2 cross connect. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 configured with L2 cross- -| ... | connect. DUT1 and DUT2 tested with 2p10GE NIC X710 by Intel. -| ... | Container is connected to VPP via Memif interface running same VPP -| ... | version as running on DUT. Resources are limited via cgroup to use 5 -| ... | cores allocated from pool of isolated CPUs. There are no memory -| ... | contraints. Cross Horizontal topology with packets flowing via DUT (VPP) -| ... | to Container, then via horizontal memif to the next Container, and so on -| ... | until the last Container then to NIC (in last Container). Single -| ... | Container is supported as of now. -| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic -| ... | at line rate and reports total received/sent packets over trial period. -| ... | Test packets are generated by TG on links to DUTs. TG traffic profile -| ... | contains two L3 flow-groups (flow-group per direction, 254 flows per -| ... | flow-group) with all packets containing Ethernet header, IPv4 header -| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC -| ... | addresses of the TG node interfaces. - -*** Variables *** -# X710-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile -| ${traffic_profile} | trex-sl-3n-ethip4-ip4src254 -# Container settings -| ${container_count}= | ${1} -| ${container_engine}= | Docker -| ${container_image}= | ubuntu:xenial-20180412 -| ${container_install_dkms}= | ${TRUE} -| ${container_chain_topology}= | cross_horiz -# CPU settings -| ${system_cpus}= | ${1} -| ${vpp_cpus}= | ${5} -| ${container_cpus}= | ${5} - -*** Keywords *** -| Check RR for l2xcbase-eth-1memif-1dcr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with ${wt} thread(s), ${wt}\ -| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for ${framesize} frames using single\ -| | ... | trial throughput test. -| | ... -| | [Arguments] | ${framesize} | ${wt} | ${rxq} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} -| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology -| | And Add single PCI device to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize L2 xconnect for single memif in 3-node circular topology -| | ... | ${rxq} -| | Then Traffic should pass with maximum rate | ${perf_trial_duration} -| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} - -*** Test Cases *** -| tc01-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 64B | 1C -| | framesize=${64} | wt=1 | rxq=1 - -| tc02-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 1518B | 1C -| | framesize=${1518} | wt=1 | rxq=1 - -| tc03-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 9000B | 1C -| | framesize=${9000} | wt=1 | rxq=1 - -| tc04-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ -| | ... | throughput test. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 1C -| | framesize=IMIX_v4_1 | wt=1 | rxq=1 - -| tc05-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy cores,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 64B | 2C -| | framesize=${64} | wt=2 | rxq=1 - -| tc06-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy cores,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 1518B | 2C -| | framesize=${1518} | wt=2 | rxq=1 - -| tc07-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy cores,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 9000B | 2C -| | framesize=${9000} | wt=2 | rxq=1 - -| tc08-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy cores,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ -| | ... | throughput test. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 2C -| | framesize=IMIX_v4_1 | wt=2 | rxq=1 - -| tc09-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ -| | ... | 2 receive queues per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 64B | 4C -| | framesize=${64} | wt=4 | rxq=2 - -| tc10-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ -| | ... | 2 receive queues per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 1518B | 4C -| | framesize=${1518} | wt=4 | rxq=2 - -| tc11-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ -| | ... | 2 receive queues per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ -| | ... | throughput test. -| | ... -| | [Tags] | 9000B | 4C -| | framesize=${9000} | wt=4 | rxq=2 - -| tc12-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ -| | ... | 2 receive queues per NIC port. -| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ -| | ... | throughput test. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 4C -| | framesize=IMIX_v4_1 | wt=4 | rxq=2 +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X710 | ETH | L2XCFWD | BASE | MEMIF | SINGLE_MEMIF | DOCKER +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L2 +| ... | Intel-X710 +| ... | AND | Set up performance test suite with MEMIF +| ... | AND | Set up performance topology with containers +| ... +| Suite Teardown | Tear down 3-node performance topology with container +| ... +| Test Setup | Run Keywords +| ... | Set up performance test +| ... | AND | Restart VPP in all 'VNF' containers +| ... +| Test Teardown | Tear down performance mrr test +| ... +| Test Template | Check RR for l2xcbase-eth-1memif-1dcr +| ... +| Documentation | *Raw results L2XC test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for L2 cross connect. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 configured with L2 cross- +| ... | connect. DUT1 and DUT2 tested with 2p10GE NIC X710 by Intel. +| ... | Container is connected to VPP via Memif interface running same VPP +| ... | version as running on DUT. Resources are limited via cgroup to use 5 +| ... | cores allocated from pool of isolated CPUs. There are no memory +| ... | contraints. Cross Horizontal topology with packets flowing via DUT (VPP) +| ... | to Container, then via horizontal memif to the next Container, and so on +| ... | until the last Container then to NIC (in last Container). Single +| ... | Container is supported as of now. +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 254 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. + +*** Variables *** +# X710-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile +| ${traffic_profile} | trex-sl-3n-ethip4-ip4src254 +# Container settings +| ${container_count}= | ${1} +| ${container_engine}= | Docker +| ${container_image}= | ubuntu:xenial-20180412 +| ${container_install_dkms}= | ${TRUE} +| ${container_chain_topology}= | cross_horiz +# CPU settings +| ${system_cpus}= | ${1} +| ${vpp_cpus}= | ${5} +| ${container_cpus}= | ${5} + +*** Keywords *** +| Check RR for l2xcbase-eth-1memif-1dcr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with ${wt} thread(s), ${wt}\ +| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for ${framesize} frames using single\ +| | ... | trial throughput test. +| | ... +| | [Arguments] | ${framesize} | ${wt} | ${rxq} +| | ... +| | Set Test Variable | ${framesize} +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add single PCI device to all DUTs +| | And Run Keyword If | ${get_framesize} < ${1522} +| | ... | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | And Initialize L2 xconnect for single memif in 3-node circular topology +| | ... | ${rxq} +| | Then Traffic should pass with maximum rate | ${perf_trial_duration} +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 64B | 1C +| | framesize=${64} | wt=1 | rxq=1 + +| tc02-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 1518B | 1C +| | framesize=${1518} | wt=1 | rxq=1 + +| tc03-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 9000B | 1C +| | framesize=${9000} | wt=1 | rxq=1 + +| tc04-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ +| | ... | throughput test. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | wt=1 | rxq=1 + +| tc05-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy cores,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 64B | 2C +| | framesize=${64} | wt=2 | rxq=1 + +| tc06-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy cores,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 1518B | 2C +| | framesize=${1518} | wt=2 | rxq=1 + +| tc07-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy cores,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 9000B | 2C +| | framesize=${9000} | wt=2 | rxq=1 + +| tc08-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy cores,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ +| | ... | throughput test. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | wt=2 | rxq=1 + +| tc09-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ +| | ... | 2 receive queues per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 64B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 64B | 4C +| | framesize=${64} | wt=4 | rxq=2 + +| tc10-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ +| | ... | 2 receive queues per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 1518B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 1518B | 4C +| | framesize=${1518} | wt=4 | rxq=2 + +| tc11-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ +| | ... | 2 receive queues per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for 9000B frames using single trial\ +| | ... | throughput test. +| | ... +| | [Tags] | 9000B | 4C +| | framesize=${9000} | wt=4 | rxq=2 + +| tc12-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-mrr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 phy cores,\ +| | ... | 2 receive queues per NIC port. +| | ... | [Ver] Measure MaxReceivedRate for IMIX_v4_1 frames using single trial\ +| | ... | throughput test. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | wt=4 | rxq=2 diff --git a/tests/vpp/perf/container_memif/10ge2p1x710-eth-l2xcbase-eth-1memif-1dcr-ndrpdrdisc.robot b/tests/vpp/perf/container_memif/10ge2p1x710-eth-l2xcbase-eth-1memif-1dcr-ndrpdrdisc.robot index 37c8fd8664..386260c4cd 100644 --- a/tests/vpp/perf/container_memif/10ge2p1x710-eth-l2xcbase-eth-1memif-1dcr-ndrpdrdisc.robot +++ b/tests/vpp/perf/container_memif/10ge2p1x710-eth-l2xcbase-eth-1memif-1dcr-ndrpdrdisc.robot @@ -1,360 +1,360 @@ -# Copyright (c) 2018 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X710 | ETH | L2XCFWD | BASE | MEMIF | SINGLE_MEMIF | DOCKER -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L2 -| ... | Intel-X710 -| ... | AND | Set up performance test suite with MEMIF -| ... | AND | Set up performance topology with containers -| ... -| Suite Teardown | Tear down 3-node performance topology with container -| ... -| Test Setup | Run Keywords -| ... | Set up performance test -| ... | AND | Restart VPP in all 'VNF' containers -| ... -| Test Teardown | Tear down performance discovery test | ${min_rate}pps -| ... | ${framesize} | ${traffic_profile} -| ... -| Test Template | Find NDRPDR for l2xcbase-eth-1memif-1dcr -| ... -| Documentation | *Raw results L2XC test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for L2 cross connect. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 configured with L2 cross- -| ... | connect. DUT1 and DUT2 tested with 2p10GE NIC X710 by Intel. -| ... | Container is connected to VPP via Memif interface running same VPP -| ... | version as running on DUT. Resources are limited via cgroup to use 5 -| ... | cores allocated from pool of isolated CPUs. There are no memory -| ... | contraints. Cross Horizontal topology with packets flowing via DUT (VPP) -| ... | to Container, then via horizontal memif to the next Container, and so on -| ... | until the last Container then to NIC (in last Container). Single -| ... | Container is supported as of now. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, 254 flows per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with IP protocol=61 and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X710-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile -| ${traffic_profile} | trex-sl-3n-ethip4-ip4src254 -# Container settings -| ${container_count}= | ${1} -| ${container_engine}= | Docker -| ${container_image}= | ubuntu:xenial-20180412 -| ${container_install_dkms}= | ${TRUE} -| ${container_chain_topology}= | cross_horiz -# CPU settings -| ${system_cpus}= | ${1} -| ${vpp_cpus}= | ${5} -| ${container_cpus}= | ${5} - -*** Keywords *** -| Find NDRPDR for l2xcbase-eth-1memif-1dcr -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with ${wt} thread(s), ${wt}\ -| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. -| | ... | [Ver] Find ${search_type} for ${framesize} frames using binary search\ -| | ... | start at 10GE linerate, step 50kpps. -| | ... -| | [Arguments] | ${framesize} | ${wt} | ${rxq} | ${search_type} -| | ... | ${min_rate}=${50000} -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology -| | And Add single PCI device to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize L2 xconnect for single memif in 3-node circular topology -| | ... | ${rxq} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 64B | 1C | NDRDISC -| | framesize=${64} | wt=1 | rxq=1 | search_type=NDR - -| tc02-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 64B | 1C | PDRDISC -| | framesize=${64} | wt=1 | rxq=1 | search_type=PDR - -| tc03-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 1518B | 1C | NDRDISC -| | framesize=${1518} | wt=1 | rxq=1 | search_type=NDR - -| tc04-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 1518B | 1C | PDRDISC -| | framesize=${1518} | wt=1 | rxq=1 | search_type=PDR - -| tc05-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 9000B | 1C | NDRDISC -| | framesize=${9000} | wt=1 | rxq=1 | search_type=NDR - -| tc06-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 9000B | 1C | PDRDISC -| | framesize=${9000} | wt=1 | rxq=1 | search_type=PDR - -| tc07-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 1C | NDRDISC -| | framesize=IMIX_v4_1 | wt=1 | rxq=1 | search_type=NDR - -| tc08-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 1C | PDRDISC -| | framesize=IMIX_v4_1 | wt=1 | rxq=1 | search_type=PDR - -| tc09-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 64B | 2C | NDRDISC -| | framesize=${64} | wt=2 | rxq=1 | search_type=NDR - -| tc10-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 64B | 1C | PDRDISC -| | framesize=${64} | wt=2 | rxq=1 | search_type=PDR - -| tc11-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 1518B | 2C | NDRDISC -| | framesize=${1518} | wt=2 | rxq=1 | search_type=NDR - -| tc12-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 1518B | 2C | PDRDISC -| | framesize=${1518} | wt=2 | rxq=1 | search_type=PDR - -| tc13-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 9000B | 2C | NDRDISC -| | framesize=${9000} | wt=2 | rxq=1 | search_type=NDR - -| tc14-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 9000B | 2C | PDRDISC -| | framesize=${9000} | wt=2 | rxq=1 | search_type=PDR - -| tc15-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 2C | NDRDISC -| | framesize=IMIX_v4_1 | wt=2 | rxq=1 | search_type=NDR - -| tc16-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ -| | ... | 1 receive queue per NIC port. -| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 2C | PDRDISC -| | framesize=IMIX_v4_1 | wt=2 | rxq=1 | search_type=PDR - -| tc17-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 64B | 4C | NDRDISC -| | framesize=${64} | wt=4 | rxq=2 | search_type=NDR - -| tc18-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 64B | 4C | PDRDISC -| | framesize=${64} | wt=4 | rxq=2 | search_type=PDR - -| tc19-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 1518B | 4C | NDRDISC -| | framesize=${1518} | wt=4 | rxq=2 | search_type=NDR - -| tc20-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 1518B | 4C | PDRDISC -| | framesize=${1518} | wt=4 | rxq=2 | search_type=PDR - -| tc21-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... -| | [Tags] | 9000B | 4C | NDRDISC -| | framesize=${9000} | wt=4 | rxq=2 | search_type=NDR - -| tc22-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... -| | [Tags] | 9000B | 4C | PDRDISC -| | framesize=${9000} | wt=4 | rxq=2 | search_type=PDR - -| tc23-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 4C | NDRDISC -| | framesize=IMIX_v4_1 | wt=4 | rxq=2 | search_type=NDR - -| tc24-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ -| | ... | 2 receive queue per NIC port. -| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ -| | ... | linerate, step 50kpps, LT=0.5%. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | ... -| | [Tags] | IMIX | 4C | PDRDISC -| | framesize=IMIX_v4_1 | wt=4 | rxq=2 | search_type=PDR +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X710 | ETH | L2XCFWD | BASE | MEMIF | SINGLE_MEMIF | DOCKER +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L2 +| ... | Intel-X710 +| ... | AND | Set up performance test suite with MEMIF +| ... | AND | Set up performance topology with containers +| ... +| Suite Teardown | Tear down 3-node performance topology with container +| ... +| Test Setup | Run Keywords +| ... | Set up performance test +| ... | AND | Restart VPP in all 'VNF' containers +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Find NDRPDR for l2xcbase-eth-1memif-1dcr +| ... +| Documentation | *Raw results L2XC test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for L2 cross connect. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 configured with L2 cross- +| ... | connect. DUT1 and DUT2 tested with 2p10GE NIC X710 by Intel. +| ... | Container is connected to VPP via Memif interface running same VPP +| ... | version as running on DUT. Resources are limited via cgroup to use 5 +| ... | cores allocated from pool of isolated CPUs. There are no memory +| ... | contraints. Cross Horizontal topology with packets flowing via DUT (VPP) +| ... | to Container, then via horizontal memif to the next Container, and so on +| ... | until the last Container then to NIC (in last Container). Single +| ... | Container is supported as of now. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using either binary search or linear search +| ... | algorithms with configured starting rate and final step that determines +| ... | throughput measurement resolution. Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, 254 flows per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with IP protocol=61 and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile +| ${traffic_profile} | trex-sl-3n-ethip4-ip4src254 +# Container settings +| ${container_count}= | ${1} +| ${container_engine}= | Docker +| ${container_image}= | ubuntu:xenial-20180412 +| ${container_install_dkms}= | ${TRUE} +| ${container_chain_topology}= | cross_horiz +# CPU settings +| ${system_cpus}= | ${1} +| ${vpp_cpus}= | ${5} +| ${container_cpus}= | ${5} + +*** Keywords *** +| Find NDRPDR for l2xcbase-eth-1memif-1dcr +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with ${wt} thread(s), ${wt}\ +| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. +| | ... | [Ver] Find ${search_type} for ${framesize} frames using binary search\ +| | ... | start at 10GE linerate, step 50kpps. +| | ... +| | [Arguments] | ${framesize} | ${wt} | ${rxq} | ${search_type} +| | ... | ${min_rate}=${50000} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add single PCI device to all DUTs +| | And Run Keyword If | ${get_framesize} < ${1522} +| | ... | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | And Initialize L2 xconnect for single memif in 3-node circular topology +| | ... | ${rxq} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 64B | 1C | NDRDISC +| | framesize=${64} | wt=1 | rxq=1 | search_type=NDR + +| tc02-64B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1C | PDRDISC +| | framesize=${64} | wt=1 | rxq=1 | search_type=PDR + +| tc03-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 1518B | 1C | NDRDISC +| | framesize=${1518} | wt=1 | rxq=1 | search_type=NDR + +| tc04-1518B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 1518B | 1C | PDRDISC +| | framesize=${1518} | wt=1 | rxq=1 | search_type=PDR + +| tc05-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 9000B | 1C | NDRDISC +| | framesize=${9000} | wt=1 | rxq=1 | search_type=NDR + +| tc06-9000B-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 9000B | 1C | PDRDISC +| | framesize=${9000} | wt=1 | rxq=1 | search_type=PDR + +| tc07-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 1C | NDRDISC +| | framesize=IMIX_v4_1 | wt=1 | rxq=1 | search_type=NDR + +| tc08-IMIX-1t1c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 1 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 1C | PDRDISC +| | framesize=IMIX_v4_1 | wt=1 | rxq=1 | search_type=PDR + +| tc09-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 64B | 2C | NDRDISC +| | framesize=${64} | wt=2 | rxq=1 | search_type=NDR + +| tc10-64B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1C | PDRDISC +| | framesize=${64} | wt=2 | rxq=1 | search_type=PDR + +| tc11-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 1518B | 2C | NDRDISC +| | framesize=${1518} | wt=2 | rxq=1 | search_type=NDR + +| tc12-1518B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 1518B | 2C | PDRDISC +| | framesize=${1518} | wt=2 | rxq=1 | search_type=PDR + +| tc13-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 9000B | 2C | NDRDISC +| | framesize=${9000} | wt=2 | rxq=1 | search_type=NDR + +| tc14-9000B-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 9000B | 2C | PDRDISC +| | framesize=${9000} | wt=2 | rxq=1 | search_type=PDR + +| tc15-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 2C | NDRDISC +| | framesize=IMIX_v4_1 | wt=2 | rxq=1 | search_type=NDR + +| tc16-IMIX-2t2c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 2 phy core,\ +| | ... | 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 2C | PDRDISC +| | framesize=IMIX_v4_1 | wt=2 | rxq=1 | search_type=PDR + +| tc17-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 64B | 4C | NDRDISC +| | framesize=${64} | wt=4 | rxq=2 | search_type=NDR + +| tc18-64B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 4C | PDRDISC +| | framesize=${64} | wt=4 | rxq=2 | search_type=PDR + +| tc19-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find NDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 1518B | 4C | NDRDISC +| | framesize=${1518} | wt=4 | rxq=2 | search_type=NDR + +| tc20-1518B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find PDR for 1518 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 1518B | 4C | PDRDISC +| | framesize=${1518} | wt=4 | rxq=2 | search_type=PDR + +| tc21-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find NDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... +| | [Tags] | 9000B | 4C | NDRDISC +| | framesize=${9000} | wt=4 | rxq=2 | search_type=NDR + +| tc22-9000B-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find PDR for 9000 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... +| | [Tags] | 9000B | 4C | PDRDISC +| | framesize=${9000} | wt=4 | rxq=2 | search_type=PDR + +| tc23-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find NDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 4C | NDRDISC +| | framesize=IMIX_v4_1 | wt=4 | rxq=2 | search_type=NDR + +| tc24-IMIX-4t4c-eth-l2xcbase-eth-1memif-1dcr-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs L2XC switching config with 4 thread, 4 phy core,\ +| | ... | 2 receive queue per NIC port. +| | ... | [Ver] Find PDR for IMIX_v4_1 frames using binary search start at 10GE\ +| | ... | linerate, step 50kpps, LT=0.5%. +| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) +| | ... +| | [Tags] | IMIX | 4C | PDRDISC +| | framesize=IMIX_v4_1 | wt=4 | rxq=2 | search_type=PDR -- 2.16.6