From 1749fc8da247a5aacd772b7f7b7353b3ee0733db Mon Sep 17 00:00:00 2001 From: Maciek Konstantynowicz Date: Wed, 1 Nov 2017 14:33:12 +0000 Subject: [PATCH] take-2: csit report static content updates for rls1710. Change-Id: I097d62a36473303f9e0c6c06a38a68e077f86b14 Signed-off-by: Maciek Konstantynowicz --- .../vpp_unittest_results/vpp_unittest_results.rst | 554 +++++++++++++++++---- docs/report/introduction/general_notes.rst | 53 +- docs/report/introduction/overview.rst | 7 + docs/report/vpp_unit_tests/overview.rst | 62 +-- 4 files changed, 533 insertions(+), 143 deletions(-) diff --git a/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst b/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst index 63bcb1ec2c..f20bb98055 100644 --- a/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst +++ b/docs/report/detailed_test_results/vpp_unittest_results/vpp_unittest_results.rst @@ -8,14 +8,43 @@ CRUD Loopback test_crud (test_interface_crud.TestLoopbackInterfaceCRUD) OK test_down (test_interface_crud.TestLoopbackInterfaceCRUD) OK -Flow-per-packet plugin -`````````````````````` +Flowprobe feature +````````````````` :: ============================================================================== - Flow-per-packet plugin: test both L2 and IP4 reporting + Re-enable Flowprobe feature ============================================================================== - Flow per packet L3 test OK + disable flowprobe feature after first packets and re-enable SKIP + ============================================================================== + collect information on Ethernet, IP4 and IP6 datapath (no timers) + ============================================================================== + no timers, one CFLOW packet, 9 Flows inside OK + no timers, two CFLOW packets (mtu=256), 3 Flows in each OK + L2 data on IP4 datapath OK + L2 data on IP6 datapath OK + L2 data on L2 datapath OK + L3 data on IP4 datapath OK + L3 data on IP6 datapath OK + L3 data on L2 datapath OK + L4 data on IP4 datapath OK + L4 data on IP6 datapath OK + L4 data on L2 datapath OK + verify templates on IP6 datapath OK + verify templates on IP4 datapath OK + verify template on L2 datapath OK + ============================================================================== + Disable Flowprobe feature + ============================================================================== + disable flowprobe feature after first packets SKIP + ============================================================================== + Re-enable IPFIX + ============================================================================== + disable IPFIX after first packets and re-enable after few packets SKIP + ============================================================================== + Disable IPFIX + ============================================================================== + disable IPFIX after first packets SKIP DHCP ```` @@ -25,6 +54,7 @@ DHCP DHCP Test Case ============================================================================== DHCPv6 Proxy OK + DHCP Client OK DHCPv4 Proxy OK IPv4 VRF Multi-instance @@ -70,8 +100,13 @@ BFD IPv6 echo function used SKIP echo packets looped back SKIP hold BFD session up SKIP + interface with bfd session deleted SKIP bring BFD session up SKIP bring BFD session up - first frame looked up by address pair SKIP + ============================================================================== + BFD-FIB interactions (IPv6) + ============================================================================== + BFD-FIB interactions SKIP BFD authentication `````````````````` @@ -175,6 +210,8 @@ BFD IPv4 stale echo packets don't keep a session up SKIP no packets when zero remote required min rx interval SKIP + + IPv6 Tests `````````` :: @@ -184,29 +221,38 @@ IPv6 Tests ============================================================================== IPv6 FIB test OK IPv6 Neighbour Solicitation Exceptions OK + ND Duplicates OK IPv6 Router Solicitation Exceptions OK ============================================================================== IPv6 disabled ============================================================================== IP Disabled OK - IPv6 ND Proxy ============================================================================== IPv6 ND ProxyTest Case ============================================================================== IPv6 Proxy ND OK ============================================================================== + IPv6 Load-Balancing + ============================================================================== + IPv6 Load-Balancing OK + ============================================================================== IPv6 routes via NULL ============================================================================== IP NULL route OK -SPAN Test -````````` +SPAN Tests +`````````` :: ============================================================================== SPAN Test Case ============================================================================== - SPAN test OK + SPAN device rx mirror test OK + SPAN l2 tx mirror test OK + SPAN l2 rx mirror test OK + SPAN l2 rx mirror into gre-subif+vtr OK + SPAN l2 rx mirror into subif+vtr OK + SPAN l2 rx mirror into vxlan test OK GRE Tests ````````` @@ -215,7 +261,8 @@ GRE Tests ============================================================================== GRE Test Case ============================================================================== - GRE tunnel Tests OK + GRE IPv4 tunnel Tests OK + GRE IPv6 tunnel Tests OK GRE tunnel L2 Tests OK GRE tunnel VRF Tests OK @@ -250,40 +297,79 @@ LISP tests ============================================================================== Test case for basic encapsulation OK -NAT Test Cases -`````````````` +NAT44 Test Cases +```````````````` :: ============================================================================== - SNAT Test Cases + NAT44 Test Cases ============================================================================== - SNAT dynamic translation test OK - SNAT handling of client packets with TTL=1 OK - SNAT handling of error responses to client packets with TTL=2 OK - SNAT handling of server packets with TTL=1 OK - SNAT handling of error responses to server packets with TTL=2 OK - SNAT hairpinning OK - SNAT multiple inside interfaces with overlapping address space OK - Acquire SNAT addresses from interface OK + NAT44 dynamic translation test OK + NAT44 handling of client packets with TTL=1 OK + NAT44 handling of error responses to client packets with TTL=2 OK + NAT44 handling of server packets with TTL=1 OK + NAT44 handling of error responses to server packets with TTL=2 OK + NAT44 interfaces without configured IP address OK + NAT44 hairpinning - 1:1 NAPT OK + NAT44 hairpinning - 1:1 NAT OK + 1:1 NAT translate packet with unknown protocol - hairpinning OK + NAT44 translate packet with unknown protocol - hairpinning OK + NAT44 multiple inside interfaces with overlapping address space OK + Acquire NAT44 addresses from interface OK Static mapping with addresses from interface OK - S-NAT IPFIX logging NAT addresses exhausted OK - S-NAT IPFIX logging NAT44 session created/delted OK + IPFIX logging NAT addresses exhausted OK + IPFIX logging NAT44 session created/delted OK MAX translations per user - recycle the least recently used OK - SNAT multiple inside interfaces (non-overlapping address space) OK + NAT44 multiple non-overlapping address space inside interfaces OK + NAT44 interface output feature (in2out postrouting) OK + NAT44 interface output feature hairpinning (in2out postrouting) OK + NAT44 interface output feature VRF aware (in2out postrouting) OK Ping internal host from outside network OK - Ping SNAT out interface from outside network OK - S-NAT add pool addresses to FIB OK - SNAT 1:1 NAT initialized from inside network OK - SNAT 1:1 NAT initialized from outside network OK - SNAT 1:1 NAT VRF awareness OK - SNAT 1:1 NAT with port initialized from inside network OK - SNAT 1:1 NAT with port initialized from outside network OK - S-NAT tenant VRF independent address pool mode OK - S-NAT tenant VRF aware address pool mode OK + Ping NAT44 out interface from outside network OK + NAT44 add pool addresses to FIB OK + 1:1 NAT initialized from inside network OK + NAT44 interfaces without configured IP address - 1:1 NAT OK + NAT44 local service load balancing OK + 1:1 NAT initialized from outside network OK + 1:1 NAT translate packet with unknown protocol OK + 1:1 NAT VRF awareness OK + 1:1 NAPT initialized from inside network OK + NAT44 interfaces without configured IP address - 1:1 NAPT OK + 1:1 NAPT initialized from outside network OK + NAT44 translate packet with unknown protocol OK + NAT44 tenant VRF independent address pool mode OK + NAT44 tenant VRF aware address pool mode OK ============================================================================== Deterministic NAT Test Cases ============================================================================== - S-NAT run deterministic mode OK + Deterministic NAT translation test (TCP, UDP, ICMP) OK + NAT plugin run deterministic mode OK + Deterministic NAT multiple users OK + Deterministic NAT maximum sessions per user limit SKIP + Deterministic NAT session timeouts SKIP + Set deterministic NAT timeouts OK + Deterministic NAT TCP session close from inside network OK + Deterministic NAT TCP session close from outside network OK + +NAT64 Test Cases +```````````````` +:: + + ============================================================================== + NAT64 Test Cases + ============================================================================== + NAT64 dynamic translation test OK + NAT64 hairpinning OK + NAT64 translate packet with unknown protocol - hairpinning OK + NAT64 ICMP Error message translation OK + Enable/disable NAT64 feature on the interface OK + Add/delete address to NAT64 pool OK + NAT64 Network-Specific Prefix OK + NAT64 session timeout SKIP + Set NAT64 timeouts OK + NAT64 static translation test OK + Add/delete static BIB entry OK + NAT64 translate packet with unknown protocol OK L2XC Multi-instance ``````````````````` @@ -313,6 +399,18 @@ IPv4 Tests IPv4 disabled ============================================================================== IP Disabled OK + ============================================================================== + IPv4 Subnets + ============================================================================== + IP Sub Nets OK + ============================================================================== + IPv4 VLAN-0 + ============================================================================== + IP VLAN-0 OK + ============================================================================== + IPv4 Load-Balancing + ============================================================================== + IP Load-Balancing OK IPv4 FIB CRUD ````````````` @@ -333,8 +431,39 @@ IRB Tests ============================================================================== IRB Test Case ============================================================================== - IPv4 IRB test 1 OK - IPv4 IRB test 2 OK + ACL plugin prepare OK + ACL IPv6 routed -> bridged, L2 ACL deny OK + ACL IPv6 routed -> bridged, L3 ACL deny OK + ACL IPv4 routed -> bridged, L2 ACL deny OK + ACL IPv4 routed -> bridged, L3 ACL deny OK + ACL IPv6 bridged -> routed, L2 ACL deny OK + ACL IPv6 bridged -> routed, L3 ACL deny OK + ACL IPv4 bridged -> routed, L2 ACL deny OK + ACL IPv4 bridged -> routed, L3 ACL deny OK + ACL IPv6 routed -> bridged, L2 ACL permit+reflect OK + ACL IPv6 bridged -> routed, L2 ACL permit+reflect OK + ACL IPv4 routed -> bridged, L2 ACL permit+reflect OK + ACL IPv4 bridged -> routed, L2 ACL permit+reflect OK + ACL IPv6 routed -> bridged, L3 ACL permit+reflect OK + ACL IPv6 bridged -> routed, L3 ACL permit+reflect OK + ACL IPv4 routed -> bridged, L3 ACL permit+reflect OK + ACL IPv4 bridged -> routed, L3 ACL permit+reflect OK + ACL IPv6+EH routed -> bridged, L2 ACL deny OK + ACL IPv6+EH routed -> bridged, L3 ACL deny OK + ACL IPv6+EH bridged -> routed, L2 ACL deny OK + ACL IPv6+EH bridged -> routed, L3 ACL deny OK + ACL IPv6+EH routed -> bridged, L2 ACL permit+reflect OK + ACL IPv6+EH bridged -> routed, L2 ACL permit+reflect OK + ACL IPv6+EH routed -> bridged, L3 ACL permit+reflect OK + ACL IPv6+EH bridged -> routed, L3 ACL permit+reflect OK + ACL IPv4+MF routed -> bridged, L2 ACL deny OK + ACL IPv4+MF routed -> bridged, L3 ACL deny OK + ACL IPv4+MF bridged -> routed, L2 ACL deny OK + ACL IPv4+MF bridged -> routed, L3 ACL deny OK + ACL IPv4+MF routed -> bridged, L2 ACL permit+reflect OK + ACL IPv4+MF bridged -> routed, L2 ACL permit+reflect OK + ACL IPv4+MF routed -> bridged, L3 ACL permit+reflect OK + ACL IPv4+MF bridged -> routed, L3 ACL permit+reflect OK ACL Security Groups ``````````````````` @@ -344,7 +473,7 @@ ACL Security Groups ACL plugin Test Case ============================================================================== ACL plugin version check; learn MACs OK - ACL create test OK + ACL create/delete test OK permit ACL apply test OK deny ACL apply test OK VPP_624 permit ICMPv4 OK @@ -365,22 +494,75 @@ ACL Security Groups deny single TCPv4/v6 OK deny single UDPv4/v6 OK deny single UDPv4/v6, permit ip any, verify non-initial fragment blocked OK + VPP-687 zero length udp ipv4 packet OK + VPP-687 zero length udp ipv6 packet OK + permit TCPv4 + non-match range OK + permit TCPv6 + non-match range OK + permit UDPv4 + non-match range OK + permit UDPv6 + non-match range OK + deny TCPv4/v6 + non-match range OK + deny UDPv4/v6 + non-match range OK + ============================================================================== + ACL plugin connection-oriented extended testcases + ============================================================================== + Prepare the settings SKIP + IPv4: Basic conn timeout test reflect on ingress SKIP + IPv4: Basic conn timeout test reflect on egress SKIP + IPv4: reflect egress, clear conn SKIP + IPv4: reflect ingress, clear conn SKIP + IPv4: Idle conn behind active conn, reflect on ingress SKIP + IPv4: Idle conn behind active conn, reflect on egress SKIP + IPv6: Basic conn timeout test reflect on ingress SKIP + IPv6: Basic conn timeout test reflect on egress SKIP + IPv6: reflect egress, clear conn SKIP + IPv6: reflect ingress, clear conn SKIP + IPv6: Idle conn behind active conn, reflect on ingress SKIP + IPv6: Idle conn behind active conn, reflect on egress SKIP + Prepare for TCP session tests SKIP + IPv4: transient TCP session (incomplete 3WHS), ref. on ingress SKIP + IPv4: transient TCP session (incomplete 3WHS), ref. on egress SKIP + IPv4: established TCP session (complete 3WHS), ref. on ingress SKIP + IPv4: established TCP session (complete 3WHS), ref. on egress SKIP + IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP + IPv4: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP + IPv6: transient TCP session (incomplete 3WHS), ref. on ingress SKIP + IPv6: transient TCP session (incomplete 3WHS), ref. on egress SKIP + IPv6: established TCP session (complete 3WHS), ref. on ingress SKIP + IPv6: established TCP session (complete 3WHS), ref. on egress SKIP + IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on ingress SKIP + IPv6: transient TCP session (3WHS,ACK,FINACK), ref. on egress SKIP MPLS Tests `````````` :: + ============================================================================== + MPLS-L2 + ============================================================================== + Virtual Private LAN Service OK + Virtual Private Wire Service OK ============================================================================== MPLS Test Case ============================================================================== MPLS Local Label Binding test OK MPLS Deagg OK MPLS label imposition test OK + MPLS Interface Receive OK + MPLS Multicast Head-end OK + MPLS IPv4 Multicast Tail OK + MPLS IPv6 Multicast Tail OK + MPLS Multicast Mid Point OK MPLS label swap tests OK MPLS Tunnel Tests OK MPLS V4 Explicit NULL test OK MPLS V6 Explicit NULL test OK ============================================================================== + MPLS PIC edge convergence + ============================================================================== + MPLS eBGP PIC edge convergence OK + MPLS iBGP PIC edge convergence OK + MPLSv6 eBGP PIC edge convergence OK + ============================================================================== MPLS disabled ============================================================================== MPLS Disabled OK @@ -424,14 +606,16 @@ IP Multicast Tests IP Multicast Test Case ============================================================================== IPv6 Multicast Replication OK + IPv6 Multicast Replication in non-default table OK IP Multicast Replication OK IP Multicast Connected Source check OK IP Multicast Signal OK + IP Multicast Replication in non-default table OK Classifier `````````` :: - +####### ============================================================================== Classifier Test Case ============================================================================== @@ -439,56 +623,6 @@ Classifier MAC ACL test OK IP PBR test OK -IRB Tests -````````` -:: - - ============================================================================== - IRB Test Case - ============================================================================== - ACL plugin prepare OK - ACL IPv6 routed -> bridged, L2 ACL deny OK - ACL IPv6 routed -> bridged, L3 ACL deny OK - ACL IPv4 routed -> bridged, L2 ACL deny OK - ACL IPv4 routed -> bridged, L3 ACL deny OK - ACL IPv6 bridged -> routed, L2 ACL deny OK - ACL IPv6 bridged -> routed, L3 ACL deny OK - ACL IPv4 bridged -> routed, L2 ACL deny OK - ACL IPv4 bridged -> routed, L3 ACL deny OK - ACL IPv6 routed -> bridged, L2 ACL permit+reflect OK - ACL IPv6 bridged -> routed, L2 ACL permit+reflect OK - ACL IPv4 routed -> bridged, L2 ACL permit+reflect OK - ACL IPv4 bridged -> routed, L2 ACL permit+reflect OK - ACL IPv6 routed -> bridged, L3 ACL permit+reflect OK - ACL IPv6 bridged -> routed, L3 ACL permit+reflect OK - ACL IPv4 routed -> bridged, L3 ACL permit+reflect OK - ACL IPv4 bridged -> routed, L3 ACL permit+reflect OK - ACL IPv6+EH routed -> bridged, L2 ACL deny OK - ACL IPv6+EH routed -> bridged, L3 ACL deny OK - ACL IPv6+EH bridged -> routed, L2 ACL deny OK - ACL IPv6+EH bridged -> routed, L3 ACL deny OK - ACL IPv6+EH routed -> bridged, L2 ACL permit+reflect OK - ACL IPv6+EH bridged -> routed, L2 ACL permit+reflect OK - ACL IPv6+EH routed -> bridged, L3 ACL permit+reflect OK - ACL IPv6+EH bridged -> routed, L3 ACL permit+reflect OK - ACL IPv4+MF routed -> bridged, L2 ACL deny OK - ACL IPv4+MF routed -> bridged, L3 ACL deny OK - ACL IPv4+MF bridged -> routed, L2 ACL deny OK - ACL IPv4+MF bridged -> routed, L3 ACL deny OK - ACL IPv4+MF routed -> bridged, L2 ACL permit+reflect OK - ACL IPv4+MF bridged -> routed, L2 ACL permit+reflect OK - ACL IPv4+MF routed -> bridged, L3 ACL permit+reflect OK - ACL IPv4+MF bridged -> routed, L3 ACL permit+reflect OK - ACL plugin set old L2 datapath OK - ACL IPv6 routed -> bridged, L2 ACL deny OK - ACL IPv6 routed -> bridged, L3 ACL deny OK - ACL IPv4 routed -> bridged, L2 ACL deny OK - ACL IPv4 routed -> bridged, L3 ACL deny OK - ACL IPv6 bridged -> routed, L2 ACL deny OK - ACL IPv6 bridged -> routed, L3 ACL deny OK - ACL IPv4 bridged -> routed, L2 ACL deny OK - ACL IPv4 bridged -> routed, L3 ACL deny OK - VXLAN Tests ``````````` :: @@ -502,6 +636,19 @@ VXLAN Tests Multicast receive test OK Unicast flood test OK +VXLAN-GPE Tests +``````````` +:: + + ============================================================================== + VXLAN-GPE Test Case + ============================================================================== + Decapsulation test SKIP + Encapsulation test SKIP + Multicast flood test SKIP + Multicast receive test SKIP + Unicast flood test SKIP + L2 FIB CRUD ``````````` :: @@ -513,6 +660,11 @@ L2 FIB CRUD L2 FIB test 2 - delete 12 MAC entries OK L2 FIB test 3 - program new 100 MAC addresses OK L2 FIB test 4 - delete 160 MAC entries OK + L2 FIB test 5 - Program 10 new MAC entries, learn 10 OK + L2 FIB test 6 - flush first interface OK + L2 FIB test 7 - flush bd_id OK + L2 FIB test 8 - flush all OK + L2 FIB test 9 - mac learning events OK ARP Tests ````````` @@ -534,3 +686,231 @@ L2BD Tests ============================================================================== L2BD MAC learning dual-loop test OK L2BD MAC learning single-loop test OK + +MACIP Access Control Tests +`````````` +:: + + ============================================================================== + MACIP Test Case + ============================================================================== + IP4 MACIP wildcardMAC|exactIP ACL bridged traffic OK + MACIP 10 ACLs each with 100+ entries OK + MACIP 10 ACLs each with 100+ entries with IP4 traffic OK + MACIP 10 ACLs each with 100+ entries with IP6 traffic OK + MACIP ACL with 10 entries OK + MACIP ACL with 100 entries OK + MACIP ACL with 2 entries OK + MACIP ACL with 20 entries OK + MACIP ACL with 5 entries OK + MACIP ACL with 50 entries OK + MACIP 2 ACLs each with 100+ entries OK + IP4 MACIP exactMAC|exactIP ACL bridged traffic OK + IP4 MACIP exactMAC|subnetIP ACL bridged traffic OK + IP4 MACIP exactMAC|wildIP ACL bridged traffic OK + IP4 MACIP ouiMAC|exactIP ACL bridged traffic OK + IP4 MACIP ouiMAC|subnetIP ACL bridged traffic OK + IP4 MACIP ouiMAC|wildIP ACL bridged traffic OK + IP4 MACIP wildcardMAC|subnetIP ACL bridged traffic OK + IP4 MACIP wildcardMAC|wildIP ACL bridged traffic OK + IP6 MACIP exactMAC|exactIP ACL bridged traffic OK + IP6 MACIP exactMAC|subnetIP ACL bridged traffic OK + IP6 MACIP exactMAC|wildIP ACL bridged traffic OK + IP6 MACIP oui_MAC|exactIP ACL bridged traffic OK + IP6 MACIP ouiMAC|subnetIP ACL bridged traffic OK + IP6 MACIP ouiMAC|wildIP ACL bridged traffic OK + IP6 MACIP wildcardMAC|exactIP ACL bridged traffic OK + IP6 MACIP wildcardMAC|subnetIP ACL bridged traffic OK + IP6 MACIP wildcardMAC|wildIP ACL bridged traffic OK + MACIP replace ACL OK + MACIP replace ACL with IP4 traffic OK + MACIP replace ACL with IP6 traffic OK + IP4 MACIP exactMAC|exactIP ACL routed traffic OK + IP4 MACIP exactMAC|subnetIP ACL routed traffic OK + IP4 MACIP exactMAC|wildIP ACL routed traffic OK + IP4 MACIP ouiMAC|exactIP ACL routed traffic OK + IP4 MACIP ouiMAC|subnetIP ACL routed traffic OK + IP4 MACIP ouiMAC|wildIP ACL routed traffic OK + IP4 MACIP wildcardMAC|exactIP ACL routed traffic OK + IP4 MACIP wildcardMAC|subnetIP ACL routed traffic OK + IP4 MACIP wildcardMAC|wildIP ACL OK + IP6 MACIP exactMAC|exactIP ACL routed traffic OK + IP6 MACIP exactMAC|subnetIP ACL routed traffic OK + IP6 MACIP exactMAC|wildIP ACL routed traffic OK + IP6 MACIP ouiMAC|exactIP ACL routed traffic OK + IP6 MACIP ouiMAC|subnetIP ACL routed traffic OK + IP6 MACIP ouiMAC|wildIP ACL routed traffic OK + IP6 MACIP wildcardMAC|exactIP ACL routed traffic OK + IP6 MACIP wildcardMAC|subnetIP ACL routed traffic OK + IP6 MACIP wildcardMAC|wildIP ACL OK + MACIP ACL delete intf with acl OK + +SRv6 Tests +`````````` +:: + + ============================================================================== + SRv6 Test Case + ============================================================================== + Test SRv6 End (without PSP) behavior. OK + Test SRv6 End.DT4 behavior. OK + Test SRv6 End.DT6 behavior. OK + Test SRv6 End.DX2 behavior. OK + Test SRv6 End.DX4 behavior. OK + Test SRv6 End.DX6 behavior. OK + Test SRv6 End.X (without PSP) behavior. OK + Test SRv6 End.X with PSP behavior. OK + Test SRv6 End with PSP behavior. OK + Test SRv6 Transit.Encaps behavior for IPv6. OK + Test SRv6 Transit.Encaps behavior for IPv4. OK + Test SRv6 Transit.Encaps behavior for L2. SKIP + Test SRv6 Transit.Insert behavior (IPv6 only). OK + +PPPoE Tests +``````````` +:: + + ============================================================================== + PPPoE Test Case + ============================================================================== + PPPoE Add Same Session Twice Test OK + PPPoE Decap Test OK + PPPoE Decap Multiple Sessions Test OK + PPPoE Delete Same Session Twice Test OK + PPPoE Encap Test OK + PPPoE Encap Multiple Sessions Test OK + +VTR VLAN Tag Rewrite Tests +`````````````````````````` +:: + + ============================================================================== + VTR Test Case + ============================================================================== + 1AD VTR pop 1 test OK + 1AD VTR pop 2 test OK + 1AD VTR push 1 1AD test OK + 1AD VTR push 1 1Q test OK + 1AD VTR push 2 1AD test OK + 1AD VTR push 2 1Q test OK + 1AD VTR translate 1 -> 1 1AD test OK + 1AD VTR translate 1 -> 1 1Q test OK + 1AD VTR translate 1 -> 2 1AD test OK + 1AD VTR translate 1 -> 2 1Q test OK + 1AD VTR translate 2 -> 1 1AD test OK + 1AD VTR translate 2 -> 1 1Q test OK + 1AD VTR translate 2 -> 2 1AD test OK + 1AD VTR translate 2 -> 2 1Q test OK + 1Q VTR pop 1 test OK + 1Q VTR push 1 test OK + 1Q VTR push 2 test OK + 1Q VTR translate 1 -> 1 test OK + 1Q VTR translate 1 -> 2 test OK + +ARP Tests +````````` +:: + + ============================================================================== + ARP Test Case + ============================================================================== + ARP OK + ARP Duplicates OK + ARP Static OK + ARP reply with VRRP virtual src hw addr OK + MPLS OK + Proxy ARP OK + Interface Mirror Proxy ARP OK + ============================================================================== + L2BD arp termination Test Case + ============================================================================== + L2BD arp term - add 5 hosts, verify arp responses OK + L2BD arp term - delete 3 hosts, verify arp responses OK + L2BD arp term - recreate BD1, readd 3 hosts, verify arp responses OK + L2BD arp term - 2 IP4 addrs per host OK + L2BD arp term - create and update 10 IP4-mac pairs OK + L2BD arp/ND term - hosts with both ip4/ip6 OK + L2BD ND term - Add and Del hosts, verify ND replies OK + L2BD ND term - Add and update IP+mac, verify ND replies OK + L2BD arp term - send garps, verify arp event reports OK + L2BD arp term - send duplicate garps, verify suppression OK + L2BD arp term - disable ip4 arp events,send garps, verify no events OK + L2BD ND term - send NS packets verify reports OK + L2BD ND term - send duplicate ns, verify suppression OK + L2BD ND term - disable ip4 arp events,send ns, verify no events OK + +P2P Ethernet Subinterface ests +`````````````````````````````` +:: + + ============================================================================== + P2P Ethernet tests + ============================================================================== + delete/create p2p subif OK + create 100k of p2p subifs SKIP + create 1k of p2p subifs OK + ============================================================================== + P2P Ethernet IPv4 tests + ============================================================================== + receive ipv4 packet via p2p subinterface OK + route rx packet not matching p2p subinterface OK + send ip4 packet via p2p subinterface OK + drop tx ip4 packet not matching p2p subinterface OK + ============================================================================== + P2P Ethernet IPv6 tests + ============================================================================== + receive ipv6 packet via p2p subinterface OK + drop rx packet not matching p2p subinterface OK + route rx ip6 packet not matching p2p subinterface OK + send packet via p2p subinterface OK + drop tx ip6 packet not matching p2p subinterface OK + standard routing without p2p subinterfaces OK + +GTPU Test Case +`````````````` +:: + + ============================================================================== + GTPU Test Case + ============================================================================== + Decapsulation test OK + Encapsulation test OK + Multicast flood test OK + Multicast receive test OK + Unicast flood test OK + +Other Tests +``````````` +:: + + ============================================================================== + VAPI test + ============================================================================== + run C VAPI tests SKIP + run C++ VAPI tests SKIP + ============================================================================== + Ping Test Case + ============================================================================== + basic ping test OK + burst ping test OK + ============================================================================== + Template verification, timer tests + ============================================================================== + timer less than template timeout OK + timer greater than template timeout OK + verify cflow packet fields OK + ============================================================================== + JVPP Core Test Case + ============================================================================== + JVPP Acl Callback Api Test Case OK + JVPP Acl Future Api Test Case OK + JVPP Core Callback Api Test Case OK + JVPP Core Future Api Test Case OK + JVPP Ioamexport Callback Api Test Case OK + JVPP Ioamexport Future Api Test Case OK + JVPP Ioampot Callback Api Test Case OK + JVPP Ioampot Future Api Test Case OK + JVPP Ioamtrace Callback Api Test Case OK + JVPP Ioamtrace Future Api Test Case OK + JVPP Snat Callback Api Test Case OK + JVPP Snat Future Api Test Case OK \ No newline at end of file diff --git a/docs/report/introduction/general_notes.rst b/docs/report/introduction/general_notes.rst index 0ddeb6a569..461e186f6d 100644 --- a/docs/report/introduction/general_notes.rst +++ b/docs/report/introduction/general_notes.rst @@ -15,32 +15,33 @@ FD.io CSIT project currently covers multiple FD.io system and sub-system testing areas and this is reflected in this report, where each testing area is listed separately, as follows: -#. **VPP Performance Tests** - VPP performance tests are executed in physical +#. **VPP - Performance** - VPP benchmarking tests are executed in physical FD.io testbeds, focusing on VPP network data plane performance at this stage, both for Phy-to-Phy (NIC-to-NIC) and Phy-to-VM-to-Phy (NIC-to-VM-to-NIC) forwarding topologies. Tested across a range of NICs, 10GE and 40GE interfaces, range of multi-thread and multi-core configurations. VPP application runs in host user-mode. TRex is used as a traffic generator. -#. **Container memif connections** - VPP memif virtual interface (shared memory - interface) tests to interconnect VPP instances. VPP vswitch instance runs in - bare-metal user-mode handling Intel x520 NIC 10GbE interfaces and connecting - over memif (Master side) virtual interfaces to more instances of VPP running - in LXC or in Docker Containers, both with memif virtual interfaces (Slave - side). Tested across a range of NICs, 10GE and 40GE interfaces, range of - multi-thread and multi-core configurations. VPP application runs in host - user-mode. TRex is used as a traffic generator. +#. **LXC and Docker Containers VPP memif - Performance** - VPP memif + virtual interface tests interconnect multiple VPP instances running in + containers. VPP vswitch instance runs in bare-metal user-mode + handling Intel x520 NIC 10GbE interfaces and connecting over memif + (Master side) virtual interfaces to more instances of VPP running in + LXC or in Docker Containers, both with memif virtual interfaces (Slave + side). Tested across a range of multi-thread and multi-core + configurations. TRex is used as a traffic generator. -#. **Container Orchestrated Performance Tests** - CSIT |release| introduced new - tests of Container topologies connected over the memif virtual interface - (shared memory interface). For these tests VPP vswitch instance runs in a - Docker Container handling Intel x520 NIC 10GbE interfaces and connecting over - memif (Master side) virtual interfaces to more instances of VPP running in - Docker Containers with memif virtual interfaces (Slave side). Tested across a - range of multi-thread and multi-core configurations. VPP application runs in - host user-mode. TRex is used as a traffic generator. +#. **Container Topologies Orchestrated by K8s - Performance** - CSIT + |release| introduced new tests of Container topologies connected over + the memif virtual interface (shared memory interface). For these tests + VPP vswitch instance runs in a Docker Container handling Intel x520 NIC + 10GbE interfaces and connecting over memif (Master side) virtual + interfaces to more instances of VPP running in Docker Containers with + memif virtual interfaces (Slave side). All containers are orchestrated + by Kubernetes, with Ligato for container networking. TRex is used as a + traffic generator. -#. **DPDK Performance Tests** - VPP is using DPDK code to control and drive +#. **DPDK Performance** - VPP is using DPDK code to control and drive the NICs and physical interfaces. Testpmd tests are used as a baseline to profile the DPDK sub-system of VPP. DPDK performance tests executed in physical FD.io testbeds, focusing on Testpmd/L3FWD data plane performance for @@ -49,23 +50,23 @@ is listed separately, as follows: Testpmd/L3FWD application runs in host user-mode. TRex is used as a traffic generator. -#. **VPP Functional Tests** - VPP functional tests are executed in virtual +#. **Honeycomb Performance** - Honeycomb performance tests are executed in + physical FD.io testbeds, focusing on the performance of Honeycomb management + and programming functionality of VPP. Tests cover a range of CRUD operations + executed against VPP. + +#. **VPP Functional** - VPP functional tests are executed in virtual FD.io testbeds focusing on VPP packet processing functionality, including network data plane and in -line control plane. Tests cover vNIC-to-vNIC vNIC-to-VM-to-vNIC forwarding topologies. Scapy is used as a traffic generator. -#. **Honeycomb Functional Tests** - Honeycomb functional tests are executed in +#. **Honeycomb Functional** - Honeycomb functional tests are executed in virtual FD.io testbeds, focusing on Honeycomb management and programming functionality of VPP. Tests cover a range of CRUD operations executed against VPP. -#. **Honeycomb Performance Tests** - Honeycomb performance tests are executed in - physical FD.io testbeds, focusing on the performance of Honeycomb management - and programming functionality of VPP. Tests cover a range of CRUD operations - executed against VPP. - -#. **NSH_SFC Functional Tests** - NSH_SFC functional tests are executed in +#. **NSH_SFC Functional** - NSH_SFC functional tests are executed in virtual FD.io testbeds focusing on NSH_SFC of VPP. Tests cover a range of CRUD operations executed against VPP. diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst index 536d5d3cf1..22bc29f071 100644 --- a/docs/report/introduction/overview.rst +++ b/docs/report/introduction/overview.rst @@ -44,6 +44,13 @@ CSIT |release| report contains following main sections and sub-sections: executions; *Test Environment* - environment description; *Documentation* - CSIT source code documentation for DPDK performance tests. +#. **Honeycomb Performance Tests** - Honeycomb performance tests executed in + physical FD.io testbeds; *Overview* - tested topologies, test coverage + and naming specifics; *CSIT Release Notes* - changes in CSIT |release|, + added tests, environment or methodology changes, known CSIT issues; + *Test Environment* - environment description; *Documentation* - source + code documentation for Honeycomb performance tests. + #. **VPP Functional Tests** - VPP functional tests executed in virtual FD.io testbeds; *Overview* - tested virtual topologies, test coverage and naming specifics; *CSIT Release Notes* - changes in CSIT |release|, added diff --git a/docs/report/vpp_unit_tests/overview.rst b/docs/report/vpp_unit_tests/overview.rst index cc32ee4f3d..90c582c943 100644 --- a/docs/report/vpp_unit_tests/overview.rst +++ b/docs/report/vpp_unit_tests/overview.rst @@ -33,40 +33,42 @@ Unit Tests Coverage ------------------- Following VPP functional test areas are covered in VPP unit test code included -in VPP rls1704 with results listed in this report: +in VPP rls1710 with results listed in this report: +- ARP - ARP, Proxy ARP. +- ACL plugin - stateful and stateless security-groups access-control-lists. +- BFD IPv4 - Bidirectional Forwarding Detection - baseline, APIs, authorization, authentication. +- BFD IPv6 - Bidirectional Forwarding Detection - baseline, APIs, authorization, authentication. +- Classifier - classification with IP ACL, MAC ACL, IP PBR. - CRUD Loopback - create, read, update, delete Loopback interfaces. -- Flow-per-packet plugin - collect and report L2 and IP4 flow statistics. +- Deterministic NAT - Carrier Grade NAT tests. - DHCP - DHCPv4 and DHCPv6 Proxy. +- FIB - baseline and scale tests. +- Flow-per-packet plugin - collect and report L2 and IP4 flow statistics. +- Flowprobe tests. +- GRE - GRE IPv4/IPv6 tunnel, L2, VRF tests. +- GTPU - baseline GTPU tests. +- IP Multicast - IPv4/IPv6 multicast replication, connected source check. - IP4 VRF Multi-instance - create, read, update, delete and verify IPv4 VRFs. -- Load Balancer - IP4 GRE4, IP4 GRE6, IP6 GRE4, IP6 GRE6. -- FIB Unit Tests. -- Bidirectional Forwarding Detection (BFD) IPv4 - baseline BFD session operation. -- Bidirectional Forwarding Detection (BFD) IPv6 - baseline BFD session operation. -- Bidirectional Forwarding Detection (BFD) - API tests. -- Bidirectional Forwarding Detection (BFD) - changing authorization. -- Bidirectional Forwarding Detection (BFD) - SHA1 authentication. -- IPv6 Tests - baseline FIB operations, NS/RS exception handling. -- SPAN Test - Switched Port Analyzer packet mirroring. -- GRE Tests - GRE tunnel tests. -- SNAT Test Cases - SNAT44 tests. -- Deterministic NAT Test Cases - Carrier Grade NAT tests. -- NAT64 Test Cases - NAT64 static and dynamic translation tests. -- L2XC Multi-instance - L2 cross-connect multi-instance tests. -- IPv4 Tests - baseline FIB tests. +- IP6 VRF Multi-instance - create, read, update, delete and verify IPv6 VRFs. +- IPv4 - baseline FIB tests. - IPv4 FIB CRUD - add/update/delete IPv4 routes. -- IRB Tests - Integrated Routing and Bridging tests. -- ACL plugin - stateful and stateless security-groups access-control-lists. -- MPLS Tests - MPLS baseline tests. +- IPv6 - baseline FIB operations, NS/RS exception handling. +- IRB - Integrated Routing and Bridging tests. +- L2 FIB CRUD - add/update/delete L2 MAC entries. +- L2BD - L2 Bridge-Domain baseline tests incl. single- and dual-loop. +- L2XC - L2 cross-connect baseline tests incl. single- and dual-loop. +- L2XC Multi-instance - L2 cross-connect multi-instance tests. +- LISP - basic LISP tests. +- Load Balancer - IP4 GRE4, IP4 GRE6, IP6 GRE4, IP6 GRE6. +- MACIP - ingress access control for IPv4, IPv6 with L2BDP and IP routing. +- MFIB Unit. +- MPLS - MPLS baseline tests. - MPLS PIC edge convergence - prefix independent convergence tests for MPLS PE. -- L2XC Tests - L2 cross-connect baseline tests incl. single- and dual-loop. -- MFIB Unit Tests. -- IP Multicast Tests - IPv4/IPv6 multicast replication, connected source check. -- Classifier - classification with IP ACL, MAC ACL, IP PBR. -- VXLAN Tests - baseline VXLAN tunneling. -- VXLAN-GPE Test Cases - baseline VXLAN-GPE tunneling tests including multicast. +- NAT44 - NAT44 tests, IPFIX logging, VRF awareness. +- NAT64 - NAT64 static and dynamic translation tests. +- SPAN - Switched Port Analyzer packet mirroring. +- SRv6 - Segment Routing IPv6 tests. - VTR Test Case - VLAN tag manipulation tests. -- L2 FIB CRUD - add/update/delete L2 MAC entries. -- L2BD Tests - L2 Bridge-Domain baseline tests incl. single- and dual-loop. -- Flowprobe tests. -- GTPU Test Cases - baseline GTPU tests. +- VXLAN - baseline VXLAN tunneling. +- VXLAN-GPE - baseline VXLAN-GPE tunneling tests including multicast. -- 2.16.6