4 Tested Physical Topologies
5 --------------------------
7 CSIT VPP performance tests are executed on physical baremetal servers hosted by
8 LF FD.io project. Testbed physical topology is shown in the figure below.::
10 +------------------------+ +------------------------+
12 | +------------------+ | | +------------------+ |
14 | | <-----------------> | |
15 | | DUT1 | | | | DUT2 | |
16 | +--^---------------+ | | +---------------^--+ |
19 +------------------------+ +------------------^-----+
24 +------------------> TG <------------------+
28 SUT1 runs VPP SW application in Linux user-mode as a
29 Device Under Test (DUT), and a python script to generate traffic. SUT2 and TG
31 sical connectivity between SUTs and to TG is provided using
32 different NIC model. Currently installed NIC models include:
34 Performance tests involve sending Netconf requests over localhost to the
35 Honeycomb listener port, and measuring response time.
37 Note that reported performance results are specific to the SUTs tested.
38 Current LF FD.io SUTs are based on Intel XEON E5-2699v3 2.3GHz CPUs. SUTs with
39 other CPUs are likely to yield different results.
41 For detailed LF FD.io test bed specification and physical topology please refer
42 to `LF FDio CSIT testbed wiki page
43 <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
45 Performance Tests Coverage
46 --------------------------
48 As of right now, there is only a single Honeycomb performance test. Measuring
49 response time for a simple read operation, performed synchronously and using
50 single (not batch) requests.
52 Currently the tests do not trigger automatically, but can be run on-demand from
55 Performance Tests Naming
56 ------------------------
58 CSIT |release| follows a common structured naming convention for all
59 performance and system functional tests, introduced in CSIT |release-1|.
61 The naming should be intuitive for majority of the tests. Complete
62 description of CSIT test naming convention is provided on `CSIT test naming wiki
63 <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
65 Here few illustrative examples of the new naming usage for performance test
68 #. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P**
70 - *PortNICConfig-WireEncapsulation-PacketForwardingFunction-
71 PacketProcessingFunction1-...-PacketProcessingFunctionN-TestType*
72 - *10ge2p1x520-dot1q-l2bdbasemaclrn-ndrdisc.robot* => 2 ports of 10GE on
73 Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain baseline switching
74 with MAC learning, NDR throughput discovery.
75 - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrchk.robot* => 2 ports of 10GE
76 on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain baseline
77 switching with MAC learning, NDR throughput discovery.
78 - *10ge2p1x520-ethip4-ip4base-ndrdisc.robot* => 2 ports of 10GE on Intel
79 x520 NIC, IPv4 baseline routed forwarding, NDR throughput discovery.
80 - *10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot* => 2 ports of 10GE on
81 Intel x520 NIC, IPv6 scaled up routed forwarding, NDR throughput
84 #. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC,
85 P2V2P, NIC2VMchain2NIC, P2V2V2P**
87 - *PortNICConfig-WireEncapsulation-PacketForwardingFunction-
88 PacketProcessingFunction1-...-PacketProcessingFunctionN-VirtEncapsulation-
89 VirtPortConfig-VMconfig-TestType*
90 - *10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 ports
91 of 10GE on Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain
92 switching to/from two vhost interfaces and one VM, NDR throughput
94 - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2
95 ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain
96 switching to/from two vhost interfaces and one VM, NDR throughput
98 - *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc.robot* => 2
99 ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain
100 switching to/from four vhost interfaces and two VMs, NDR throughput
103 Methodology: Multi-Core
104 -----------------------
106 **Multi-core Test** - CSIT |release| multi-core tests are executed in the
107 following thread and core configurations:
109 #. 1t - 1 Honeycomb Netconf thread on 1 CPU physical core.
110 #. 8t - 8 Honeycomb Netconf thread on 8 CPU physical core.
111 #. 16t - 16 Honeycomb Netconf thread on 16 CPU physical core.
113 Traffic generator also uses multiple threads/cores, to simulate multiple
114 Netconf clients accessing the Honeycomb server.
116 Methodology: Performance measurement
117 ------------------------------------
119 The following values are measured and reported in tests:
121 - Average request rate. Averaged over the entire test duration, over all client
122 threads. Negative replies (if any) are not counted and are reported separately.