1 ## FD.io CSIT Testbeds - Xeon Haswell
3 This is a low-level design implemented as an original FD.io CSIT lab based on
4 Cisco UCS-c240m4 servers based on Intel Xeon Haswell processors.
6 ### Server Hardware Configuration
8 Total of 10 Cisco UCS-c240 servers with Intel Xeon Haswell processors have been
9 used to built FD.io CSIT 3-Node Haswell (3n-hsw) performance testbeds and VIRL
10 functional testbeds. Following three HW configuration types of UCS x86 servers
14 1. Type-1: Purpose - VPP functional and performance conformance testing.
15 - Quantity: 6 computers as SUT hosts (Systems Under Test).
16 - Physical connectivity:
17 - CIMC and host management ports.
18 - NIC ports connected in 3-node topologies.
19 - Main HW configuration:
20 - Chassis: UCSC-C240-M4SX with 6 PCIe3.0 slots.
21 - Processors: 2* E5-2699v3 2.3 GHz.
22 - RAM Memory: 16* 32GB DDR4-2133MHz.
23 - Disks: 2* 2TB 12G SAS 7.2K RPM SFF HDD.
25 - Right PCIe Riser Board (Riser 1) (x8, x8, x8 PCIe3.0 lanes)
26 - PCIe Slot1: Cisco VIC 1385 2p40GE.
27 - PCIe Slot2: Intel NIC x520 2p10GE.
29 - Left PCIe Riser Board (Riser 2) (x8, x16, x8 PCIe3.0 lanes)
30 - PCIe Slot4: Intel NIC xl710 2p40GE.
31 - PCIe Slot5: Intel NIC x710 2p10GE.
32 - PCIe Slot6: Intel QAT 8950 50G (Walnut Hill)
33 - MLOM slot: Cisco VIC 1227 2p10GE (x8 PCIe2.0 lanes).
34 2. Type-2: Purpose - VPP functional and performance conformance testing.
35 - Quantity: 3 computers as TG hosts (Traffic Generators).
36 - Physical connectivity:
37 - CIMC and host management ports.
38 - NIC ports connected in 3-node topologies.
39 - Main HW configuration:
40 - Chassis: UCSC-C240-M4SX with 6 PCIe3.0 slots.
41 - Processors: 2* E5-2699v3 2.3 GHz.
42 - RAM Memory: 16* 32GB DDR4-2133MHz.
43 - Disks: 2* 2TB 12G SAS 7.2K RPM SFF HDD.
45 - Right PCIe Riser Board (Riser 1) (x8, x8, x8 lanes)
46 - PCIe Slot1: Intel NIC xl710 2p40GE.
47 - PCIe Slot2: Intel NIC x710 2p10GE.
48 - PCIe Slot3: Intel NIC x710 2p10GE.
49 - Left PCIe Riser Board (Riser 2) (x8, x16, x8 lanes)
50 - PCIe Slot4: Intel NIC xl710 2p40GE.
51 - PCIe Slot5: Intel NIC x710 2p10GE.
52 - PCIe Slot6: Intel NIC x710 2p10GE.
54 3. Type-3: Purpose - VIRL functional conformance.
55 - Quantity: 3 computers as VIRL hosts.
56 - Physical connectivity:
57 - CIMC and host management ports.
58 - no NIC ports, standalone setup.
59 - Main HW configuration:
60 - Chassis: UCSC-C240-M4SX with 6 PCIe3.0 slots.
61 - Processors: 2* E5-2699v3 2.3 GHz.
62 - RAM Memory: 16* 32GB DDR4-2133MHz.
63 - Disks: 2* 480 GB 2.5inch 6G SATA SSD.
65 - Right PCIe Riser Board (Riser 1) (x8, x8, x8 lanes)
67 - Left PCIe Riser Board (Riser 2) (x8, x16, x8 lanes)
72 ## Testbeds1,2,3 Connectivity
76 Total of 10 UCSC-C240-M4SX servers is made available for FD.IO CSIT testbed. For
77 management purposes, each server must have following two ports connected to the
82 - CIMC - Cisco Integrated Management Controller.
83 - Required for provides embedded server management with WebUI, CLI, SNMPv3,
85 2. 1GE management port
86 - hostOS management port.
91 Following naming convention is used within this page to specify physical
92 connectivity and wiring across defined CSIT testbeds:
95 - testbedname: testbedN.
97 - traffic-generator: tN-tgW.
98 - system-under-testX: tN-sutX.
103 - N - testbed number.
104 - tgW - server acts as traffic-generator with W index.
105 - sutX - server acts as system-under-test with X index.
106 - Y - PCIe slot number denoting a NIC card number within the host.
107 - Y=1,2,3 - slots in Riser 1, Right PCIe Riser Board, NUMA node 0.
108 - Y=4,5,6 - slots in Riser 2, Left PCIe Riser Board, NUMA node 1.
109 - Y=m - the MLOM slot.
110 - Z - port number on the NIC card.
113 ### 3-node Topology Testbeds for Performance
115 Nine servers are used to build three of 3-node topologies, with each topology
116 using two servers of Type-1 (SUT function) and one server of Type-2
117 (TG function). Server NIC cards are placed and NIC ports are connected using the
118 scheme defined in next sections.
120 ### LOM (CIMC) and Management networks
122 Each server has a LOM (Lights-Out-Management e.g. Cisco CIMC) and a Management
123 port, which are connected to two different VLANs.
127 - Subnet: 10.30.50.0/24
128 - Gateway: 10.30.50.1
129 - Broadcast: 10.30.50.255
130 - DNS1: 199.204.44.24
131 - DNS2: 199.204.47.54
133 - Subnet: 10.30.51.0/24
134 - Gateway: 10.30.51.1
135 - Broadcast: 10.30.51.255
136 - DNS1: 199.204.44.24
137 - DNS2: 199.204.47.54
139 To access these hosts, an VPN connection is required.
141 #### LOM (CIMC) VLAN IP Addresses allocation
147 10.30.50.2 | LF Reserved
148 10.30.50.3 | LF Reserved
149 10.30.50.4 | LF Reserved
150 10.30.50.5 | LF Reserved
151 10.30.50.6 | LF Reserved
152 10.30.50.7 | LF Reserved
153 10.30.50.8 | LF Reserved
154 10.30.50.9 | LF Reserved
155 10.30.50.10 | LF Reserved
156 10.30.50.11 | LF Reserved
157 10.30.50.12 | LF Reserved
158 10.30.50.13 | LF Reserved
159 10.30.50.14 | LF Reserved
160 10.30.50.15 | LF Reserved
162 10.30.50.17 | t1-sut1
163 10.30.50.18 | t1-sut2
165 10.30.50.21 | t2-sut1
166 10.30.50.22 | t2-sut2
168 10.30.50.25 | t3-sut1
169 10.30.50.26 | t3-sut-2
170 10.30.50.28 | t4-sut1
171 10.30.50.29 | t4-sut2
172 10.30.50.30 | t4-sut3
173 10.30.50.255 | Broadcast
175 #### Management VLAN IP Addresses allocation
181 10.30.51.2 | LF Reserved
182 10.30.51.3 | LF Reserved
183 10.30.51.4 | LF Reserved
184 10.30.51.5 | LF Reserved
185 10.30.51.6 | LF Reserved
186 10.30.51.7 | LF Reserved
187 10.30.51.8 | LF Reserved
188 10.30.51.9 | netgate-1
189 10.30.51.10 | netgate-2
190 10.30.51.11 | netgate-3
191 10.30.51.12 | softiron-1
192 10.30.51.13 | softiron-2
193 10.30.51.14 | softiron-3
194 10.30.51.15 | LF Reserved
196 10.30.51.17 | t1-sut1
197 10.30.51.18 | t1-sut2
199 10.30.51.21 | t2-sut1
200 10.30.51.22 | t2-sut2
202 10.30.51.25 | t3-sut1
203 10.30.51.26 | t3-sut-2
204 10.30.51.28 | t4-sut1
205 10.30.51.29 | t4-sut2
206 10.30.51.30 | t4-sut3
207 10.30.51.31-10.30.51.105 | VIRL1
208 10.30.51.106-10.30.51.180 | VIRL2
209 10.30.51.181-10.30.51.254 | VIRL3
210 10.30.51.255 | Broadcast
212 ### Testbeds1,2,3 Naming: Servers, Ports
214 Each server in 3-node Topology has its NIC cards placed, and NIC cards and ports
215 indexed using defined naming convention:
218 1. Server1 of Type-1:
219 - testbedname: testbed1.
221 - CIMC IP: 10.30.50.17
222 - Host IP: 10.30.51.17
224 - t1-sut1-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE.
225 - t1-sut1-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE.
226 - t1-sut1-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE.
227 - t1-sut1-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE.
228 - t1-sut1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
229 - t1-sut1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
230 - t1-sut1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE.
231 - t1-sut1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE.
232 - t1-sut1-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE.
233 - t1-sut1-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE.
234 2. Server2 of Type-1:
235 - testbedname: testbed1.
237 - CIMC IP: 10.30.50.18
238 - Host IP: 10.30.51.18
240 - t1-sut2-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE.
241 - t1-sut2-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE.
242 - t1-sut2-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE.
243 - t1-sut2-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE.
244 - t1-sut2-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
245 - t1-sut2-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
246 - t1-sut2-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE.
247 - t1-sut2-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE.
248 - t1-sut2-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE.
249 - t1-sut2-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE.
250 3. Server3 of Type-2:
251 - testbedname: testbed1.
253 - CIMC IP: 10.30.50.16
254 - Host IP: 10.30.51.16
256 - t1-tg1-c1/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
257 - t1-tg1-c1/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
258 - t1-tg1-c2/p1 - 10GE port1 on Intel NIC x710 2p10GE.
259 - t1-tg1-c2/p2 - 10GE port2 on Intel NIC x710 2p10GE.
260 - t1-tg1-c3/p1 - 10GE port1 on Intel NIC x710 2p10GE.
261 - t1-tg1-c3/p2 - 10GE port2 on Intel NIC x710 2p10GE.
262 - t1-tg1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
263 - t1-tg1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
264 - t1-tg1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE.
265 - t1-tg1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE.
266 - t1-tg1-c6/p1 - 10GE port1 on Intel NIC x710 2p10GE.
267 - t1-tg1-c6/p2 - 10GE port2 on Intel NIC x710 2p10GE.
268 4. Server4 of Type-1:
269 - testbedname: testbed2.
271 - CIMC IP: 10.30.50.21
272 - Host IP: 10.30.51.21
274 - t2-sut1-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE.
275 - t2-sut1-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE.
276 - t2-sut1-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE.
277 - t2-sut1-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE.
278 - t2-sut1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
279 - t2-sut1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
280 - t2-sut1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE.
281 - t2-sut1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE.
282 - t2-sut1-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE.
283 - t2-sut1-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE.
284 5. Server5 of Type-1:
285 - testbedname: testbed2.
287 - CIMC IP: 10.30.50.22
288 - Host IP: 10.30.51.22
290 - t2-sut2-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE.
291 - t2-sut2-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE.
292 - t2-sut2-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE.
293 - t2-sut2-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE.
294 - t2-sut2-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
295 - t2-sut2-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
296 - t2-sut2-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE.
297 - t2-sut2-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE.
298 - t2-sut2-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE.
299 - t2-sut2-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE.
300 6. Server6 of Type-2:
301 - testbedname: testbed2.
303 - CIMC IP: 10.30.50.20
304 - Host IP: 10.30.51.20
306 - t2-tg1-c1/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
307 - t2-tg1-c1/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
308 - t2-tg1-c2/p1 - 10GE port1 on Intel NIC x710 2p10GE.
309 - t2-tg1-c2/p2 - 10GE port2 on Intel NIC x710 2p10GE.
310 - t2-tg1-c3/p1 - 10GE port1 on Intel NIC x710 2p10GE.
311 - t2-tg1-c3/p2 - 10GE port2 on Intel NIC x710 2p10GE.
312 - t2-tg1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
313 - t2-tg1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
314 - t2-tg1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE.
315 - t2-tg1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE.
316 - t2-tg1-c6/p1 - 10GE port1 on Intel NIC x710 2p10GE.
317 - t2-tg1-c6/p2 - 10GE port2 on Intel NIC x710 2p10GE.
318 7. Server7 of Type-1:
319 - testbedname: testbed3.
321 - CIMC IP: 10.30.50.25
322 - Host IP: 10.30.51.25
324 - t3-sut1-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE.
325 - t3-sut1-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE.
326 - t3-sut1-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE.
327 - t3-sut1-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE.
328 - t3-sut1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
329 - t3-sut1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
330 - t3-sut1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE.
331 - t3-sut1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE.
332 - t3-sut1-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE.
333 - t3-sut1-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE.
334 8. Server8 of Type-1:
335 - testbedname: testbed3.
337 - CIMC IP: 10.30.50.26
338 - Host IP: 10.30.51.26
340 - t3-sut2-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE.
341 - t3-sut2-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE.
342 - t3-sut2-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE.
343 - t3-sut2-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE.
344 - t3-sut2-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
345 - t3-sut2-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
346 - t3-sut2-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE.
347 - t3-sut2-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE.
348 - t3-sut2-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE.
349 - t3-sut2-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE.
350 9. Server9 of Type-2:
351 - testbedname: testbed3.
353 - CIMC IP: 10.30.50.24
354 - Host IP: 10.30.51.24
356 - t3-tg1-c1/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
357 - t3-tg1-c1/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
358 - t3-tg1-c2/p1 - 10GE port1 on Intel NIC x710 2p10GE.
359 - t3-tg1-c2/p2 - 10GE port2 on Intel NIC x710 2p10GE.
360 - t3-tg1-c3/p1 - 10GE port1 on Intel NIC x710 2p10GE.
361 - t3-tg1-c3/p2 - 10GE port2 on Intel NIC x710 2p10GE.
362 - t3-tg1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE.
363 - t3-tg1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE.
364 - t3-tg1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE.
365 - t3-tg1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE.
366 - t3-tg1-c6/p1 - 10GE port1 on Intel NIC x710 2p10GE.
367 - t3-tg1-c6/p2 - 10GE port2 on Intel NIC x710 2p10GE.
370 ### Testbeds1,2,3 Network Connectivity
372 Listed nine servers are connected into the three of 3-node testbeds, testbed1,
373 testbed2 and testbed3, using defined naming convention as follows:
377 - ring of 40GE ports on Cisco VIC 1385 2p40GE on SUTs
378 - t1-tg1-c1/p1 to t1-sut1-c1/p2.
379 - t1-sut1-c1/p1 to t1-sut2-c1/p2.
380 - t1-sut2-c1/p1 to t1-tg1-c1/p2.
381 - ring of 10GE ports on Intel NIC x520 2p10GE on SUTs
382 - t1-tg1-c2/p1 to t1-sut1-c2/p2.
383 - t1-sut1-c2/p1 to t1-sut2-c2/p2.
384 - t1-sut2-c2/p1 to t1-tg1-c2/p2.
385 - ring of 40GE ports on Intel NIC xl710 2p40GE on SUTs
386 - t1-tg1-c4/p1 to t1-sut1-c4/p2.
387 - t1-sut1-c4/p1 to t1-sut2-c4/p2.
388 - t1-sut2-c4/p1 to t1-tg1-c4/p2.
389 - ring of 10GE ports on Intel NIC x710 2p10GE on SUTs
390 - t1-tg1-c5/p1 to t1-sut1-c5/p2.
391 - t1-sut1-c5/p1 to t1-sut2-c5/p2.
392 - t1-sut2-c5/p1 to t1-tg1-c5/p2.
393 - ring of 10GE ports on Cisco VIC 1227 2p10GE on SUTs
394 - t1-tg1-c2/p1 to t1-sut1-cm/p2.
395 - t1-sut1-cm/p1 to t1-sut2-cm/p2.
396 - t1-sut2-cm/p1 to t1-tg1-c2/p2.
397 - TG loopback ports Intel NIC x710 2p10GE
398 - t1-tg1-c6/p1 to t1-tg1-c6/p2.
401 - ring of 40GE ports on Cisco VIC 1385 2p40GE on SUTs
402 - t2-tg1-c1/p1 to t2-sut1-c1/p2.
403 - t2-sut1-c1/p1 to t2-sut2-c1/p2.
404 - t2-sut2-c1/p1 to t2-tg1-c1/p2.
405 - ring of 10GE ports on Intel NIC x520 2p10GE on SUTs
406 - t2-tg1-c2/p1 to t2-sut1-c2/p2.
407 - t2-sut1-c2/p1 to t2-sut2-c2/p2.
408 - t2-sut2-c2/p1 to t2-tg1-c2/p2.
409 - ring of 40GE ports on Intel NIC xl710 2p40GE on SUTs
410 - t2-tg1-c4/p1 to t2-sut1-c4/p2.
411 - t2-sut1-c4/p1 to t2-sut2-c4/p2.
412 - t2-sut2-c4/p1 to t2-tg1-c4/p2.
413 - ring of 10GE ports on Intel NIC x710 2p10GE on SUTs
414 - t2-tg1-c5/p1 to t2-sut1-c5/p2.
415 - t2-sut1-c5/p1 to t2-sut2-c5/p2.
416 - t2-sut2-c5/p1 to t2-tg1-c5/p2.
417 - ring of 10GE ports on Cisco VIC 1227 2p10GE on SUTs
418 - t2-tg1-c2/p1 to t2-sut1-cm/p2.
419 - t2-sut1-cm/p1 to t2-sut2-cm/p2.
420 - t2-sut2-cm/p1 to t2-tg1-c2/p2.
421 - TG loopback ports Intel NIC x710 2p10GE
422 - t2-tg1-c6/p1 to t2-tg1-c6/p2.
425 - ring of 40GE ports on Cisco VIC 1385 2p40GE on SUTs
426 - t3-tg1-c1/p1 to t3-sut1-c1/p2.
427 - t3-sut1-c1/p1 to t3-sut2-c1/p2.
428 - t3-sut2-c1/p1 to t3-tg1-c1/p2.
429 - ring of 10GE ports on Intel NIC x520 2p10GE on SUTs
430 - t3-tg1-c2/p1 to t3-sut1-c2/p2.
431 - t3-sut1-c2/p1 to t3-sut2-c2/p2.
432 - t3-sut2-c2/p1 to t3-tg1-c2/p2.
433 - ring of 40GE ports on Intel NIC xl710 2p40GE on SUTs
434 - t3-tg1-c4/p1 to t3-sut1-c4/p2.
435 - t3-sut1-c4/p1 to t3-sut2-c4/p2.
436 - t3-sut2-c4/p1 to t3-tg1-c4/p2.
437 - ring of 10GE ports on Intel NIC x710 2p10GE on SUTs
438 - t3-tg1-c5/p1 to t3-sut1-c5/p2.
439 - t3-sut1-c5/p1 to t3-sut2-c5/p2.
440 - t3-sut2-c5/p1 to t3-tg1-c5/p2.
441 - ring of 10GE ports on Cisco VIC 1227 2p10GE on SUTs
442 - t3-tg1-c2/p1 to t3-sut1-cm/p2.
443 - t3-sut1-cm/p1 to t3-sut2-cm/p2.
444 - t3-sut2-cm/p1 to t3-tg1-c2/p2.
445 - TG loopback ports Intel NIC x710 2p10GE
446 - t3-tg1-c6/p1 to t3-tg1-c6/p2.
449 ## Testbeds1,2,3 Server Specifications
456 CPU op-mode(s): 32-bit, 64-bit
457 Byte Order: Little Endian
459 On-line CPU(s) list: 0-35
460 Thread(s) per core: 1
461 Core(s) per socket: 18
464 Vendor ID: GenuineIntel
467 Model name: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
476 NUMA node0 CPU(s): 0-17
477 NUMA node1 CPU(s): 18-35
478 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq
479 dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt
480 cqm_llc cqm_occup_llc dtherm arat pln pts
483 ### Linux dmidecode pci
486 $ dmidecode --type 9 | grep 'Handle\|Slot\|Type\|Address'
487 Handle 0x0046, DMI type 9, 17 bytes
488 System Slot Information
489 Designation: SlotID:1
490 Type: x8 PCI Express 3 x8
491 Bus Address: 0000:0a:02.0
492 Handle 0x0047, DMI type 9, 17 bytes
493 System Slot Information
494 Designation: SlotID:2
495 Type: x8 PCI Express 3 x8
496 Bus Address: 0000:0e:03.2
497 Handle 0x0048, DMI type 9, 17 bytes
498 System Slot Information
499 Designation: SlotID:3
500 Type: x8 PCI Express 3 x8
501 Bus Address: 0000:0d:03.0
502 Handle 0x0049, DMI type 9, 17 bytes
503 System Slot Information
504 Designation: SlotID:4
505 Type: x8 PCI Express 3 x8
506 Bus Address: 0000:85:02.2
507 Handle 0x004A, DMI type 9, 17 bytes
508 System Slot Information
509 Designation: SlotID:5
510 Type: x8 PCI Express 3 x16
511 Bus Address: 0000:87:03.0
512 Handle 0x004B, DMI type 9, 17 bytes
513 System Slot Information
514 Designation: SlotID:6
515 Type: x8 PCI Express 3 x8
516 Bus Address: 0000:84:02.0
517 Handle 0x004C, DMI type 9, 17 bytes
518 System Slot Information
519 Designation: SlotID:MLOM
521 Bus Address: 0000:01:01.0
522 Handle 0x004D, DMI type 9, 17 bytes
523 System Slot Information
524 Designation: SlotID:HBA
526 Bus Address: 0000:0c:02.2
527 Handle 0x004E, DMI type 9, 17 bytes
528 System Slot Information
529 Designation: SlotID:NVMe1
530 Type: x4 PCI Express 3 x4
531 Bus Address: 0000:82:01.0
532 Handle 0x004F, DMI type 9, 17 bytes
533 System Slot Information
534 Designation: SlotID:NVMe2
535 Type: x4 PCI Express 3 x4
536 Bus Address: 0000:83:01.1
537 Handle 0x0050, DMI type 9, 17 bytes
538 System Slot Information
539 Designation: SlotID:2.1
540 Type: x16 PCI Express 3 x16
541 Bus Address: 0000:00:1f.7
542 Handle 0x0051, DMI type 9, 17 bytes
543 System Slot Information
544 Designation: SlotID:2.2
545 Type: x16 PCI Express 3 x16
546 Bus Address: 0000:00:1f.7
547 Handle 0x0052, DMI type 9, 17 bytes
548 System Slot Information
549 Designation: SlotID:2.3
550 Type: x16 PCI Express 3 x16
551 Bus Address: 0000:00:1f.7
552 Handle 0x0053, DMI type 9, 17 bytes
553 System Slot Information
554 Designation: SlotID:2.4
555 Type: x16 PCI Express 3 x16
556 Bus Address: 0000:00:1f.7
557 Handle 0x0054, DMI type 9, 17 bytes
558 System Slot Information
559 Designation: SlotID:2.5
560 Type: x16 PCI Express 3 x16
561 Bus Address: 0000:00:1f.7
562 Handle 0x0055, DMI type 9, 17 bytes
563 System Slot Information
564 Designation: SlotID:5.1
565 Type: x8 PCI Express 3 x16
566 Bus Address: 0000:00:1f.7
567 Handle 0x0056, DMI type 9, 17 bytes
568 System Slot Information
569 Designation: SlotID:5.2
570 Type: x8 PCI Express 3 x16
571 Bus Address: 0000:00:1f.7
572 Handle 0x0057, DMI type 9, 17 bytes
573 System Slot Information
574 Designation: SlotID:5.3
575 Type: x8 PCI Express 3 x16
576 Bus Address: 0000:00:1f.7
577 Handle 0x0058, DMI type 9, 17 bytes
578 System Slot Information
579 Designation: SlotID:5.4
580 Type: x8 PCI Express 3 x16
581 Bus Address: 0000:00:1f.7
582 Handle 0x0059, DMI type 9, 17 bytes
583 System Slot Information
584 Designation: SlotID:5.5
585 Type: x8 PCI Express 3 x16
586 Bus Address: 0000:00:1f.7
589 ### Linux dmidecode memory
592 $ dmidecode -t memory
596 Handle 0x0022, DMI type 16, 23 bytes
597 Physical Memory Array
598 Location: System Board Or Motherboard
600 Error Correction Type: Multi-bit ECC
601 Maximum Capacity: 1536 GB
602 Error Information Handle: Not Provided
603 Number Of Devices: 24
605 Handle 0x0024, DMI type 17, 34 bytes
608 Error Information Handle: Not Provided
615 Bank Locator: NODE 0 CHANNEL 0 DIMM 0
620 Serial Number: 80EC7380
621 Asset Tag: DIMM_A1_AssetTag
622 Part Number: HMA84GL7MMR4N-TF
624 Configured Clock Speed: 2133 MHz
626 Handle 0x0025, DMI type 17, 34 bytes
629 Error Information Handle: Not Provided
636 Bank Locator: NODE 0 CHANNEL 0 DIMM 1
641 Serial Number: 80EC73D1
642 Asset Tag: DIMM_A2_AssetTag
643 Part Number: HMA84GL7MMR4N-TF
645 Configured Clock Speed: 2133 MHz
647 Handle 0x0026, DMI type 17, 34 bytes
650 Error Information Handle: Not Provided
653 Size: No Module Installed
657 Bank Locator: NODE 0 CHANNEL 0 DIMM 2
661 Manufacturer: NO DIMM
662 Serial Number: NO DIMM
666 Configured Clock Speed: Unknown
668 Handle 0x0027, DMI type 17, 34 bytes
671 Error Information Handle: Not Provided
678 Bank Locator: NODE 0 CHANNEL 1 DIMM 0
683 Serial Number: 80EC7325
684 Asset Tag: DIMM_B1_AssetTag
685 Part Number: HMA84GL7MMR4N-TF
687 Configured Clock Speed: 2133 MHz
689 Handle 0x0028, DMI type 17, 34 bytes
692 Error Information Handle: Not Provided
699 Bank Locator: NODE 0 CHANNEL 1 DIMM 1
704 Serial Number: 80EC7334
705 Asset Tag: DIMM_B2_AssetTag
706 Part Number: HMA84GL7MMR4N-TF
708 Configured Clock Speed: 2133 MHz
710 Handle 0x0029, DMI type 17, 34 bytes
713 Error Information Handle: Not Provided
716 Size: No Module Installed
720 Bank Locator: NODE 0 CHANNEL 1 DIMM 2
724 Manufacturer: NO DIMM
725 Serial Number: NO DIMM
729 Configured Clock Speed: Unknown
731 Handle 0x002A, DMI type 17, 34 bytes
734 Error Information Handle: Not Provided
741 Bank Locator: NODE 0 CHANNEL 2 DIMM 0
746 Serial Number: 80EC7329
747 Asset Tag: DIMM_C1_AssetTag
748 Part Number: HMA84GL7MMR4N-TF
750 Configured Clock Speed: 2133 MHz
752 Handle 0x002B, DMI type 17, 34 bytes
755 Error Information Handle: Not Provided
762 Bank Locator: NODE 0 CHANNEL 2 DIMM 1
767 Serial Number: 80EC732D
768 Asset Tag: DIMM_C2_AssetTag
769 Part Number: HMA84GL7MMR4N-TF
771 Configured Clock Speed: 2133 MHz
773 Handle 0x002C, DMI type 17, 34 bytes
776 Error Information Handle: Not Provided
779 Size: No Module Installed
783 Bank Locator: NODE 0 CHANNEL 2 DIMM 2
787 Manufacturer: NO DIMM
788 Serial Number: NO DIMM
792 Configured Clock Speed: Unknown
794 Handle 0x002D, DMI type 17, 34 bytes
797 Error Information Handle: Not Provided
804 Bank Locator: NODE 0 CHANNEL 3 DIMM 0
809 Serial Number: 80EC73D3
810 Asset Tag: DIMM_D1_AssetTag
811 Part Number: HMA84GL7MMR4N-TF
813 Configured Clock Speed: 2133 MHz
815 Handle 0x002E, DMI type 17, 34 bytes
818 Error Information Handle: Not Provided
825 Bank Locator: NODE 0 CHANNEL 3 DIMM 1
830 Serial Number: 80EC7330
831 Asset Tag: DIMM_D2_AssetTag
832 Part Number: HMA84GL7MMR4N-TF
834 Configured Clock Speed: 2133 MHz
836 Handle 0x002F, DMI type 17, 34 bytes
839 Error Information Handle: Not Provided
842 Size: No Module Installed
846 Bank Locator: NODE 0 CHANNEL 3 DIMM 2
850 Manufacturer: NO DIMM
851 Serial Number: NO DIMM
855 Configured Clock Speed: Unknown
857 Handle 0x0030, DMI type 17, 34 bytes
860 Error Information Handle: Not Provided
867 Bank Locator: NODE 1 CHANNEL 0 DIMM 0
872 Serial Number: 80E54252
873 Asset Tag: DIMM_E1_AssetTag
874 Part Number: HMA84GL7MMR4N-TF
876 Configured Clock Speed: 2133 MHz
878 Handle 0x0031, DMI type 17, 34 bytes
881 Error Information Handle: Not Provided
888 Bank Locator: NODE 1 CHANNEL 0 DIMM 1
893 Serial Number: 80E54235
894 Asset Tag: DIMM_E2_AssetTag
895 Part Number: HMA84GL7MMR4N-TF
897 Configured Clock Speed: 2133 MHz
899 Handle 0x0032, DMI type 17, 34 bytes
902 Error Information Handle: Not Provided
905 Size: No Module Installed
909 Bank Locator: NODE 1 CHANNEL 0 DIMM 2
913 Manufacturer: NO DIMM
914 Serial Number: NO DIMM
918 Configured Clock Speed: Unknown
920 Handle 0x0033, DMI type 17, 34 bytes
923 Error Information Handle: Not Provided
930 Bank Locator: NODE 1 CHANNEL 1 DIMM 0
935 Serial Number: 80E54218
936 Asset Tag: DIMM_F1_AssetTag
937 Part Number: HMA84GL7MMR4N-TF
939 Configured Clock Speed: 2133 MHz
941 Handle 0x0034, DMI type 17, 34 bytes
944 Error Information Handle: Not Provided
951 Bank Locator: NODE 1 CHANNEL 1 DIMM 1
956 Serial Number: 80E54236
957 Asset Tag: DIMM_F2_AssetTag
958 Part Number: HMA84GL7MMR4N-TF
960 Configured Clock Speed: 2133 MHz
962 Handle 0x0035, DMI type 17, 34 bytes
965 Error Information Handle: Not Provided
968 Size: No Module Installed
972 Bank Locator: NODE 1 CHANNEL 1 DIMM 2
976 Manufacturer: NO DIMM
977 Serial Number: NO DIMM
981 Configured Clock Speed: Unknown
983 Handle 0x0036, DMI type 17, 34 bytes
986 Error Information Handle: Not Provided
993 Bank Locator: NODE 1 CHANNEL 2 DIMM 0
998 Serial Number: 80E54247
999 Asset Tag: DIMM_G1_AssetTag
1000 Part Number: HMA84GL7MMR4N-TF
1002 Configured Clock Speed: 2133 MHz
1004 Handle 0x0037, DMI type 17, 34 bytes
1006 Array Handle: 0x0022
1007 Error Information Handle: Not Provided
1008 Total Width: 72 bits
1014 Bank Locator: NODE 1 CHANNEL 2 DIMM 1
1018 Manufacturer: 0xAD00
1019 Serial Number: 80E5421E
1020 Asset Tag: DIMM_G2_AssetTag
1021 Part Number: HMA84GL7MMR4N-TF
1023 Configured Clock Speed: 2133 MHz
1025 Handle 0x0038, DMI type 17, 34 bytes
1027 Array Handle: 0x0022
1028 Error Information Handle: Not Provided
1029 Total Width: Unknown
1031 Size: No Module Installed
1035 Bank Locator: NODE 1 CHANNEL 2 DIMM 2
1039 Manufacturer: NO DIMM
1040 Serial Number: NO DIMM
1042 Part Number: NO DIMM
1044 Configured Clock Speed: Unknown
1046 Handle 0x0039, DMI type 17, 34 bytes
1048 Array Handle: 0x0022
1049 Error Information Handle: Not Provided
1050 Total Width: 72 bits
1056 Bank Locator: NODE 1 CHANNEL 3 DIMM 0
1060 Manufacturer: 0xAD00
1061 Serial Number: 80E5423C
1062 Asset Tag: DIMM_H1_AssetTag
1063 Part Number: HMA84GL7MMR4N-TF
1065 Configured Clock Speed: 2133 MHz
1067 Handle 0x003A, DMI type 17, 34 bytes
1069 Array Handle: 0x0022
1070 Error Information Handle: Not Provided
1071 Total Width: 72 bits
1077 Bank Locator: NODE 1 CHANNEL 3 DIMM 1
1081 Manufacturer: 0xAD00
1082 Serial Number: 80E5424D
1083 Asset Tag: DIMM_H2_AssetTag
1084 Part Number: HMA84GL7MMR4N-TF
1086 Configured Clock Speed: 2133 MHz
1088 Handle 0x003B, DMI type 17, 34 bytes
1090 Array Handle: 0x0022
1091 Error Information Handle: Not Provided
1092 Total Width: Unknown
1094 Size: No Module Installed
1098 Bank Locator: NODE 1 CHANNEL 3 DIMM 2
1102 Manufacturer: NO DIMM
1103 Serial Number: NO DIMM
1105 Part Number: NO DIMM
1107 Configured Clock Speed: Unknown
1110 ### Xeon Hsw Server BIOS Configuration
1114 C240 /bios # show advanced detail
1116 Intel(R) VT-d ATS Support: Enabled
1117 Adjacent Cache Line Prefetcher: Enabled
1118 All Onboard LOM Ports: Enabled
1120 Bits per second: 115200
1121 Power Technology: Performance
1122 Channel Interleaving: Auto
1123 Intel(R) VT-d Coherency Support: Disabled
1124 Console Redirection: COM 0
1125 Number of Enabled Cores: All
1126 Energy Performance: Performance
1127 CPU Performance: Enterprise
1128 DCU IP Prefetcher: Enabled
1129 DCU Streamer Prefetch: Enabled
1130 Demand Scrub: Enabled
1131 Direct Cache Access Support: Auto
1132 Enhanced Intel Speedstep(R) Tec: Disabled
1133 Execute Disable: Enabled
1135 Hardware Prefetcher: Enabled
1136 Intel(R) Hyper-Threading Techno: Disabled
1137 Intel(R) Turbo Boost Technology: Disabled
1138 Intel(R) VT: Enabled
1139 Intel(R) VT-d: Enabled
1140 Intel(R) Interrupt Remapping: Enabled
1141 Legacy USB Support: Enabled
1142 Extended APIC: XAPIC
1143 LOM Port 1 OptionROM: Enabled
1144 LOM Port 2 OptionROM: Enabled
1145 MMIO above 4GB: Enabled
1147 PCI ROM CLP: Disabled
1148 Package C State Limit: C6 Retention
1149 Intel(R) Pass Through DMA: Disabled
1150 Patrol Scrub: Enabled
1152 All PCIe Slots OptionROM: Enabled
1153 PCIe Slot:1 OptionROM: Disabled
1154 PCIe Slot:2 OptionROM: Disabled
1155 PCIe Slot:3 OptionROM: Disabled
1156 PCIe Slot:4 OptionROM: Disabled
1157 PCIe Slot:5 OptionROM: Disabled
1158 PCIe Slot:6 OptionROM: Disabled
1159 PCIe Slot:HBA Link Speed: GEN3
1160 PCIe Slot:HBA OptionROM: Enabled
1161 PCIe Slot:MLOM OptionROM: Enabled
1162 PCIe Slot:N1 OptionROM: Enabled
1163 PCIe Slot:N2 OptionROM: Enabled
1164 Processor Power state C1 Enhanc: Disabled
1165 Processor C3 Report: Disabled
1166 Processor C6 Report: Disabled
1167 P-STATE Coordination: HW ALL
1169 Energy Performance Tuning: BIOS
1170 QPI Link Frequency Select: Auto
1171 QPI Snoop Mode: Home Snoop
1172 Rank Interleaving: Auto
1173 Redirection After BIOS POST: Always Enable
1175 Select Memory RAS: Maximum Performance
1176 SR-IOV Support: Enabled
1177 Terminal Type: VT100
1178 Port 60/64 Emulation: Enabled
1179 Workload Configuration: Balanced
1180 CDN Support for VIC: Disabled
1181 Out-of-Band Management: Disabled
1182 C240-FCH1950V1H5 /bios/advanced #