X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=docs%2Flab%2Ftestbeds_sm_skx_hw_bios_cfg.md;h=d1e0ea30fadb8a499ff93018835ab6585c1861a6;hp=6a2bfd782ddb2ee7193e631134fa1260dd57a356;hb=b82474874d4329d3e82ea8a22754b7b04cf969ee;hpb=ddbfd9e68e81e77bd253c8b6cd81839ec3bd2715 diff --git a/docs/lab/testbeds_sm_skx_hw_bios_cfg.md b/docs/lab/testbeds_sm_skx_hw_bios_cfg.md index 6a2bfd782d..d1e0ea30fa 100644 --- a/docs/lab/testbeds_sm_skx_hw_bios_cfg.md +++ b/docs/lab/testbeds_sm_skx_hw_bios_cfg.md @@ -1,44 +1,273 @@ - +# SuperMicro Skylake Servers - HW and BIOS Configuration + +1. [Linux lscpu](#linux-lscpu) +1. [Linux dmidecode](#dmidecode) +1. [Xeon Skx Server BIOS Configuration](#xeon-skx-server-bios-configuration) + 1. [Boot Feature](#boot-feature) + 1. [CPU Configuration](#cpu-configuration) + 1. [Advanced Power Management Configuration](#advanced-power-management-configuration) + 1. [CPU P State Control](#cpu-p-state-control) + 1. [Hardware PM State Control](#hardware-pm-state-control) + 1. [CPU C State Control](#cpu-c-state-control) + 1. [Package C State Control](#package-c-state-control) + 1. [CPU T State Control](#cpu-t-state-control) + 1. [Chipset Configuration](#chipset-configuration) + 1. [North Bridge](#north-bridge) + 1. [UPI Configuration](#upi-configuration) + 1. [Memory Configuration](#memory-configuration) + 1. [IIO Configuration](#iio-configuration) + 1. [CPU1 Configuration](#cpu1-configuration) + 1. [CPU2 Configuration](#cpu2-configuration) + 1. [South Bridge](#south-bridge) + 1. [PCIe/PCI/PnP Configuration](#pciepcipnp-configuration) + 1. [ACPI Settings](#acpi-settings) +1. [Xeon Skx Server Firmware Inventory](#xeon-skx-server-firmware-inventory) + +## Linux lscpu + +``` +$ lscpu +Architecture: x86_64 +CPU op-mode(s): 32-bit, 64-bit +Byte Order: Little Endian +CPU(s): 112 +On-line CPU(s) list: 0-111 +Thread(s) per core: 2 +Core(s) per socket: 28 +Socket(s): 2 +NUMA node(s): 2 +Vendor ID: GenuineIntel +CPU family: 6 +Model: 85 +Model name: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz +Stepping: 4 +CPU MHz: 2500.550 +BogoMIPS: 5000.00 +Virtualization: VT-x +L1d cache: 32K +L1i cache: 32K +L2 cache: 1024K +L3 cache: 39424K +NUMA node0 CPU(s): 0-27,56-83 +NUMA node1 CPU(s): 28-55,84-111 +Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca +cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx +pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology +nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 +ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt +tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch +cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti ssbd mba ibrs ibpb stibp +tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjustbmi1 hle avx2 smep bmi2 +erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb +intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc +cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke +md_clear flush_l1d +``` + +### Linux dmidecode - - [SuperMicro Xeon Skylake Servers - Hardware and BIOS Configuration](#supermicro-xeon-skylake-servers-hardware-and-bios-configuration) - - [Linux lscpu TODO](#linux-lscpu-todo) - - [Linux dmidecode pci TODO](#linux-dmidecode-pci-todo) - - [Linux dmidecode memory TODO](#linux-dmidecode-memory-todo) - - [Xeon Skx Server BIOS Configuration](#xeon-skx-server-bios-configuration) - - [Boot Feature](#boot-feature) - - [CPU Configuration](#cpu-configuration) - - [Advanced Power Management Configuration](#advanced-power-management-configuration) - - [CPU P State Control](#cpu-p-state-control) - - [Hardware PM State Control](#hardware-pm-state-control) - - [CPU C State Control](#cpu-c-state-control) - - [Package C State Control](#package-c-state-control) - - [CPU T State Control](#cpu-t-state-control) - - [Chipset Configuration](#chipset-configuration) - - [North Bridge](#north-bridge) - - [UPI Configuration](#upi-configuration) - - [Memory Configuration](#memory-configuration) - - [IIO Configuration](#iio-configuration) - - [CPU1 Configuration](#cpu1-configuration) - - [CPU2 Configuration](#cpu2-configuration) - - [South Bridge](#south-bridge) - - [PCIe/PCI/PnP Configuration](#pciepcipnp-configuration) - - [ACPI Settings](#acpi-settings) - - [DMIDECODE](#dmidecode) - - [Xeon Skx Server Firmware Inventory](#xeon-skx-server-firmware-inventory) +``` + # dmidecode 3.1 + Getting SMBIOS data from sysfs. + SMBIOS 3.1.1 present. + Table at 0x000E89C0. - + Handle 0x0000, DMI type 0, 26 bytes + BIOS Information + Vendor: American Megatrends Inc. + Version: 2.0 + Release Date: 11/29/2017 + Address: 0xF0000 + Runtime Size: 64 kB + ROM Size: 64 MB + Characteristics: + PCI is supported + BIOS is upgradeable + BIOS shadowing is allowed + Boot from CD is supported + Selectable boot is supported + BIOS ROM is socketed + EDD is supported + 5.25"/1.2 MB floppy services are supported (int 13h) + 3.5"/720 kB floppy services are supported (int 13h) + 3.5"/2.88 MB floppy services are supported (int 13h) + Print screen service is supported (int 5h) + Serial services are supported (int 14h) + Printer services are supported (int 17h) + ACPI is supported + USB legacy is supported + BIOS boot specification is supported + Targeted content distribution is supported + UEFI is supported + BIOS Revision: 5.12 -## SuperMicro Xeon Skylake Servers - Hardware and BIOS Configuration + Handle 0x0001, DMI type 1, 27 bytes + System Information + Manufacturer: Supermicro + Product Name: SYS-7049GP-TRT + Version: 0123456789 + Serial Number: S291427X8332242 + UUID: 00000000-0000-0000-0000-AC1F6B8A8DB6 + Wake-up Type: Power Switch + SKU Number: To be filled by O.E.M. + Family: To be filled by O.E.M. -### Linux lscpu TODO + Handle 0x0002, DMI type 2, 15 bytes + Base Board Information + Manufacturer: Supermicro + Product Name: X11DPG-QT + Version: 1.02 + Serial Number: VM183S014930 + Asset Tag: To be filled by O.E.M. + Features: + Board is a hosting board + Board is replaceable + Location In Chassis: To be filled by O.E.M. + Chassis Handle: 0x0003 + Type: Motherboard + Contained Object Handles: 0 -### Linux dmidecode pci TODO + Handle 0x0003, DMI type 3, 22 bytes + Chassis Information + Manufacturer: Supermicro + Type: Other + Lock: Not Present + Version: 0123456789 + Serial Number: C7470KH06A20167 + Asset Tag: To be filled by O.E.M. + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 -### Linux dmidecode memory TODO + Handle 0x0050, DMI type 4, 48 bytes + Processor Information + Socket Designation: CPU1 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: 54 06 05 00 FF FB EB BF + Signature: Type 0, Family 6, Model 85, Stepping 4 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4000 MHz + Current Speed: 2500 MHz + Status: Populated, Enabled + Upgrade: Other + L1 Cache Handle: 0x004D + L2 Cache Handle: 0x004E + L3 Cache Handle: 0x004F + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 28 + Core Enabled: 28 + Thread Count: 56 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control + + + Handle 0x0054, DMI type 4, 48 bytes + Processor Information + Socket Designation: CPU2 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: 54 06 05 00 FF FB EB BF + Signature: Type 0, Family 6, Model 85, Stepping 4 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4000 MHz + Current Speed: 2500 MHz + Status: Populated, Enabled + Upgrade: Other + L1 Cache Handle: 0x0051 + L2 Cache Handle: 0x0052 + L3 Cache Handle: 0x0053 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 28 + Core Enabled: 28 + Thread Count: 56 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control +``` -### Xeon Skx Server BIOS Configuration +## Xeon Skx Server BIOS Configuration -#### Boot Feature +### Boot Feature ``` | Quiet Boot [Enabled] |Boot option | @@ -58,7 +287,7 @@ | Throttle on Power Fail [Disabled] | | ``` -#### CPU Configuration +### CPU Configuration ``` | Processor Configuration |Enables Hyper Threading | @@ -93,7 +322,7 @@ |> Advanced Power Management Configuration | | ``` -##### Advanced Power Management Configuration +#### Advanced Power Management Configuration ``` | Advanced Power Management Configuration |Switch CPU Power Management | @@ -108,7 +337,7 @@ |> CPU T State Control | | ``` -###### CPU P State Control +##### CPU P State Control ``` | CPU P State Control |Enable/Disable EIST | @@ -117,7 +346,7 @@ | EIST PSD Function [HW_ALL] | | ``` -###### Hardware PM State Control +##### Hardware PM State Control ``` | Hardware PM State Control |Disable: Hardware chooses a | @@ -131,7 +360,7 @@ | |P-state (no OS guidance) | ``` -###### CPU C State Control +##### CPU C State Control ``` | CPU C State Control |Autonomous Core C-State | @@ -141,7 +370,7 @@ | Enhanced Halt State (C1E) [Disable] | | ``` -###### Package C State Control +##### Package C State Control ``` | Package C State Control |Package C State limit | @@ -149,7 +378,7 @@ | Package C State [C0/C1 state] | | ``` -###### CPU T State Control +##### CPU T State Control ``` | CPU T State Control |Enable/Disable Software | @@ -157,7 +386,7 @@ | Software Controlled T-States [Disable] | | ``` -##### Chipset Configuration +#### Chipset Configuration ``` | WARNING: Setting wrong values in below sections may cause |North Bridge Parameters | @@ -166,7 +395,7 @@ |> South Bridge | | ``` -###### North Bridge +##### North Bridge ``` |> UPI Configuration |Displays and provides | @@ -174,7 +403,7 @@ |> IIO Configuration |Settings | ``` -###### UPI Configuration +##### UPI Configuration ``` | UPI Configuration |Choose Topology Precedence | @@ -199,7 +428,7 @@ | Isoc Mode [Auto] | | ``` -###### Memory Configuration +##### Memory Configuration ``` | |POR - Enforces Plan Of | @@ -220,7 +449,7 @@ |> Memory RAS Configuration | | ``` -###### IIO Configuration +##### IIO Configuration ``` | IIO Configuration |Expose IIO DFX devices and | @@ -238,7 +467,7 @@ | PCI-E Completion Timeout Disable [No] | | ``` -###### CPU1 Configuration +##### CPU1 Configuration ``` | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | @@ -249,7 +478,7 @@ |> CPU1 SLOT9 PCI-E 3.0 X16 | | ``` -###### CPU2 Configuration +##### CPU2 Configuration ``` | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | @@ -260,7 +489,7 @@ |> CPU2 SLOT10 PCI-E 3.0 X16 | | ``` -##### South Bridge +#### South Bridge ``` | |Enables Legacy USB support. | @@ -280,7 +509,7 @@ | Azalia PME Enable [Disabled] | | ``` -#### PCIe/PCI/PnP Configuration +### PCIe/PCI/PnP Configuration ``` | PCI Bus Driver Version A5.01.12 |Enables or Disables 64bit | @@ -308,7 +537,7 @@ |> Network Stack Configuration | | ``` -#### ACPI Settings +### ACPI Settings ``` | ACPI Settings |Enable or Disable Non | @@ -319,208 +548,7 @@ | ACPI Sleep State [S3 (Suspend to RAM)] | | ``` -#### DMIDECODE - -``` - # dmidecode 3.1 - Getting SMBIOS data from sysfs. - SMBIOS 3.1.1 present. - Table at 0x000E89C0. - - Handle 0x0000, DMI type 0, 26 bytes - BIOS Information - Vendor: American Megatrends Inc. - Version: 2.0 - Release Date: 11/29/2017 - Address: 0xF0000 - Runtime Size: 64 kB - ROM Size: 64 MB - Characteristics: - PCI is supported - BIOS is upgradeable - BIOS shadowing is allowed - Boot from CD is supported - Selectable boot is supported - BIOS ROM is socketed - EDD is supported - 5.25"/1.2 MB floppy services are supported (int 13h) - 3.5"/720 kB floppy services are supported (int 13h) - 3.5"/2.88 MB floppy services are supported (int 13h) - Print screen service is supported (int 5h) - Serial services are supported (int 14h) - Printer services are supported (int 17h) - ACPI is supported - USB legacy is supported - BIOS boot specification is supported - Targeted content distribution is supported - UEFI is supported - BIOS Revision: 5.12 - - Handle 0x0001, DMI type 1, 27 bytes - System Information - Manufacturer: Supermicro - Product Name: SYS-7049GP-TRT - Version: 0123456789 - Serial Number: S291427X8332242 - UUID: 00000000-0000-0000-0000-AC1F6B8A8DB6 - Wake-up Type: Power Switch - SKU Number: To be filled by O.E.M. - Family: To be filled by O.E.M. - - Handle 0x0002, DMI type 2, 15 bytes - Base Board Information - Manufacturer: Supermicro - Product Name: X11DPG-QT - Version: 1.02 - Serial Number: VM183S014930 - Asset Tag: To be filled by O.E.M. - Features: - Board is a hosting board - Board is replaceable - Location In Chassis: To be filled by O.E.M. - Chassis Handle: 0x0003 - Type: Motherboard - Contained Object Handles: 0 - - Handle 0x0003, DMI type 3, 22 bytes - Chassis Information - Manufacturer: Supermicro - Type: Other - Lock: Not Present - Version: 0123456789 - Serial Number: C7470KH06A20167 - Asset Tag: To be filled by O.E.M. - Boot-up State: Safe - Power Supply State: Safe - Thermal State: Safe - Security Status: None - OEM Information: 0x00000000 - - Handle 0x0050, DMI type 4, 48 bytes - Processor Information - Socket Designation: CPU1 - Type: Central Processor - Family: Xeon - Manufacturer: Intel(R) Corporation - ID: 54 06 05 00 FF FB EB BF - Signature: Type 0, Family 6, Model 85, Stepping 4 - Flags: - FPU (Floating-point unit on-chip) - VME (Virtual mode extension) - DE (Debugging extension) - PSE (Page size extension) - TSC (Time stamp counter) - MSR (Model specific registers) - PAE (Physical address extension) - MCE (Machine check exception) - CX8 (CMPXCHG8 instruction supported) - APIC (On-chip APIC hardware supported) - SEP (Fast system call) - MTRR (Memory type range registers) - PGE (Page global enable) - MCA (Machine check architecture) - CMOV (Conditional move instruction supported) - PAT (Page attribute table) - PSE-36 (36-bit page size extension) - CLFSH (CLFLUSH instruction supported) - DS (Debug store) - ACPI (ACPI supported) - MMX (MMX technology supported) - FXSR (FXSAVE and FXSTOR instructions supported) - SSE (Streaming SIMD extensions) - SSE2 (Streaming SIMD extensions 2) - SS (Self-snoop) - HTT (Multi-threading) - TM (Thermal monitor supported) - PBE (Pending break enabled) - Version: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz - Voltage: 1.6 V - External Clock: 100 MHz - Max Speed: 4000 MHz - Current Speed: 2500 MHz - Status: Populated, Enabled - Upgrade: Other - L1 Cache Handle: 0x004D - L2 Cache Handle: 0x004E - L3 Cache Handle: 0x004F - Serial Number: Not Specified - Asset Tag: UNKNOWN - Part Number: Not Specified - Core Count: 28 - Core Enabled: 28 - Thread Count: 56 - Characteristics: - 64-bit capable - Multi-Core - Hardware Thread - Execute Protection - Enhanced Virtualization - Power/Performance Control - - - Handle 0x0054, DMI type 4, 48 bytes - Processor Information - Socket Designation: CPU2 - Type: Central Processor - Family: Xeon - Manufacturer: Intel(R) Corporation - ID: 54 06 05 00 FF FB EB BF - Signature: Type 0, Family 6, Model 85, Stepping 4 - Flags: - FPU (Floating-point unit on-chip) - VME (Virtual mode extension) - DE (Debugging extension) - PSE (Page size extension) - TSC (Time stamp counter) - MSR (Model specific registers) - PAE (Physical address extension) - MCE (Machine check exception) - CX8 (CMPXCHG8 instruction supported) - APIC (On-chip APIC hardware supported) - SEP (Fast system call) - MTRR (Memory type range registers) - PGE (Page global enable) - MCA (Machine check architecture) - CMOV (Conditional move instruction supported) - PAT (Page attribute table) - PSE-36 (36-bit page size extension) - CLFSH (CLFLUSH instruction supported) - DS (Debug store) - ACPI (ACPI supported) - MMX (MMX technology supported) - FXSR (FXSAVE and FXSTOR instructions supported) - SSE (Streaming SIMD extensions) - SSE2 (Streaming SIMD extensions 2) - SS (Self-snoop) - HTT (Multi-threading) - TM (Thermal monitor supported) - PBE (Pending break enabled) - Version: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz - Voltage: 1.6 V - External Clock: 100 MHz - Max Speed: 4000 MHz - Current Speed: 2500 MHz - Status: Populated, Enabled - Upgrade: Other - L1 Cache Handle: 0x0051 - L2 Cache Handle: 0x0052 - L3 Cache Handle: 0x0053 - Serial Number: Not Specified - Asset Tag: UNKNOWN - Part Number: Not Specified - Core Count: 28 - Core Enabled: 28 - Thread Count: 56 - Characteristics: - 64-bit capable - Multi-Core - Hardware Thread - Execute Protection - Enhanced Virtualization - Power/Performance Control -``` - -### Xeon Skx Server Firmware Inventory +## Xeon Skx Server Firmware Inventory ``` Host. IPMI IP. BIOS. CPLD. Aptio SU. CPU Microcode. PCI Bus. ME Operation FW. X710 Firmware. XXV710 Firmware. i40e.