X-Git-Url: https://gerrit.fd.io/r/gitweb?a=blobdiff_plain;ds=inline;f=docs%2Freport%2Fintroduction%2Ftest_environment_sut_calib_clx.rst;fp=docs%2Freport%2Fintroduction%2Ftest_environment_sut_calib_clx.rst;h=43e2f599eab5ccff084b2d8e687e27bf651c4b87;hb=b82474874d4329d3e82ea8a22754b7b04cf969ee;hp=0000000000000000000000000000000000000000;hpb=2fe2a2c140bffa10678e3e217c8a5cba0fd4dbd0;p=csit.git diff --git a/docs/report/introduction/test_environment_sut_calib_clx.rst b/docs/report/introduction/test_environment_sut_calib_clx.rst new file mode 100644 index 0000000000..43e2f599ea --- /dev/null +++ b/docs/report/introduction/test_environment_sut_calib_clx.rst @@ -0,0 +1,224 @@ +Calibration Data - Cascadelake +------------------------------ + +Following sections include sample calibration data measured on +s32-t27-sut1 server running in one of the Intel Xeon Skylake testbeds as +specified in `FD.io CSIT testbeds - Xeon Cascadelake`_. + +Calibration data obtained from all other servers in Cascadelake testbeds +shows the same or similar values. + + +Linux cmdline +~~~~~~~~~~~~~ + +:: + + $ cat /proc/cmdline + BOOT_IMAGE=/boot/vmlinuz-4.15.0-60-generic root=UUID=1d03969e-a2a0-41b2-a97e-1cc171b07e88 ro isolcpus=1-23,25-47,49-71,73-95 nohz_full=1-23,25-47,49-71,73-95 rcu_nocbs=1-23,25-47,49-71,73-95 numa_balancing=disable intel_pstate=disable intel_iommu=on iommu=pt nmi_watchdog=0 audit=0 nosoftlockup processor.max_cstate=1 intel_idle.max_cstate=1 hpet=disable tsc=reliable mce=off console=tty0 console=ttyS0,115200n8 + +Linux uname +~~~~~~~~~~~ + +:: + + $ uname -a + Linux s32-t27-sut1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux + + +System-level Core Jitter +~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + $ sudo taskset -c 3 /home/testuser/pma_tools/jitter/jitter -i 30 + Linux Jitter testing program version 1.9 + Iterations=30 + The pragram will execute a dummy function 80000 times + Display is updated every 20000 displayUpdate intervals + Thread affinity will be set to core_id:7 + Timings are in CPU Core cycles + Inst_Min: Minimum Excution time during the display update interval(default is ~1 second) + Inst_Max: Maximum Excution time during the display update interval(default is ~1 second) + Inst_jitter: Jitter in the Excution time during rhe display update interval. This is the value of interest + last_Exec: The Excution time of last iteration just before the display update + Abs_Min: Absolute Minimum Excution time since the program started or statistics were reset + Abs_Max: Absolute Maximum Excution time since the program started or statistics were reset + tmp: Cumulative value calcualted by the dummy function + Interval: Time interval between the display updates in Core Cycles + Sample No: Sample number + + Inst_Min,Inst_Max,Inst_jitter,last_Exec,Abs_min,Abs_max,tmp,Interval,Sample No + 160022,167590,7568,160026,160022,167590,2057568256,3203711852,1 + 160022,170628,10606,160024,160022,170628,4079222784,3204010824,2 + 160022,169824,9802,160024,160022,170628,1805910016,3203812064,3 + 160022,168832,8810,160030,160022,170628,3827564544,3203792594,4 + 160022,168248,8226,160026,160022,170628,1554251776,3203765920,5 + 160022,167834,7812,160028,160022,170628,3575906304,3203761114,6 + 160022,167442,7420,160024,160022,170628,1302593536,3203769250,7 + 160022,169120,9098,160028,160022,170628,3324248064,3203853340,8 + 160022,170710,10688,160024,160022,170710,1050935296,3203985878,9 + 160022,167952,7930,160024,160022,170710,3072589824,3203733756,10 + 160022,168314,8292,160030,160022,170710,799277056,3203741152,11 + 160022,169672,9650,160024,160022,170710,2820931584,3203739910,12 + 160022,168684,8662,160024,160022,170710,547618816,3203727336,13 + 160022,168246,8224,160024,160022,170710,2569273344,3203739052,14 + 160022,168134,8112,160030,160022,170710,295960576,3203735874,15 + 160022,170230,10208,160024,160022,170710,2317615104,3203996356,16 + 160022,167190,7168,160024,160022,170710,44302336,3203713628,17 + 160022,167304,7282,160024,160022,170710,2065956864,3203717954,18 + 160022,167500,7478,160024,160022,170710,4087611392,3203706674,19 + 160022,167302,7280,160024,160022,170710,1814298624,3203726452,20 + 160022,167266,7244,160024,160022,170710,3835953152,3203702804,21 + 160022,167820,7798,160022,160022,170710,1562640384,3203719138,22 + 160022,168100,8078,160024,160022,170710,3584294912,3203716636,23 + 160022,170408,10386,160024,160022,170710,1310982144,3203946958,24 + 160022,167276,7254,160024,160022,170710,3332636672,3203706236,25 + 160022,167052,7030,160024,160022,170710,1059323904,3203696444,26 + 160022,170322,10300,160024,160022,170710,3080978432,3203747514,27 + 160022,167332,7310,160024,160022,170710,807665664,3203716210,28 + 160022,167426,7404,160026,160022,170710,2829320192,3203700630,29 + 160022,168840,8818,160024,160022,170710,556007424,3203727658,30 + + +Memory Bandwidth +~~~~~~~~~~~~~~~~ + +:: + + $ sudo /home/testuser/mlc --bandwidth_matrix + Intel(R) Memory Latency Checker - v3.7 + Command line parameters: --bandwidth_matrix + + Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes + Measuring Memory Bandwidths between nodes within system + Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec) + Using all the threads from each core if Hyper-threading is enabled + Using Read-only traffic type + Numa node + Numa node 0 1 + 0 122097.7 51327.9 + 1 51309.2 122005.5 + +:: + + $ sudo /home/testuser/mlc --peak_injection_bandwidth + Intel(R) Memory Latency Checker - v3.7 + Command line parameters: --peak_injection_bandwidth + + Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes + + Measuring Peak Injection Memory Bandwidths for the system + Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec) + Using all the threads from each core if Hyper-threading is enabled + Using traffic with the following read-write ratios + ALL Reads : 243159.4 + 3:1 Reads-Writes : 219132.5 + 2:1 Reads-Writes : 216603.1 + 1:1 Reads-Writes : 203713.0 + Stream-triad like: 193790.8 + +:: + + $ sudo /home/testuser/mlc --max_bandwidth + Intel(R) Memory Latency Checker - v3.7 + Command line parameters: --max_bandwidth + + Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes + + Measuring Maximum Memory Bandwidths for the system + Will take several minutes to complete as multiple injection rates will be tried to get the best bandwidth + Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec) + Using all the threads from each core if Hyper-threading is enabled + Using traffic with the following read-write ratios + ALL Reads : 244114.27 + 3:1 Reads-Writes : 219441.97 + 2:1 Reads-Writes : 216603.72 + 1:1 Reads-Writes : 203679.09 + Stream-triad like: 214902.80 + + +Memory Latency +~~~~~~~~~~~~~~ + +:: + + $ sudo /home/testuser/mlc --latency_matrix + Intel(R) Memory Latency Checker - v3.7 + Command line parameters: --latency_matrix + + Using buffer size of 2000.000MiB + Measuring idle latencies (in ns)... + Numa node + Numa node 0 1 + 0 81.2 130.2 + 1 130.2 81.1 + +:: + + $ sudo /home/testuser/mlc --idle_latency + Intel(R) Memory Latency Checker - v3.7 + Command line parameters: --idle_latency + + Using buffer size of 2000.000MiB + Each iteration took 186.1 core clocks ( 80.9 ns) + +:: + + $ sudo /home/testuser/mlc --loaded_latency + Intel(R) Memory Latency Checker - v3.7 + Command line parameters: --loaded_latency + + Using buffer size of 100.000MiB/thread for reads and an additional 100.000MiB/thread for writes + + Measuring Loaded Latencies for the system + Using all the threads from each core if Hyper-threading is enabled + Using Read-only traffic type + Inject Latency Bandwidth + Delay (ns) MB/sec + ========================== + 00000 233.86 243421.9 + 00002 230.61 243544.1 + 00008 232.56 243394.5 + 00015 229.52 244076.6 + 00050 225.82 244290.6 + 00100 161.65 236744.8 + 00200 100.63 133844.0 + 00300 96.84 90548.2 + 00400 95.71 68504.3 + 00500 95.68 55139.0 + 00700 88.77 39798.4 + 01000 84.74 28200.1 + 01300 83.08 21915.5 + 01700 82.27 16969.3 + 02500 81.66 11810.6 + 03500 81.98 8662.9 + 05000 81.48 6306.8 + 09000 81.17 3857.8 + 20000 80.19 2179.9 + + +L1/L2/LLC Latency +~~~~~~~~~~~~~~~~~ + +:: + + $ sudo /home/testuser/mlc --c2c_latency + Intel(R) Memory Latency Checker - v3.7 + Command line parameters: --c2c_latency + + Measuring cache-to-cache transfer latency (in ns)... + Local Socket L2->L2 HIT latency 55.5 + Local Socket L2->L2 HITM latency 55.6 + Remote Socket L2->L2 HITM latency (data address homed in writer socket) + Reader Numa Node + Writer Numa Node 0 1 + 0 - 115.6 + 1 115.6 - + Remote Socket L2->L2 HITM latency (data address homed in reader socket) + Reader Numa Node + Writer Numa Node 0 1 + 0 - 178.2 + 1 178.4 - + +.. include:: ../introduction/test_environment_sut_meltspec_clx.rst