1 Calibration Data - Haswell
2 --------------------------
4 Following sections include sample calibration data measured on t1-sut1
5 server running in one of the Intel Xeon Haswell testbeds as specified in
6 `CSIT/Testbeds: Xeon Hsw, VIRL
7 <https://wiki.fd.io/view/CSIT/Testbeds:_Xeon_Hsw,_VIRL.#FD.io_CSIT_testbeds_-_Xeon_Haswell.2C_VIRL>`_.
9 Calibration data obtained from all other servers in Haswell testbeds
10 shows the same or similar values.
19 BOOT_IMAGE=/vmlinuz-4.4.0-72-generic root=UUID=efb7e8b3-3548-4440-98f6-6ebe102e9ec6 ro isolcpus=1-17,19-35 nohz_full=1-17,19-35 rcu_nocbs=1-17,19-35 intel_pstate=disable console=tty0 console=ttyS0,115200n8
28 Linux t3-sut2 4.4.0-72-generic #93-Ubuntu SMP Fri Mar 31 14:07:41 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
31 System-level core jitter
32 ~~~~~~~~~~~~~~~~~~~~~~~~
36 $ sudo taskset -c 3 /home/testuser/pma_tools/jitter/jitter -i 30
37 Linux Jitter testing program version 1.8
39 The pragram will execute a dummy function 80000 times
40 Display is updated every 20000 displayUpdate intervals
41 Timings are in CPU Core cycles
42 Inst_Min: Minimum Excution time during the display update interval(default is ~1 second)
43 Inst_Max: Maximum Excution time during the display update interval(default is ~1 second)
44 Inst_jitter: Jitter in the Excution time during rhe display update interval. This is the value of interest
45 last_Exec: The Excution time of last iteration just before the display update
46 Abs_Min: Absolute Minimum Excution time since the program started or statistics were reset
47 Abs_Max: Absolute Maximum Excution time since the program started or statistics were reset
48 tmp: Cumulative value calcualted by the dummy function
49 Interval: Time interval between the display updates in Core Cycles
50 Sample No: Sample number
52 Inst_Min Inst_Max Inst_jitter last_Exec Abs_min Abs_max tmp Interval Sample No
53 160024 172636 12612 160028 160024 172636 1573060608 3205463144 1
54 160024 188236 28212 160028 160024 188236 958595072 3205500844 2
55 160024 185676 25652 160028 160024 188236 344129536 3205485976 3
56 160024 172608 12584 160024 160024 188236 4024631296 3205472740 4
57 160024 179260 19236 160028 160024 188236 3410165760 3205502164 5
58 160024 172432 12408 160024 160024 188236 2795700224 3205452036 6
59 160024 178820 18796 160024 160024 188236 2181234688 3205455408 7
60 160024 172512 12488 160028 160024 188236 1566769152 3205461528 8
61 160024 172636 12612 160028 160024 188236 952303616 3205478820 9
62 160024 173676 13652 160028 160024 188236 337838080 3205470412 10
63 160024 178776 18752 160028 160024 188236 4018339840 3205481472 11
64 160024 172788 12764 160028 160024 188236 3403874304 3205492336 12
65 160024 174616 14592 160028 160024 188236 2789408768 3205474904 13
66 160024 174440 14416 160028 160024 188236 2174943232 3205479448 14
67 160024 178748 18724 160024 160024 188236 1560477696 3205482668 15
68 160024 172588 12564 169404 160024 188236 946012160 3205510496 16
69 160024 172636 12612 160024 160024 188236 331546624 3205472204 17
70 160024 172480 12456 160024 160024 188236 4012048384 3205455864 18
71 160024 172740 12716 160028 160024 188236 3397582848 3205464932 19
72 160024 179200 19176 160028 160024 188236 2783117312 3205476012 20
73 160024 172480 12456 160028 160024 188236 2168651776 3205465632 21
74 160024 172728 12704 160024 160024 188236 1554186240 3205497204 22
75 160024 172620 12596 160028 160024 188236 939720704 3205466972 23
76 160024 172640 12616 160028 160024 188236 325255168 3205471216 24
77 160024 172484 12460 160028 160024 188236 4005756928 3205467388 25
78 160024 172636 12612 160028 160024 188236 3391291392 3205482748 26
79 160024 179056 19032 160024 160024 188236 2776825856 3205467152 27
80 160024 172672 12648 160024 160024 188236 2162360320 3205483268 28
81 160024 176932 16908 160024 160024 188236 1547894784 3205488536 29
82 160024 172452 12428 160028 160024 188236 933429248 3205440636 30
90 $ sudo /home/testuser/mlc --bandwidth_matrix
91 Intel(R) Memory Latency Checker - v3.5
92 Command line parameters: --bandwidth_matrix
94 Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
95 Measuring Memory Bandwidths between nodes within system
96 Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
97 Using all the threads from each core if Hyper-threading is enabled
98 Using Read-only traffic type
106 $ sudo /home/testuser/mlc --peak_injection_bandwidth
107 Intel(R) Memory Latency Checker - v3.5
108 Command line parameters: --peak_injection_bandwidth
110 Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
112 Measuring Peak Injection Memory Bandwidths for the system
113 Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
114 Using all the threads from each core if Hyper-threading is enabled
115 Using traffic with the following read-write ratios
117 3:1 Reads-Writes : 106242.2
118 2:1 Reads-Writes : 103031.8
119 1:1 Reads-Writes : 87943.7
120 Stream-triad like: 100048.4
124 $ sudo /home/testuser/mlc --max_bandwidth
125 Intel(R) Memory Latency Checker - v3.5
126 Command line parameters: --max_bandwidth
128 Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
130 Measuring Maximum Memory Bandwidths for the system
131 Will take several minutes to complete as multiple injection rates will be tried to get the best bandwidth
132 Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
133 Using all the threads from each core if Hyper-threading is enabled
134 Using traffic with the following read-write ratios
135 ALL Reads : 115782.41
136 3:1 Reads-Writes : 105965.78
137 2:1 Reads-Writes : 103162.38
138 1:1 Reads-Writes : 88255.82
139 Stream-triad like: 105608.10
147 $ sudo /home/testuser/mlc --latency_matrix
148 Intel(R) Memory Latency Checker - v3.5
149 Command line parameters: --latency_matrix
151 Using buffer size of 200.000MB
152 Measuring idle latencies (in ns)...
160 $ sudo /home/testuser/mlc --idle_latency
161 Intel(R) Memory Latency Checker - v3.5
162 Command line parameters: --idle_latency
164 Using buffer size of 200.000MB
165 Each iteration took 227.2 core clocks ( 99.0 ns)
169 $ sudo /home/testuser/mlc --loaded_latency
170 Intel(R) Memory Latency Checker - v3.5
171 Command line parameters: --loaded_latency
173 Using buffer size of 100.000MB/thread for reads and an additional 100.000MB/thread for writes
175 Measuring Loaded Latencies for the system
176 Using all the threads from each core if Hyper-threading is enabled
177 Using Read-only traffic type
178 Inject Latency Bandwidth
180 ==========================
181 00000 294.08 115841.6
182 00002 294.27 115851.5
183 00008 293.67 115821.8
184 00015 278.92 115587.5
185 00050 246.80 113991.2
186 00100 206.86 104508.1
207 $ sudo /home/testuser/mlc --c2c_latency
208 Intel(R) Memory Latency Checker - v3.5
209 Command line parameters: --c2c_latency
211 Measuring cache-to-cache transfer latency (in ns)...
212 Local Socket L2->L2 HIT latency 42.1
213 Local Socket L2->L2 HITM latency 47.0
214 Remote Socket L2->L2 HITM latency (data address homed in writer socket)
219 Remote Socket L2->L2 HITM latency (data address homed in reader socket)