Alcides Fonseca

40.197958, -8.408312

Pitfalls of Benchmarking on Modern Systems

What could it be? It could be that the operating system decided to give it different physical memory, or run it on a different kind of core (many CPUs these days have 2 or more different types of cores with different performance). Perhaps, we ran out of thermal budget and the CPU ran at a lower clock speed to avoid overheating? Or, since we are on a JVM, perhaps the compiler saw slightly different information for the types/behavior seen in the first and second iterations, and thus, made slightly different optimization decisions? This is possible because compilation happens on a background thread, and thus, even when the benchmark is deterministic, the used profiling information is to some degree racy.

Pitfalls of Benchmarking on Modern Systems by Stefan Marr

In my PhD I’ve spent a lot of time doing careful benchmarking of parallel programs. During that time, I’ve learn a lot of variables that matter when doing this type of work: CPU, RAM, interpreter/compiler, room temperature, stack size, CPU layout (big.Little is completely impossible to properly benchmark, even with pinning), DVFS settings. At some half-way point, I’ve learned about the internal optimizations of CPUs, including dynamic prefetching, branch prediction and other non-deterministic behavior. Because of that, all of my approaches were statistical, assuming non-deterministic executions. And one annoying detail: the standard deviation of multi runs varied with the length of the program (traditionally huge for <1s workloads, stable for 2-60s and increases again after that).

If you are looking for help in designing these types of performance benchmarks, get in touch.