Interview Prep Zoneby CuriouserLabs

Question 4 of 5 · staff level

Mechanical sympathy in practice: how do you measure before optimizing — JMH, perf counters, flame graphs?

🎤 Say this first

The discipline is a loop: profile in production conditions → form a hypothesis at the right layer → microbenchmark the hypothesis → verify with hardware counters → re-measure end-to-end. Tools per layer: async-profiler/JFR flame graphs (where CPU time goes — safepoint-bias-free), JMH (isolated benchmarks that defeat JIT lies: warmup, forks, Blackhole), perf stat (IPC, cache/branch misses — why it's slow, not just where), and allocation profiling (GC pressure is a performance bug wearing a memory costume). The cardinal sins: benchmarking without warmup/forks, optimizing unprofiled code, trusting averages (p99!), and A/B-ing on different data shapes.

The full picture

JMH that doesn't lie
@BenchmarkMode(Mode.SampleTime)          // distribution, not just mean
@Warmup(iterations = 5, time = 1)        // let C2 finish its business
@Fork(3)                                  // JIT nondeterminism across JVMs
@State(Scope.Benchmark)
public class IndexOfBench {
    @Param({"16", "1024", "65536"}) int size;   // curves, not points
    byte[] haystack;

    @Setup public void setup() { haystack = randomBytes(size); }

    @Benchmark
    public int indexOf(Blackhole bh) {          // Blackhole: no dead-code elim
        return ArraysSupport.mismatch(haystack, target, size);
    }
}
// Then: perf stat -e cycles,instructions,cache-misses,branch-misses java ...
// IPC < 1 on a modern core = memory/branch bound -> layout fix, not algorithm fix.
  • Why naive benchmarks lie, specifically: dead-code elimination (result unused → loop deleted), constant folding (JIT precomputes your 'input'), OSR-compiled loops differing from real call paths, monomorphic call sites in the bench vs megamorphic in prod — JMH's ceremony exists per-lie; being able to name the lies is the credential.
  • Reading counters like a clinician: high IPC (≥2) + hot flame frame → algorithmic/compute; IPC < 1 + LLC misses → memory-bound (layout/locality toolbox); branch-miss % high → unpredictable control flow (sortedness, branchless); kernel-heavy frames → syscall/IO path (batching toolbox). The counters route you to the correct earlier question of this track.
  • Production-first profiling: async-profiler in prod (low overhead, no safepoint bias) or JFR continuous recording — the 3am profile of the actual regression beats any staging reproduction; wall-clock mode for 'where does latency go' (includes waiting), CPU mode for 'what burns cores'.
  • The allocation blind spot: alloc flame graphs frequently explain p99s better than CPU ones (young-gen churn → GC frequency) — one mapToObj in a hot loop shows up as GC time everywhere else.

🔄 Likely follow-up questions

  • What is safepoint bias and why does async-profiler avoid it?
  • Walk through diagnosing 'CPU is 40% but p99 doubled' with these tools.
  • How do you keep benchmarks honest in CI (variance, noisy neighbors)?