Interview Prep Zoneby CuriouserLabs

Question 4 of 5 · architect level

A production JVM's old gen keeps growing until OutOfMemoryError. How do you find and fix the leak?

🎤 Say this first

Confirm it's a leak (old-gen sawtooth trending up after full GCs), capture evidence (-XX:+HeapDumpOnOutOfMemoryError, or jcmd GC.heap_dump at two points in time), then diff dominator trees in Eclipse MAT/JFR to find what's retaining the memory. Usual suspects: unbounded caches/maps, listener registrations never removed, ThreadLocals in pooled threads, classloader leaks on redeploy.

The full picture

  1. Confirm: GC logs or metrics — if full GCs reclaim less each time, it's a leak; if memory returns after GC, it's just churn or an undersized heap.
  2. Capture: heap dump at low load + at high water mark. Always run with -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=… in prod — the dump after the crash is free evidence.
  3. Analyze: in MAT open the dominator tree and 'Leak Suspects' — you're looking for the retained size champion and the reference path to GC roots that keeps it alive.
  4. Fix pattern, not instance: bound the cache (Caffeine with maximumSize/expiry), deregister listeners (try-with-resources / weak listeners), remove() ThreadLocals in pooled threads, isolate classloaders.
Three classic leaks
// 1. The static map that only grows
static final Map<UserId, Session> CACHE = new HashMap<>(); // no eviction = leak

// 2. ThreadLocal in an app running on a shared pool
static final ThreadLocal<BigBuffer> BUF = new ThreadLocal<>();
// pool threads never die -> value never GC'd. Always BUF.remove() in finally.

// 3. Mutated HashMap key
map.put(order, x); order.setId(99); // bucket now wrong -> unreachable entry

🔄 Likely follow-up questions

  • How do you take a heap dump with minimal impact on a live node? (jcmd, out-of-band on a drained instance)
  • What's a classloader leak and why do hot-redeploy servers suffer from it?
  • When is java.lang.ref.Cleaner the right tool?