Interview Prep Zoneby CuriouserLabs

Question 1 of 5 · senior level

Describe the memory hierarchy and the latency numbers every senior engineer should carry in their head.

🎤 Say this first

Registers (~0.3ns) → L1 (~1ns) → L2 (~4ns) → L3 (~12ns, shared) → DRAM (~100ns) → NVMe SSD (~100µs) → datacenter round trip (~500µs) → disk/cross-region (ms). Each level trades speed for size because fast memory (SRAM, 6 transistors/bit) is expensive and physics taxes distance. The hierarchy works only because programs exhibit locality; the numbers matter because every systems argument — 'cache it', 'batch it', 'keep it sequential', 'don't chase pointers' — is arithmetic over this table.

The full picture

🎬 The memory hierarchy & why locality is everything

1 / 5
Registers · ~1 KB · 0.3 nsL1 cache · 64 KB · ~1 nsL2 cache · 1 MB · ~4 nsL3 cache · 32 MB · ~12 nsRAM (DRAM) · 64 GB · ~100 nsSSD · 2 TB · ~100 µs 😱Each level exists to hide the latency of the one below it.

Fast memory is expensive and tiny; big memory is slow. So machines stack them: registers → L1/L2/L3 caches (SRAM) → main memory (DRAM) → SSD. Each level is ~10× slower and ~1000× bigger than the one above.

A load searching down the hierarchy — each miss costs roughly 10×.
  • Use the ratios, not the absolutes: DRAM is ~100× L1; a miss is ~300+ potential instructions; SSD is ~1000× DRAM. Back-of-envelope: 'this hash join walks 10M random pointers → 10M × 100ns = 1 second of pure memory stalls' — that estimate, said aloud, is the senior skill.
  • Why hierarchy instead of one fast memory: SRAM cost/density and the speed-of-light-ish reality that bigger = farther = slower. It's caching all the way down — CDN→Redis→page cache→L3→L1 are one design pattern at seven scales.
  • The memory wall: CPU speed grew ~50%/yr for decades, DRAM latency barely moved — the entire modern core (OoO, prefetch, SMT, giant caches) is a coping mechanism. GC theory, columnar databases and cache-oblivious algorithms are all downstream of this one divergence.
  • Numbers date-stamp: quote them as orders of magnitude ('L1 ~1ns, DRAM ~100ns, NVMe ~100µs') and you're immune to hardware-generation nitpicks.

🔄 Likely follow-up questions

  • Estimate: how long to sum a 1GB array vs follow a 1GB linked list?
  • Why is L3 shared while L1/L2 are per-core?
  • How do these numbers reshape when your 'memory' is a remote cache like Redis?