Interview Prep Zoneby CuriouserLabs

Question 5 of 5 · architect level

What is NUMA, and when does it start affecting your services and databases?

🎤 Say this first

On multi-socket (and now multi-chiplet) machines, each CPU has its own memory controller and local DRAM; accessing the other socket's memory crosses an interconnect at ~1.5–2× latency and bounded bandwidth — Non-Uniform Memory Access. The OS tries (first-touch allocation, NUMA-aware scheduling) but big processes spanning nodes get remote-access roulette. You care when: machines have 2+ sockets, heaps/buffer pools exceed one node, latency SLOs are tight, or throughput mysteriously caps at ~half of spec. Levers: numactl binding, per-node sharding (one instance per socket), NUMA-aware allocators (-XX:+UseNUMA), and in clouds — instance types that keep you on one node.

The full picture

  • Why it exists: one shared memory controller became the bottleneck as core counts grew — distributing memory with the sockets restored bandwidth at the price of uniformity. (Chiplet CPUs — EPYC's CCDs — bring mild NUMA-ness inside a single socket now.)
  • First-touch is the policy to know: memory is placed on the node of the thread that first touches it — so an init thread that pre-touches the whole heap puts it all on one node, and worker threads elsewhere pay remote latency forever. This interacts directly with -XX:+AlwaysPreTouch and explains 'why is half my pool slower'.
  • Database lore is NUMA lore: MySQL's historical swap-despite-free-memory (one node full, allocator refusing remote) → numa_interleave; Redis/nginx per-socket instances; JVM -XX:+UseNUMA making Parallel/G1 allocate young gen locally.
  • The architect decision pattern: either embrace (shard: one process per node, pin threads+memory, network-shard between them — the low-variance choice) or interleave (spread pages round-robin: uniform mediocrity, no cliffs — right for one big cache). What's wrong is pretending the machine is uniform.
  • Kubernetes angle: Topology Manager + CPU Manager static policy align container CPUs and memory to a node for the pods that ask — the 'Guaranteed QoS' story extends to NUMA.

🔄 Likely follow-up questions

  • How does first-touch interact with JVM pre-touch and thread pools?
  • When would you run two JVMs (one per socket) instead of one big one?
  • What do perf's local vs remote DRAM counters tell you, and what's a bad ratio?