Interview Prep Zoneby CuriouserLabs

Question 3 of 5 · staff level

Cores vs hardware threads (SMT): what does the OS see, and how does it change thread-count decisions?

🎤 Say this first

A core is a full execution engine; SMT/hyper-threading exposes 2 hardware threads per core that share its execution units, caches and bandwidth — the second thread fills stall slots (cache misses) in the first, yielding ~10–30% extra throughput, not 2×. The OS schedules them as 'CPUs' (16 cores → 32 CPUs), and availableProcessors() reports the doubled number — so CPU-bound pools sized 'one per CPU' oversubscribe the actual ALUs by 2×: fine for stall-heavy work, counterproductive for dense numeric code (two threads thrash one core's units and L1). Decisions: throughput services → SMT on and sized to logical CPUs; latency-critical or vector-dense → consider physical-core sizing or SMT off; multi-tenant → note SMT's side-channel history (L1TF et al.).

The full picture

  • Why SMT works at all: an OoO core stalls constantly on memory (the running theme of this track) — a second architectural register file + shared everything else lets another thread use those dead cycles. It monetizes stalls, so its benefit is proportional to how stall-y the workload is: memory-bound servers ~20–30%, dense AVX kernels ~0% or negative.
  • The sizing corollary people miss: 'threads = availableProcessors' for CPU-bound work already includes SMT's optimism; JVM defaults (GC threads, FJ common pool) inherit it. For a latency-critical numeric service, benchmark at physical-core count — the p99 sometimes improves double digits just from halving the pool.
  • Topology awareness beyond SMT: sibling threads share L1/L2; cores share L3 within a socket/CCX; sockets are NUMA nodes — pinning two collaborating threads to SMT siblings (shared L2, cheap communication) vs separate cores (full units each) are different optimizations for different sharing patterns. lscpu -e shows the map.
  • Cloud print: a 'vCPU' is typically one SMT thread, not a core — capacity planning that treats 16 vCPUs as 16 cores over-promises by ~40%; and some latency-sensitive fleets (and security-sensitive multi-tenant hosts) buy SMT-off instance shapes deliberately.

🔄 Likely follow-up questions

  • When would you disable SMT in production, and what evidence justifies it?
  • Why is a vCPU not a core, and how do you capacity-plan around that?
  • How do you pin threads in Java (or arrange for the OS to), and when is it worth the rigidity?