Interview Prep Zoneby CuriouserLabs

Question 5 of 5 · architect level

How do container CPU limits actually work, and why does your service get throttled at 40% CPU usage?

🎤 Say this first

K8s CPU requests become cgroup weights (proportional share under contention — never throttling). CPU limits become CFS bandwidth quotas: e.g. limit 2 = 200ms of CPU per 100ms period across ALL threads. A 16-thread JVM can burn 200ms in 12.5ms of wall time — then every thread freezes for the remaining 87.5ms: that's throttling at '40% average CPU', and it destroys p99s. Architect stance: set requests honestly, be very skeptical of limits for latency-critical services (or size them generously), watch container_cpu_cfs_throttled_periods_total, and remember the JVM must know its budget (ActiveProcessorCount).

The full picture

The throttling arithmetic
# K8s: limits.cpu = "2"  ->  cgroup: quota=200ms per 100ms period

16 GC/app threads run in parallel
  -> 16 threads × 12.5ms = 200ms quota consumed in 12.5ms wall time
  -> ALL threads frozen for 87.5ms          # request in flight? +87.5ms p99. GC pause? worse.
  -> dashboard average: "40% CPU"  😵        # averages hide 100ms-scale stalls

# Signals:
container_cpu_cfs_throttled_periods_total   # rising = you're in this story
sched_debug / pidstat -w                    # involuntary switches spike
  • Requests vs limits, restated as policy: requests = guaranteed floor + weight under contention (work-conserving, safe); limits = hard ceiling enforced by freezing (non-work-conserving — you throttle even on an idle node).
  • Why people set limits anyway: noisy-neighbor fear, Guaranteed QoS class for static CPU pinning, chargeback hygiene. Legit — but for latency-critical JVMs the emerging consensus is requests-only (or limits ≥ 2–3× requests), because freezing a JVM mid-GC is worse than brief neighbor noise.
  • JVM-specific traps: availableProcessors() derives from quota — a limit of 1 gives you 1 GC thread, 1 JIT thread, tiny default pools; conversely no limit on a 64-core node can mean 64 GC threads in a 512MB pod. Set -XX:ActiveProcessorCount deliberately when it matters.
  • The multi-thread burst insight generalizes: any parallel runtime (Go, parallel streams, ForkJoin) turns 'small average usage' into quota-burst-freeze cycles — throttling metrics, not CPU averages, are the truth.

🔄 Likely follow-up questions

  • When is the Guaranteed QoS class + static CPU manager worth the trade?
  • How would you detect and prove throttling is causing p99 spikes?
  • What changes with cgroup v2 and the cpu.max.burst feature?