What is pipelining, what are hazards, and how do stalls actually eat your performance?

🎤 Say this first

Pipelining overlaps instruction stages like a laundry line: while instruction 1 executes, 2 decodes, 3 fetches — one instruction completes per cycle even though each takes five. Hazards break the flow: structural (two stages need one resource), data (instruction needs a result not yet produced — forwarding wires fix most, but a load-use gap still stalls), and control (a branch: which instruction is next? The pipeline guesses — see prediction — and a wrong guess flushes ~15–20 stages of work). Real cost model: modern CPUs are limited far less by ALU speed than by stalls — from branch mispredicts and, above all, memory waits.

The full picture

Throughput vs latency, cleanly: the pipeline improves instruction throughput (one/cycle steady-state), never single-instruction latency — the same distinction as any assembly line or async pipeline you've built.
Data hazards & forwarding: a = b + c; d = a + e — the second add needs a before write-back; bypass wires forward ALU output directly to the next instruction's input. Only the load-use case (need data from a cache access) forces a genuine bubble — why compilers/JITs hoist loads early.
Control hazards dominate: every ~5th instruction is a branch; with 15–20 stage pipelines and 4–6-wide issue, a mispredict throws away 50–100 instruction slots — the arithmetic that makes branch prediction existential, not optional.
Where a JVM engineer sees it: JIT loop unrolling (fewer branches), inlining (removes call/return control flow), and the classic 'sorted array is faster to sum with an if' — pipeline effects visible from Java.

🎤 Say this first

The full picture

🔄 Likely follow-up questions