The full picture
| Symptom | Likely cause | Fix |
|---|---|---|
| One key, huge read volume | celebrity / viral item | cache in front; read replicas for that shard |
| One key, huge write volume | counter or feed fan-in | salt the key into K sub-keys, aggregate on read |
| One tenant dominating | multi-tenant skew | dedicated shard for large tenants |
| Newest range hot | timestamp partition key | bucket by entity + time window |
| Even keys, one shard hot | hash collision cluster / bad boundary | re-check hash; split the range |
// Before: every like on a viral post hits ONE partition.
key = "post:" + postId;
// After: N sub-counters spread across N partitions.
int bucket = ThreadLocalRandom.current().nextInt(SALT_BUCKETS); // e.g. 16
key = "post:" + postId + "#" + bucket;
// Reads now fan out and aggregate — bounded, parallel, but no longer a single GET.
long total = IntStream.range(0, SALT_BUCKETS).parallel()
.mapToLong(b -> store.get("post:" + postId + "#" + b))
.sum();
// Only salt the keys that are actually hot: a dynamic list of hot keys beats
// salting everything, which multiplies read cost across the whole workload.- Adaptive beats static: keep a small hot-key detector (count-min sketch or top-K over a sliding window) and apply salting or caching only to detected hot keys. Salting the entire keyspace to survive one celebrity makes every ordinary read 16× more expensive — a cure worse than the disease.
- Modern managed stores partly solve this for you: DynamoDB's adaptive capacity reallocates throughput toward hot partitions, and DAX caches hot items. Knowing the platform already handles moderate skew — and that it does not handle a single key exceeding one partition's hard limit — is exactly the practical nuance interviewers want.
- Skew is often a product event, not a systems bug: a Black Friday SKU, a celebrity account, a bot hammering one endpoint. The fastest mitigation may be a rate limit or a cache with a 5-second TTL, not a re-shard — and recognizing that saves an outage rather than architecting through one.
- Watch for the write-hot vs read-hot distinction: read-hot is easy (cache, replicas). Write-hot is genuinely hard, because caching can't absorb writes — you need salting, batching/coalescing (aggregate in memory, flush periodically), or a different data type (per-replica counters that sum on read).
- Big-tenant isolation is underrated: in multi-tenant systems, the top 1% of tenants often generate most load. Giving them dedicated shards (or dedicated clusters) simplifies capacity planning, blast radius and billing simultaneously — one decision that pays off three ways.