Cache Stampede (Thundering Herd) in Node.js & Redis: Preventing Database Meltdowns on Cache Misses
You added Redis in front of an expensive query, and it worked beautifully. The dashboard that took 800 ms now returns in 3 ms. The database CPU dropped from 70% to 10%. Everyone is happy.
Then, one afternoon, the database CPU spikes to 100%, latency goes vertical, and the incident channel lights up. The graphs show a sharp, periodic pattern: calm, spike, calm, spike. Nothing deployed. Traffic didn't change.
What happened is a cache stampede (also called the thundering herd or dogpile problem). A hot key expired, and for the next few hundred milliseconds, every request that wanted it missed the cache at the same moment and ran the expensive query itself.
This post explains exactly how that happens and builds up a set of defenses in Node.js with ioredis, from a ten-line fix to a production-grade helper.
---
Anatomy of a stampede
Here's the classic cache-aside code:
async function getLeaderboard() {
const cached = await redis.get('leaderboard');
if (cached) return JSON.parse(cached);
const data = await db.query(EXPENSIVE_SQL); // takes 800 ms
await redis.set('leaderboard', JSON.stringify(data), 'EX', 60);
return data;
}
Perfectly reasonable. Now watch what happens at the moment the key expires, with 500 requests per second hitting this endpoint:
t=0.000 key expires
t=0.001 request 1: cache miss → starts the 800 ms query
t=0.003 request 2: cache miss → starts the 800 ms query
t=0.004 request 3: cache miss → starts the 800 ms query
...
t=0.800 request 1 finishes, writes the cache
(by now ~400 requests have missed and started the same query)
The query takes 800 ms, so for 800 ms every request is a miss. At 500 req/s that's ~400 identical expensive queries launched against the database in under a second, instead of one. The database slows down under that load, which makes the queries take longer, which widens the miss window, which launches more queries. It's a feedback loop.
Related problems with similar names
People mix these up, but the fixes differ:
| Problem | What happens | Typical cause |
|---|---|---|
| Stampede / thundering herd | Many concurrent misses on one hot key | A popular key expires |
| Avalanche | Many different keys expire together, or the cache is gone | Same TTL set in a batch; Redis restart or failover |
| Penetration | Requests for data that doesn't exist always miss and hit the database | Missing IDs, scanners, bad input |
We'll address all three, since they feed each other.
---
Defense 1: Single-flight (collapse concurrent misses in one process)
The cheapest fix: if a load for a key is already in progress in this process, wait for it instead of starting another. Store the in-flight promise in a map.
const inflight = new Map<string, Promise<unknown>>();
async function singleFlight<T>(key: string, fn: () => Promise<T>): Promise<T> {
const existing = inflight.get(key);
if (existing) return existing as Promise<T>;
const promise = fn().finally(() => inflight.delete(key));
inflight.set(key, promise);
return promise;
}
async function getLeaderboard() {
const cached = await redis.get('leaderboard');
if (cached) return JSON.parse(cached);
return singleFlight('leaderboard', async () => {
const data = await db.query(EXPENSIVE_SQL);
await redis.set('leaderboard', JSON.stringify(data), 'EX', 60);
return data;
});
}
Now 400 concurrent misses in one process become one database query, and all 400 callers share its result. Because Node.js is single-threaded, the check-then-set on the map has no race.
The limitation is in the name: it's per process. With 20 instances you get up to 20 concurrent loads per expiry, not 400, a big improvement but not a guarantee. For that, you need coordination through Redis.
---
Defense 2: A distributed lock so only one process rebuilds the value
Use Redis itself to elect one loader across the whole fleet. SET key value NX PX ttl sets the key only if it doesn't exist, atomically:
import { randomUUID } from 'node:crypto';
async function tryLock(key: string, ttlMs: number): Promise<string | null> {
const token = randomUUID();
const ok = await redis.set(`lock:${key}`, token, 'PX', ttlMs, 'NX');
return ok === 'OK' ? token : null;
}
// Release only if we still own the lock. Compare-and-delete must be atomic, so use Lua.
const RELEASE = `
if redis.call('get', KEYS[1]) == ARGV[1] then
return redis.call('del', KEYS[1])
else
return 0
end`;
async function unlock(key: string, token: string) {
await redis.eval(RELEASE, 1, `lock:${key}`, token);
}
Two details matter:
- The random token + Lua release ensures you never delete someone else's lock. Without it, if your work outlasts the lock's TTL, another process acquires the lock, and your delayed DEL releases their lock.
- The lock TTL is a safety net, not a timer. It must exceed your worst-case load time, so a crashed process can't hold the lock forever.
Then the miss path becomes:
async function getWithLock<T>(key: string, loader: () => Promise<T>, ttlSec: number): Promise<T> {
const cached = await redis.get(key);
if (cached) return JSON.parse(cached);
const token = await tryLock(key, 10_000);
if (token) {
try {
const value = await loader();
await redis.set(key, JSON.stringify(value), 'EX', ttlSec);
return value;
} finally {
await unlock(key, token);
}
}
// Someone else is loading. Wait briefly for their result instead of piling onto the database.
for (let i = 0; i < 40; i++) { // ~2s max
await sleep(50);
const filled = await redis.get(key);
if (filled) return JSON.parse(filled);
}
return loader(); // gave up waiting: load it ourselves
}
Is this a "correct" distributed lock? Not in the strict sense. A paused process can outlive its lease, and a single Redis node can fail over mid-lock. But for cache filling that doesn't matter: the worst case is that two processes load the same value and write the same result. You're using the lock to reduce duplicate work, not to guard a critical section, so a simple SET NX PX is enough. (If you're protecting something where duplicates are harmful, that's a different problem and needs fencing tokens or a database constraint, not a cache lock.)
The downside: waiters are blocked. Everyone who missed is stuck polling for up to the load time. The next defense removes that wait entirely.
---
Defense 3: Stale-while-revalidate (never make users wait)
The insight: for most cached data, slightly old data is better than a slow response. So don't delete the value when it "expires". Keep it around, mark it stale, and refresh it in the background while still serving the old copy.
Store two expirations: a soft TTL (when it becomes stale) and a hard TTL (when Redis actually deletes it), longer by a grace window.
interface Entry<T> {
v: T;
softExp: number; // epoch ms: after this, the value is stale and should be refreshed
}
async function set<T>(key: string, value: T, softTtlMs: number, graceMs: number) {
const entry: Entry<T> = { v: value, softExp: Date.now() + softTtlMs };
await redis.set(key, JSON.stringify(entry), 'PX', softTtlMs + graceMs);
}
On read:
- Fresh (now < softExp): return it.
- Stale (past softExp but still in Redis): return it immediately, and if you win the lock, refresh in the background.
- Missing (past the hard TTL): load synchronously. This is the only path where someone waits.
Now the expensive load happens off the request path. Users see 3 ms responses all the time, and the database sees one refresh per interval. As a bonus, if the refresh fails (the database is struggling), you keep serving the stale copy, a "stale-if-error" safety net that turns a database outage into slightly old data rather than errors.
---
Defense 4: Probabilistic early refresh (no locks needed)
Another approach avoids coordination entirely: have requests randomly refresh the value shortly before it expires, with probability that rises as expiry approaches. By the time the key would expire, someone has almost certainly already refreshed it.
This is the XFetch algorithm from the paper Optimal Probabilistic Cache Stampede Prevention (Vattani et al.). Store how long the value took to compute (delta), and on each read decide:
// Refresh early if: now - delta * beta * ln(random()) >= expiry
function shouldRefreshEarly(entry: { softExp: number; delta: number }, beta = 1): boolean {
return Date.now() - entry.delta * beta * Math.log(Math.random()) >= entry.softExp;
}
Math.log(Math.random()) is negative, so the subtraction pushes "now" forward by a random amount. The expression is scaled by delta:
- Expensive values (large delta) start refreshing earlier, because they need more lead time.
- Cheap values refresh almost right at expiry.
- beta (default 1) tunes aggressiveness. Higher means earlier refreshes and more cache churn.
Used alone, XFetch can still let a few concurrent requests decide to refresh simultaneously, but instead of hundreds it's a handful. Combined with single-flight or a lock, it's very effective.
---
Defense 5: Prevent synchronized expiry with TTL jitter
If you warm 10,000 keys at deploy time with EX 3600, they all expire in the same second an hour later. That's the avalanche pattern. The fix is one line: add randomness.
const jitter = (ms: number, spread = 0.1) =>
Math.round(ms * (1 - spread + Math.random() * spread * 2)); // ±10%
await redis.set(key, value, 'PX', jitter(60 * 60 * 1000));
Keys written together now expire spread across a window instead of all at once. It's trivial, it costs nothing, and it should be your default for every cached key.
---
Defense 6: Negative caching (stop penetration)
If a client requests /users/999999999 (an ID that doesn't exist), the cache never has it, so every request goes to the database, forever. Scanners and bugs generate these constantly.
Cache the absence too, with a short TTL:
// Wrap values so that "null" is a legitimate cached result, distinct from "key not in cache"
const entry = { v: user ?? null, softExp: Date.now() + (user ? 60_000 : 10_000) };
Use a shorter TTL for "not found" (seconds, not minutes) so a newly created record shows up quickly, or invalidate that key on create. This is why the entry wrapper above stores v inside an object. A raw null would be indistinguishable from a miss.
For very large, hostile key spaces, a Bloom filter in front of the cache can reject IDs that definitely don't exist, but start with negative caching.
---
Putting it together: a reusable helper
Here's a compact implementation combining single-flight, Redis lock, stale-while-revalidate, XFetch, jitter, and a safe fallback if Redis is unhealthy:
// cache.ts
import { randomUUID } from 'node:crypto';
import type Redis from 'ioredis';
interface Entry<T> { v: T; softExp: number; delta: number }
export interface CacheOpts {
ttlMs: number; // how long the value is considered fresh
staleMs?: number; // extra time it may be served stale while refreshing (default: 5x ttlMs)
lockMs?: number; // lock safety TTL: must exceed worst-case load time
waitMs?: number; // how long a cold-miss waiter polls for another loader's result
}
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
const jitter = (ms: number) => Math.round(ms * (0.9 + Math.random() * 0.2));
const RELEASE = `
if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end`;
export function createCache(redis: Redis, log = console) {
const inflight = new Map<string, Promise<unknown>>();
async function tryLock(key: string, ms: number) {
const token = randomUUID();
const ok = await redis.set(`lock:${key}`, token, 'PX', ms, 'NX');
return ok === 'OK' ? token : null;
}
const unlock = (key: string, token: string) =>
redis.eval(RELEASE, 1, `lock:${key}`, token).catch(() => {});
async function load<T>(key: string, loader: () => Promise<T>, o: Required<CacheOpts>): Promise<T> {
const started = Date.now();
const v = await loader();
const delta = Date.now() - started; // how long the load took (for XFetch)
const softTtl = jitter(o.ttlMs);
const entry: Entry<T> = { v, softExp: Date.now() + softTtl, delta };
await redis.set(key, JSON.stringify(entry), 'PX', softTtl + o.staleMs).catch((err) =>
log.error('cache set failed', err),
);
return v;
}
async function read<T>(key: string, loader: () => Promise<T>, o: Required<CacheOpts>): Promise<T> {
let raw: string | null;
try {
raw = await redis.get(key);
} catch (err) {
log.error('redis get failed; loading directly', err);
return loader(); // fail open (see "When Redis is down")
}
const entry: Entry<T> | null = raw ? JSON.parse(raw) : null;
const now = Date.now();
// 1. Fresh and not selected for early refresh: serve it
if (entry && now < entry.softExp && now - entry.delta * Math.log(Math.random()) < entry.softExp) {
return entry.v;
}
// 2. Stale or about to expire: serve the old value, one refresher updates it in the background
if (entry) {
const token = await tryLock(key, o.lockMs).catch(() => null);
if (token) {
load(key, loader, o)
.catch((err) => log.error('background refresh failed; serving stale', err))
.finally(() => unlock(key, token));
}
return entry.v;
}
// 3. True miss: one caller loads; the others wait briefly for the result
const token = await tryLock(key, o.lockMs).catch(() => null);
if (token) {
try { return await load(key, loader, o); } finally { await unlock(key, token); }
}
const deadline = Date.now() + o.waitMs;
while (Date.now() < deadline) {
await sleep(50 + Math.random() * 50);
const filled = await redis.get(key).catch(() => null);
if (filled) return (JSON.parse(filled) as Entry<T>).v;
}
return load(key, loader, o); // waited long enough: load it ourselves
}
return function cached<T>(key: string, loader: () => Promise<T>, opts: CacheOpts): Promise<T> {
const o: Required<CacheOpts> = {
staleMs: opts.ttlMs * 5,
lockMs: 15_000,
waitMs: 3_000,
...opts,
};
// Single-flight inside this process
const existing = inflight.get(key);
if (existing) return existing as Promise<T>;
const p = read<T>(key, loader, o).finally(() => inflight.delete(key));
inflight.set(key, p);
return p;
};
}
Usage:
const cached = createCache(redis);
app.get('/leaderboard', async (_req, res) => {
const data = await cached('leaderboard:v1', () => db.getLeaderboard(), {
ttlMs: 60_000, // fresh for ~1 minute
staleMs: 5 * 60_000, // may be served stale for up to 5 more minutes while refreshing
});
res.json(data);
});
A walkthrough of what the helper does:
1. Fresh hit: returns immediately (the common path, one Redis GET).
2. Stale or nearing expiry: returns the old value, and the one process that wins the lock refreshes in the background. XFetch randomly pulls some refreshes forward, so expiry rarely arrives "cold".
3. Cold miss: one caller across the fleet loads; the others poll briefly. If they time out, they load themselves rather than failing.
4. TTL jitter is applied to every write.
5. Single-flight collapses concurrent in-process callers before Redis is even consulted.
6. Lock released safely via the Lua compare-and-delete.
Because the value is wrapped in an entry object, caching null (negative caching) just works. Use a shorter ttlMs when the loader returns "not found."
Notes on the choices:
- lockMs must exceed your worst-case loader time, or a slow load will let a second loader in. It's still only a duplicate load, not corruption.
- Version your keys (leaderboard:v1) so you can invalidate by bumping the version when the shape changes. Old entries age out on their own.
- Don't cache errors. The helper only writes on success, so a failing loader leaves the previous (stale) value in place.
---
When Redis itself is down
The try/catch around redis.get makes the helper fail open: if Redis is unreachable, it calls the loader directly. That keeps the app working, but it's also a recipe for a stampede, because now every request goes to the database. Single-flight limits it to one load per key per process, but with many distinct keys the database still takes the full force.
Add backstops at the database edge:
- A concurrency limit on loaders (a semaphore or p-limit) so the database never sees more than N concurrent cache-fill queries
- A circuit breaker around the loader: when it's failing or slow, fail fast or serve stale rather than piling on
- Aggressive timeouts on Redis calls (commandTimeout, connectTimeout in ioredis) so a hung Redis doesn't hang every request
- Load shedding (503 with Retry-After) on non-critical endpoints during the incident
Treat Redis as an optimization that may vanish at any moment, and make sure the database survives that moment.
---
Warming and deploys
Cold caches cause stampedes too, right after a Redis restart, failover, flush, or a deploy that bumps a key version.
- Warm critical keys before taking traffic (a startup script that loads the top N keys), or populate them from a background job
- Roll out gradually. Deploying all instances at once with a new key prefix means all of them miss simultaneously
- Refresh-ahead for known-hot keys: a scheduled job that recomputes the leaderboard every 30 seconds means requests never trigger the load at all
For a few well-known hot keys, refresh-ahead via a scheduler is often simpler and more predictable than any request-driven scheme.
---
Test it: prove the stampede is gone
Don't trust the cache until you've watched it under concurrent load. Count how many times the loader actually runs:
import { it, expect } from 'vitest';
it('runs the loader once for 200 concurrent cold misses across 2 "processes"', async () => {
const cachedA = createCache(redis); // separate in-process maps simulate two instances
const cachedB = createCache(redis);
await redis.flushdb();
let loads = 0;
const loader = async () => { loads++; await sleep(300); return { top: 'alice' }; };
const calls = Array.from({ length: 200 }, (_, i) =>
(i % 2 ? cachedA : cachedB)('leaderboard:test', loader, { ttlMs: 60_000 }),
);
const results = await Promise.all(calls);
expect(results.every((r) => r.top === 'alice')).toBe(true);
expect(loads).toBe(1); // not 200
});
it('serves stale and refreshes once after the soft TTL', async () => {
// set ttlMs small, wait past it, fire N requests, assert they return instantly
// with the old value and that the loader ran exactly once in the background
});
Also run a load test (autocannon or k6) against a staging environment with a short TTL (say 5 seconds) so that you hit expiries constantly, and watch the database's query rate and connection count. A healthy setup shows a flat line with small, steady refresh queries rather than periodic spikes.
---
What to monitor
- Hit rate, split into fresh hits, stale hits, and misses
- Loader concurrency per key. Spikes mean a stampede is forming
- Loader duration (it's also your delta, and shows when queries are slowing)
- Lock contention: how often callers fail to get the lock, and how often waiters time out and load anyway
- Database queries per second and its correlation with cache expiry times
- Redis latency and errors, since the fail-open path is your riskiest moment
---
Choosing defenses
| Technique | Prevents | Cost | Notes |
|---|---|---|---|
| Single-flight (in-process) | Stampede within one process | Tiny | Always do it, but it's per-process |
| Redis lock (SET NX PX) | Stampede across the fleet | A Redis round trip; waiters block | Duplicate work is tolerable, so a simple lock suffices |
| Stale-while-revalidate | User-visible latency on refresh | Serves slightly old data; more memory | Best user experience; adds stale-if-error |
| XFetch (early refresh) | Synchronized expiry on hot keys | Slightly more refreshes | No locking; combine with single-flight |
| TTL jitter | Avalanche (mass expiry) | None | Use on every key |
| Negative caching | Penetration | Small memory | Short TTL for "not found" |
| Refresh-ahead job | Everything for known hot keys | A scheduler | Simple and predictable |
| Loader concurrency limit + breaker | Database overload if everything else fails | Small | The last line of defense |
Checklist
- ✅ Collapse concurrent misses with single-flight
- ✅ Use a lock (SET NX PX + Lua release) so one process per key rebuilds
- ✅ Serve stale while revalidating, so users never wait on a refresh
- ✅ Add TTL jitter to every key
- ✅ Cache "not found" with a short TTL
- ✅ Make the lock TTL longer than the worst-case load time
- ✅ Bound database concurrency, and use timeouts and a circuit breaker for when Redis fails
- ✅ Warm caches before cutover, and roll deploys gradually
- ✅ Test that N concurrent misses cause exactly 1 load
- ✅ Monitor loader concurrency and the stale/fresh/miss split
Key takeaways
1. A cache stampede is a feedback loop. When a hot key expires, every concurrent request misses and runs the expensive query, which slows the database and widens the miss window.
2. Single-flight is the cheapest fix, but it only works within one process. Use a Redis lock for fleet-wide coordination.
3. Stale-while-revalidate is the best user experience. Serve the old value and refresh in the background, and you also get stale-if-error for free.
4. Probabilistic early refresh (XFetch) avoids expiry cliffs without coordination.
5. Add TTL jitter everywhere and cache misses for nonexistent keys to stop avalanches and penetration.
6. Plan for Redis being down. Fail open only if the database has its own concurrency limits and breakers.
7. Prove it with a test that fires hundreds of concurrent cold requests and asserts the loader ran once.
Have you been hit by a stampede, or found a pattern I didn't cover? Share it in the comments.
---
Further reading: the paper "Optimal Probabilistic Cache Stampede Prevention" (Vattani, Chierichetti, Lowenstein), the Redis docs on SET and distributed locks, and the ioredis README.
Redis Cluster Explained: Hash Slots, CROSSSLOT Errors, Lua Scripts, and Pipelining
If you've used standalone Redis and then switched to Redis Cluster, you may have encountered this error:
CROSSSLOT Keys in request don't hash to the same slot
Why does the same code work locally but fail in Redis Cluster?
Let's understand how Redis distributes keys and why certain operations require keys to share the same hash slot.
1. What Are Redis Hash Slots?
Standalone Redis stores all keys on a single server. Redis Cluster distributes them across multiple primary nodes using 16,384 hash slots.
Each key is assigned a slot using:
HASH_SLOT = CRC16(key) % 16384
For example (illustrative slot numbers):
Redis Cluster
|
+-- Node A (Slots 0-5460)
| +-- user:1
|
+-- Node B (Slots 5461-10922)
| +-- user:2
|
+-- Node C (Slots 10923-16383)
+-- user:3
Why use hash slots instead of assigning keys directly to nodes?
Scalability and rebalancing.
When adding or removing nodes, Redis can migrate hash slots between nodes without recalculating every key's placement.
A slot can contain millions or even billions of keys, subject to memory and Redis keyspace limits. However, one slot always belongs to one primary node at a time.
2. Why Does CROSSSLOT Happen?
Consider a Lua script that increments a topic's score while preventing repeated increments using a cooldown key.
if redis.call('EXISTS', KEYS[2]) == 1 then
return 0
end
redis.call('ZINCRBY', KEYS[1], 1, ARGV[1])
redis.call('SET', KEYS[2], '1', 'EX', 30)
return 1
With these keys:
KEYS[1] = app:topics:ranking
KEYS[2] = app:topics:cooldown:123
Both keys may belong to different hash slots.
Redis Cluster cannot execute a Lua script atomically across multiple slots, so it rejects the operation with CROSSSLOT.
Solution: Hash Tags
Redis supports hash tags using {}.
When a key contains a valid hash tag, Redis hashes only the content inside the braces.
app:{topics}:ranking
app:{topics}:cooldown:123
Both keys now hash using topics, guaranteeing the same slot.
You can verify this:
CLUSTER KEYSLOT app:{topics}:ranking
CLUSTER KEYSLOT app:{topics}:cooldown:123
Both return the same slot number.
No changes to the Lua script are necessary.
3. What About MULTI/EXEC and Pipelines?
Redis Cluster has different restrictions depending on the operation.
| Operation | Different hash slots allowed? | Atomic? |
|---|---|---|
| Independent GET / SET | Yes | Individually |
| MGET / MSET | No | Yes |
| MULTI/EXEC | No | Yes |
| EVAL (Lua) | No | Yes |
| ioredis cluster.pipeline() | Only if slots share a node | No |
| ioredis automatic pipelining | Yes, across nodes | No |
MULTI/EXEC
Transactions require all keys to share the same slot.
// May fail with CROSSSLOT
await redis
.multi()
.set('user:1', 'John')
.set('user:2', 'Jane')
.exec();
Using hash tags:
await redis
.multi()
.set('{users}:1', 'John')
.set('{users}:2', 'Jane')
.exec();
Pipeline
Unlike transactions, pipelines are not atomic. They reduce network round trips by batching commands.
With ioredis, an explicit cluster pipeline requires its commands to target slots served by the same node.
const pipeline = redis.pipeline();
pipeline.set('user:1', 'John');
pipeline.set('user:2', 'Jane');
await pipeline.exec(); // May fail across nodes
For independent operations across nodes, use automatic pipelining:
const redis = new Redis.Cluster(nodes, {
enableAutoPipelining: true,
});
await Promise.all([
redis.set('user:1', 'John'),
redis.set('user:2', 'Jane'),
]);
ioredis automatically groups eligible commands by destination node.
4. Be Careful With Hash Tags
Hash tags solve CROSSSLOT, but overusing them can create performance bottlenecks.
Consider:
app:{topics}:ranking
app:{topics}:cooldown:1
app:{topics}:cooldown:2
app:{topics}:cooldown:3
...
app:{topics}:cooldown:1000000
All these keys belong to one hash slot, meaning one primary node handles them all.
Even if you add more Redis nodes, Redis cannot distribute a single slot across multiple primaries.
For user-specific operations, a better design is:
user:{123}:profile
user:{123}:settings
user:{456}:profile
user:{456}:settings
Each user's related keys share a slot, while different users can be distributed across the cluster.
If you need an atomic operation involving one global key and many user-specific keys, you'll need to accept that shared-slot trade-off or redesign the operation.
5. Why Doesn't This Happen With Local Redis?
Standalone Redis doesn't enforce cluster hash-slot restrictions because all keys are stored on one server.
However, a local Redis Cluster with only one primary node still enforces hash-slot rules, even though all 16,384 slots belong to the same node.
If production uses Redis Cluster, consider testing against cluster mode locally to catch these problems early.
Conclusion
Redis Cluster uses 16,384 hash slots to distribute keys and simplify scaling.
Remember:
- Hash slots determine where keys are stored.
- Lua scripts and MULTI/EXEC require all keys to share the same slot.
- Hash tags {} guarantee that related keys share a slot.
- Pipelines improve performance but are not atomic.
- Overusing hash tags can concentrate traffic on a single node.
The next time you encounter CROSSSLOT, don't just add {} to every key. Consider which operations truly require atomicity and how your key design affects scalability.
---
References
- Redis Cluster Specification
- Redis Cluster Scaling
- ioredis GitHub
1 reaction · 2 comments
Mentions
Cache Stampede (Thundering Herd) in Node.js & Redis: Preventing Database Meltdowns on Cache Misses You added Redis in front of an expensive query, and it worked beautifully. The dashboard that took 800 ms now returns in 3 ms. The database CPU dropped from 70% to 10%. Everyone is happy. Then, one afternoon, the database CPU spikes to 100%, latency goes vertical, and the incident channel lights up. The graphs show a sharp, periodic pattern: calm, spike, calm, spike. Nothing deployed. Traffic didn't change. What happened is a cache stampede (also called the thundering herd or dogpile problem). A hot key expired, and for the next few hundred milliseconds, every request that wanted it missed the cache at the same moment and ran the expensive query itself. This post explains exactly how that happens and builds up a set of defenses in Node.js with ioredis, from a ten-line fix to a production-grade helper. --- Anatomy of a stampede Here's the classic cache-aside code: async function getLeaderboard() { const cached = await redis.get('leaderboard'); if (cached) return JSON.parse(cached); const data = await db.query(EXPENSIVE_SQL); // takes 800 ms await redis.set('leaderboard', JSON.stringify(data), 'EX', 60); return data; } Perfectly reasonable. Now watch what happens at the moment the key expires, with 500 requests per second hitting this endpoint: t=0.000 key expires t=0.001 request 1: cache miss → starts the 800 ms query t=0.003 request 2: cache miss → starts the 800 ms query t=0.004 request 3: cache miss → starts the 800 ms query ... t=0.800 request 1 finishes, writes the cache (by now ~400 requests have missed and started the same query) The query takes 800 ms, so for 800 ms every request is a miss. At 500 req/s that's ~400 identical expensive queries launched against the database in under a second, instead of one. The database slows down under that load, which makes the queries take longer, which widens the miss window, which launches more queries. It's a feedback loop. Related problems with similar names People mix these up, but the fixes differ: | Problem | What happens | Typical cause | |---|---|---| | Stampede / thundering herd | Many concurrent misses on one hot key | A popular key expires | | Avalanche | Many different keys expire together, or the cache is gone | Same TTL set in a batch; Redis restart or failover | | Penetration | Requests for data that doesn't exist always miss and hit the database | Missing IDs, scanners, bad input | We'll address all three, since they feed each other. --- Defense 1: Single-flight (collapse concurrent misses in one process) The cheapest fix: if a load for a key is already in progress in this process, wait for it instead of starting another. Store the in-flight promise in a map. const inflight = new Map<string, Promise<unknown>>(); async function singleFlight<T>(key: string, fn: () => Promise<T>): Promise<T> { const existing = inflight.get(key); if (existing) return existing as Promise<T>; const promise = fn().finally(() => inflight.delete(key)); inflight.set(key, promise); return promise; } async function getLeaderboard() { const cached = await redis.get('leaderboard'); if (cached) return JSON.parse(cached); return singleFlight('leaderboard', async () => { const data = await db.query(EXPENSIVE_SQL); await redis.set('leaderboard', JSON.stringify(data), 'EX', 60); return data; }); } Now 400 concurrent misses in one process become one database query, and all 400 callers share its result. Because Node.js is single-threaded, the check-then-set on the map has no race. The limitation is in the name: it's per process. With 20 instances you get up to 20 concurrent loads per expiry, not 400, a big improvement but not a guarantee. For that, you need coordination through Redis. --- Defense 2: A distributed lock so only one process rebuilds the value Use Redis itself to elect one loader across the whole fleet. SET key value NX PX ttl sets the key only if it doesn't exist, atomically: import { randomUUID } from 'node:crypto'; async function tryLock(key: string, ttlMs: number): Promise<string | null> { const token = randomUUID(); const ok = await redis.set(`lock:${key}`, token, 'PX', ttlMs, 'NX'); return ok === 'OK' ? token : null; } // Release only if we still own the lock. Compare-and-delete must be atomic, so use Lua. const RELEASE = ` if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end`; async function unlock(key: string, token: string) { await redis.eval(RELEASE, 1, `lock:${key}`, token); } Two details matter: - The random token + Lua release ensures you never delete someone else's lock. Without it, if your work outlasts the lock's TTL, another process acquires the lock, and your delayed DEL releases their lock. - The lock TTL is a safety net, not a timer. It must exceed your worst-case load time, so a crashed process can't hold the lock forever. Then the miss path becomes: async function getWithLock<T>(key: string, loader: () => Promise<T>, ttlSec: number): Promise<T> { const cached = await redis.get(key); if (cached) return JSON.parse(cached); const token = await tryLock(key, 10_000); if (token) { try { const value = await loader(); await redis.set(key, JSON.stringify(value), 'EX', ttlSec); return value; } finally { await unlock(key, token); } } // Someone else is loading. Wait briefly for their result instead of piling onto the database. for (let i = 0; i < 40; i++) { // ~2s max await sleep(50); const filled = await redis.get(key); if (filled) return JSON.parse(filled); } return loader(); // gave up waiting: load it ourselves } Is this a "correct" distributed lock? Not in the strict sense. A paused process can outlive its lease, and a single Redis node can fail over mid-lock. But for cache filling that doesn't matter: the worst case is that two processes load the same value and write the same result. You're using the lock to reduce duplicate work, not to guard a critical section, so a simple SET NX PX is enough. (If you're protecting something where duplicates are harmful, that's a different problem and needs fencing tokens or a database constraint, not a cache lock.) The downside: waiters are blocked. Everyone who missed is stuck polling for up to the load time. The next defense removes that wait entirely. --- Defense 3: Stale-while-revalidate (never make users wait) The insight: for most cached data, slightly old data is better than a slow response. So don't delete the value when it "expires". Keep it around, mark it stale, and refresh it in the background while still serving the old copy. Store two expirations: a soft TTL (when it becomes stale) and a hard TTL (when Redis actually deletes it), longer by a grace window. interface Entry<T> { v: T; softExp: number; // epoch ms: after this, the value is stale and should be refreshed } async function set<T>(key: string, value: T, softTtlMs: number, graceMs: number) { const entry: Entry<T> = { v: value, softExp: Date.now() + softTtlMs }; await redis.set(key, JSON.stringify(entry), 'PX', softTtlMs + graceMs); } On read: - Fresh (now < softExp): return it. - Stale (past softExp but still in Redis): return it immediately, and if you win the lock, refresh in the background. - Missing (past the hard TTL): load synchronously. This is the only path where someone waits. Now the expensive load happens off the request path. Users see 3 ms responses all the time, and the database sees one refresh per interval. As a bonus, if the refresh fails (the database is struggling), you keep serving the stale copy, a "stale-if-error" safety net that turns a database outage into slightly old data rather than errors. --- Defense 4: Probabilistic early refresh (no locks needed) Another approach avoids coordination entirely: have requests randomly refresh the value shortly before it expires, with probability that rises as expiry approaches. By the time the key would expire, someone has almost certainly already refreshed it. This is the XFetch algorithm from the paper Optimal Probabilistic Cache Stampede Prevention (Vattani et al.). Store how long the value took to compute (delta), and on each read decide: // Refresh early if: now - delta * beta * ln(random()) >= expiry function shouldRefreshEarly(entry: { softExp: number; delta: number }, beta = 1): boolean { return Date.now() - entry.delta * beta * Math.log(Math.random()) >= entry.softExp; } Math.log(Math.random()) is negative, so the subtraction pushes "now" forward by a random amount. The expression is scaled by delta: - Expensive values (large delta) start refreshing earlier, because they need more lead time. - Cheap values refresh almost right at expiry. - beta (default 1) tunes aggressiveness. Higher means earlier refreshes and more cache churn. Used alone, XFetch can still let a few concurrent requests decide to refresh simultaneously, but instead of hundreds it's a handful. Combined with single-flight or a lock, it's very effective. --- Defense 5: Prevent synchronized expiry with TTL jitter If you warm 10,000 keys at deploy time with EX 3600, they all expire in the same second an hour later. That's the avalanche pattern. The fix is one line: add randomness. const jitter = (ms: number, spread = 0.1) => Math.round(ms * (1 - spread + Math.random() * spread * 2)); // ±10% await redis.set(key, value, 'PX', jitter(60 * 60 * 1000)); Keys written together now expire spread across a window instead of all at once. It's trivial, it costs nothing, and it should be your default for every cached key. --- Defense 6: Negative caching (stop penetration) If a client requests /users/999999999 (an ID that doesn't exist), the cache never has it, so every request goes to the database, forever. Scanners and bugs generate these constantly. Cache the absence too, with a short TTL: // Wrap values so that "null" is a legitimate cached result, distinct from "key not in cache" const entry = { v: user ?? null, softExp: Date.now() + (user ? 60_000 : 10_000) }; Use a shorter TTL for "not found" (seconds, not minutes) so a newly created record shows up quickly, or invalidate that key on create. This is why the entry wrapper above stores v inside an object. A raw null would be indistinguishable from a miss. For very large, hostile key spaces, a Bloom filter in front of the cache can reject IDs that definitely don't exist, but start with negative caching. --- Putting it together: a reusable helper Here's a compact implementation combining single-flight, Redis lock, stale-while-revalidate, XFetch, jitter, and a safe fallback if Redis is unhealthy: // cache.ts import { randomUUID } from 'node:crypto'; import type Redis from 'ioredis'; interface Entry<T> { v: T; softExp: number; delta: number } export interface CacheOpts { ttlMs: number; // how long the value is considered fresh staleMs?: number; // extra time it may be served stale while refreshing (default: 5x ttlMs) lockMs?: number; // lock safety TTL: must exceed worst-case load time waitMs?: number; // how long a cold-miss waiter polls for another loader's result } const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); const jitter = (ms: number) => Math.round(ms * (0.9 + Math.random() * 0.2)); const RELEASE = ` if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end`; export function createCache(redis: Redis, log = console) { const inflight = new Map<string, Promise<unknown>>(); async function tryLock(key: string, ms: number) { const token = randomUUID(); const ok = await redis.set(`lock:${key}`, token, 'PX', ms, 'NX'); return ok === 'OK' ? token : null; } const unlock = (key: string, token: string) => redis.eval(RELEASE, 1, `lock:${key}`, token).catch(() => {}); async function load<T>(key: string, loader: () => Promise<T>, o: Required<CacheOpts>): Promise<T> { const started = Date.now(); const v = await loader(); const delta = Date.now() - started; // how long the load took (for XFetch) const softTtl = jitter(o.ttlMs); const entry: Entry<T> = { v, softExp: Date.now() + softTtl, delta }; await redis.set(key, JSON.stringify(entry), 'PX', softTtl + o.staleMs).catch((err) => log.error('cache set failed', err), ); return v; } async function read<T>(key: string, loader: () => Promise<T>, o: Required<CacheOpts>): Promise<T> { let raw: string | null; try { raw = await redis.get(key); } catch (err) { log.error('redis get failed; loading directly', err); return loader(); // fail open (see "When Redis is down") } const entry: Entry<T> | null = raw ? JSON.parse(raw) : null; const now = Date.now(); // 1. Fresh and not selected for early refresh: serve it if (entry && now < entry.softExp && now - entry.delta * Math.log(Math.random()) < entry.softExp) { return entry.v; } // 2. Stale or about to expire: serve the old value, one refresher updates it in the background if (entry) { const token = await tryLock(key, o.lockMs).catch(() => null); if (token) { load(key, loader, o) .catch((err) => log.error('background refresh failed; serving stale', err)) .finally(() => unlock(key, token)); } return entry.v; } // 3. True miss: one caller loads; the others wait briefly for the result const token = await tryLock(key, o.lockMs).catch(() => null); if (token) { try { return await load(key, loader, o); } finally { await unlock(key, token); } } const deadline = Date.now() + o.waitMs; while (Date.now() < deadline) { await sleep(50 + Math.random() * 50); const filled = await redis.get(key).catch(() => null); if (filled) return (JSON.parse(filled) as Entry<T>).v; } return load(key, loader, o); // waited long enough: load it ourselves } return function cached<T>(key: string, loader: () => Promise<T>, opts: CacheOpts): Promise<T> { const o: Required<CacheOpts> = { staleMs: opts.ttlMs * 5, lockMs: 15_000, waitMs: 3_000, ...opts, }; // Single-flight inside this process const existing = inflight.get(key); if (existing) return existing as Promise<T>; const p = read<T>(key, loader, o).finally(() => inflight.delete(key)); inflight.set(key, p); return p; }; } Usage: const cached = createCache(redis); app.get('/leaderboard', async (_req, res) => { const data = await cached('leaderboard:v1', () => db.getLeaderboard(), { ttlMs: 60_000, // fresh for ~1 minute staleMs: 5 * 60_000, // may be served stale for up to 5 more minutes while refreshing }); res.json(data); }); A walkthrough of what the helper does: 1. Fresh hit: returns immediately (the common path, one Redis GET). 2. Stale or nearing expiry: returns the old value, and the one process that wins the lock refreshes in the background. XFetch randomly pulls some refreshes forward, so expiry rarely arrives "cold". 3. Cold miss: one caller across the fleet loads; the others poll briefly. If they time out, they load themselves rather than failing. 4. TTL jitter is applied to every write. 5. Single-flight collapses concurrent in-process callers before Redis is even consulted. 6. Lock released safely via the Lua compare-and-delete. Because the value is wrapped in an entry object, caching null (negative caching) just works. Use a shorter ttlMs when the loader returns "not found." Notes on the choices: - lockMs must exceed your worst-case loader time, or a slow load will let a second loader in. It's still only a duplicate load, not corruption. - Version your keys (leaderboard:v1) so you can invalidate by bumping the version when the shape changes. Old entries age out on their own. - Don't cache errors. The helper only writes on success, so a failing loader leaves the previous (stale) value in place. --- When Redis itself is down The try/catch around redis.get makes the helper fail open: if Redis is unreachable, it calls the loader directly. That keeps the app working, but it's also a recipe for a stampede, because now every request goes to the database. Single-flight limits it to one load per key per process, but with many distinct keys the database still takes the full force. Add backstops at the database edge: - A concurrency limit on loaders (a semaphore or p-limit) so the database never sees more than N concurrent cache-fill queries - A circuit breaker around the loader: when it's failing or slow, fail fast or serve stale rather than piling on - Aggressive timeouts on Redis calls (commandTimeout, connectTimeout in ioredis) so a hung Redis doesn't hang every request - Load shedding (503 with Retry-After) on non-critical endpoints during the incident Treat Redis as an optimization that may vanish at any moment, and make sure the database survives that moment. --- Warming and deploys Cold caches cause stampedes too, right after a Redis restart, failover, flush, or a deploy that bumps a key version. - Warm critical keys before taking traffic (a startup script that loads the top N keys), or populate them from a background job - Roll out gradually. Deploying all instances at once with a new key prefix means all of them miss simultaneously - Refresh-ahead for known-hot keys: a scheduled job that recomputes the leaderboard every 30 seconds means requests never trigger the load at all For a few well-known hot keys, refresh-ahead via a scheduler is often simpler and more predictable than any request-driven scheme. --- Test it: prove the stampede is gone Don't trust the cache until you've watched it under concurrent load. Count how many times the loader actually runs: import { it, expect } from 'vitest'; it('runs the loader once for 200 concurrent cold misses across 2 "processes"', async () => { const cachedA = createCache(redis); // separate in-process maps simulate two instances const cachedB = createCache(redis); await redis.flushdb(); let loads = 0; const loader = async () => { loads++; await sleep(300); return { top: 'alice' }; }; const calls = Array.from({ length: 200 }, (_, i) => (i % 2 ? cachedA : cachedB)('leaderboard:test', loader, { ttlMs: 60_000 }), ); const results = await Promise.all(calls); expect(results.every((r) => r.top === 'alice')).toBe(true); expect(loads).toBe(1); // not 200 }); it('serves stale and refreshes once after the soft TTL', async () => { // set ttlMs small, wait past it, fire N requests, assert they return instantly // with the old value and that the loader ran exactly once in the background }); Also run a load test (autocannon or k6) against a staging environment with a short TTL (say 5 seconds) so that you hit expiries constantly, and watch the database's query rate and connection count. A healthy setup shows a flat line with small, steady refresh queries rather than periodic spikes. --- What to monitor - Hit rate, split into fresh hits, stale hits, and misses - Loader concurrency per key. Spikes mean a stampede is forming - Loader duration (it's also your delta, and shows when queries are slowing) - Lock contention: how often callers fail to get the lock, and how often waiters time out and load anyway - Database queries per second and its correlation with cache expiry times - Redis latency and errors, since the fail-open path is your riskiest moment --- Choosing defenses | Technique | Prevents | Cost | Notes | |---|---|---|---| | Single-flight (in-process) | Stampede within one process | Tiny | Always do it, but it's per-process | | Redis lock (SET NX PX) | Stampede across the fleet | A Redis round trip; waiters block | Duplicate work is tolerable, so a simple lock suffices | | Stale-while-revalidate | User-visible latency on refresh | Serves slightly old data; more memory | Best user experience; adds stale-if-error | | XFetch (early refresh) | Synchronized expiry on hot keys | Slightly more refreshes | No locking; combine with single-flight | | TTL jitter | Avalanche (mass expiry) | None | Use on every key | | Negative caching | Penetration | Small memory | Short TTL for "not found" | | Refresh-ahead job | Everything for known hot keys | A scheduler | Simple and predictable | | Loader concurrency limit + breaker | Database overload if everything else fails | Small | The last line of defense | Checklist - ✅ Collapse concurrent misses with single-flight - ✅ Use a lock (SET NX PX + Lua release) so one process per key rebuilds - ✅ Serve stale while revalidating, so users never wait on a refresh - ✅ Add TTL jitter to every key - ✅ Cache "not found" with a short TTL - ✅ Make the lock TTL longer than the worst-case load time - ✅ Bound database concurrency, and use timeouts and a circuit breaker for when Redis fails - ✅ Warm caches before cutover, and roll deploys gradually - ✅ Test that N concurrent misses cause exactly 1 load - ✅ Monitor loader concurrency and the stale/fresh/miss split Key takeaways 1. A cache stampede is a feedback loop. When a hot key expires, every concurrent request misses and runs the expensive query, which slows the database and widens the miss window. 2. Single-flight is the cheapest fix, but it only works within one process. Use a Redis lock for fleet-wide coordination. 3. Stale-while-revalidate is the best user experience. Serve the old value and refresh in the background, and you also get stale-if-error for free. 4. Probabilistic early refresh (XFetch) avoids expiry cliffs without coordination. 5. Add TTL jitter everywhere and cache misses for nonexistent keys to stop avalanches and penetration. 6. Plan for Redis being down. Fail open only if the database has its own concurrency limits and breakers. 7. Prove it with a test that fires hundreds of concurrent cold requests and asserts the loader ran once. Have you been hit by a stampede, or found a pattern I didn't cover? Share it in the comments. --- Further reading: the paper "Optimal Probabilistic Cache Stampede Prevention" (Vattani, Chierichetti, Lowenstein), the Redis docs on SET and distributed locks, and the ioredis README.
Redis Cluster Explained: Hash Slots, CROSSSLOT Errors, Lua Scripts, and Pipelining If you've used standalone Redis and then switched to Redis Cluster, you may have encountered this error: CROSSSLOT Keys in request don't hash to the same slot Why does the same code work locally but fail in Redis Cluster? Let's understand how Redis distributes keys and why certain operations require keys to share the same hash slot. 1. What Are Redis Hash Slots? Standalone Redis stores all keys on a single server. Redis Cluster distributes them across multiple primary nodes using 16,384 hash slots. Each key is assigned a slot using: HASH_SLOT = CRC16(key) % 16384 For example (illustrative slot numbers): Redis Cluster | +-- Node A (Slots 0-5460) | +-- user:1 | +-- Node B (Slots 5461-10922) | +-- user:2 | +-- Node C (Slots 10923-16383) +-- user:3 Why use hash slots instead of assigning keys directly to nodes? Scalability and rebalancing. When adding or removing nodes, Redis can migrate hash slots between nodes without recalculating every key's placement. A slot can contain millions or even billions of keys, subject to memory and Redis keyspace limits. However, one slot always belongs to one primary node at a time. 2. Why Does CROSSSLOT Happen? Consider a Lua script that increments a topic's score while preventing repeated increments using a cooldown key. if redis.call('EXISTS', KEYS[2]) == 1 then return 0 end redis.call('ZINCRBY', KEYS[1], 1, ARGV[1]) redis.call('SET', KEYS[2], '1', 'EX', 30) return 1 With these keys: KEYS[1] = app:topics:ranking KEYS[2] = app:topics:cooldown:123 Both keys may belong to different hash slots. Redis Cluster cannot execute a Lua script atomically across multiple slots, so it rejects the operation with CROSSSLOT. Solution: Hash Tags Redis supports hash tags using {}. When a key contains a valid hash tag, Redis hashes only the content inside the braces. app:{topics}:ranking app:{topics}:cooldown:123 Both keys now hash using topics, guaranteeing the same slot. You can verify this: CLUSTER KEYSLOT app:{topics}:ranking CLUSTER KEYSLOT app:{topics}:cooldown:123 Both return the same slot number. No changes to the Lua script are necessary. 3. What About MULTI/EXEC and Pipelines? Redis Cluster has different restrictions depending on the operation. | Operation | Different hash slots allowed? | Atomic? | |---|---|---| | Independent GET / SET | Yes | Individually | | MGET / MSET | No | Yes | | MULTI/EXEC | No | Yes | | EVAL (Lua) | No | Yes | | ioredis cluster.pipeline() | Only if slots share a node | No | | ioredis automatic pipelining | Yes, across nodes | No | MULTI/EXEC Transactions require all keys to share the same slot. // May fail with CROSSSLOT await redis .multi() .set('user:1', 'John') .set('user:2', 'Jane') .exec(); Using hash tags: await redis .multi() .set('{users}:1', 'John') .set('{users}:2', 'Jane') .exec(); Pipeline Unlike transactions, pipelines are not atomic. They reduce network round trips by batching commands. With ioredis, an explicit cluster pipeline requires its commands to target slots served by the same node. const pipeline = redis.pipeline(); pipeline.set('user:1', 'John'); pipeline.set('user:2', 'Jane'); await pipeline.exec(); // May fail across nodes For independent operations across nodes, use automatic pipelining: const redis = new Redis.Cluster(nodes, { enableAutoPipelining: true, }); await Promise.all([ redis.set('user:1', 'John'), redis.set('user:2', 'Jane'), ]); ioredis automatically groups eligible commands by destination node. 4. Be Careful With Hash Tags Hash tags solve CROSSSLOT, but overusing them can create performance bottlenecks. Consider: app:{topics}:ranking app:{topics}:cooldown:1 app:{topics}:cooldown:2 app:{topics}:cooldown:3 ... app:{topics}:cooldown:1000000 All these keys belong to one hash slot, meaning one primary node handles them all. Even if you add more Redis nodes, Redis cannot distribute a single slot across multiple primaries. For user-specific operations, a better design is: user:{123}:profile user:{123}:settings user:{456}:profile user:{456}:settings Each user's related keys share a slot, while different users can be distributed across the cluster. If you need an atomic operation involving one global key and many user-specific keys, you'll need to accept that shared-slot trade-off or redesign the operation. 5. Why Doesn't This Happen With Local Redis? Standalone Redis doesn't enforce cluster hash-slot restrictions because all keys are stored on one server. However, a local Redis Cluster with only one primary node still enforces hash-slot rules, even though all 16,384 slots belong to the same node. If production uses Redis Cluster, consider testing against cluster mode locally to catch these problems early. Conclusion Redis Cluster uses 16,384 hash slots to distribute keys and simplify scaling. Remember: - Hash slots determine where keys are stored. - Lua scripts and MULTI/EXEC require all keys to share the same slot. - Hash tags {} guarantee that related keys share a slot. - Pipelines improve performance but are not atomic. - Overusing hash tags can concentrate traffic on a single node. The next time you encounter CROSSSLOT, don't just add {} to every key. Consider which operations truly require atomicity and how your key design affects scalability. --- References - Redis Cluster Specification - Redis Cluster Scaling - ioredis GitHub