cache-stampede-lab

Every hot key expires at once

When a popular cached value expires, every request that arrives before it is rebuilt goes to the database for the same answer. Below, one seeded workload runs against five policies: a plain TTL, single-flight, a memcache-style lease, XFetch early expiration and stale-while-revalidate. Everything runs in this tab.

1. The expiry

One key, read across 8 app servers, cached for 5 seconds. Rebuilding it is a query on a database that runs 16 at a time and queues the rest. Each chart shows database queries queued or running over 60 seconds, with a tick at every expiry.

2. Why XFetch spreads out

Each read refreshes early with probability exp(-x / (delta * beta)), where x is the time left before expiry and delta is how long the last rebuild took. The curve is that chance for a single read. With thousands of reads a second the first vote lands well before expiry, and the rebuild finishes before anyone sees a miss.

3. In code

createCache is a read-through cache with all three defences. Misses on one key share a single loader call, the loader's measured duration becomes delta for XFetch, and an optional stale window serves the old value while one refresh runs.

  1. Vattani, Chierichetti, Lowenstein, Optimal Probabilistic Cache Stampede Prevention, VLDB 2015
  2. Nishtala et al., Scaling Memcache at Facebook, NSDI 2013
  3. RFC 5861, HTTP Cache-Control Extensions for Stale Content
  4. golang.org/x/sync/singleflight
import { createCache } from "./src/cache.js";

const cache = createCache({
  ttl: 5_000,     // ms
  beta: 1,        // XFetch; 0 turns it off
  staleMs: 2_000, // serve stale while refreshing
});

const user = await cache.get(`user:${id}`, () =>
  db.query("select * from users where id = $1", [id]));