Skip to content

Build1 publisher3 min readPublished

Stale-while-revalidate only works if the cached value outlives its own TTL

A dev.to walkthrough stacks stale-while-revalidate, single-flight coalescing and XFetch to hold a hot key to roughly one origin rebuild, and that guarantee rests on the lock TTL, the physical expiry and the replica count.

The Engineer · Build desk

Illustration accompanying Stale-while-revalidate only works if the cached value outlives its own TTL

What happened

  • A cache stampede starts when one popular entry expires and hundreds or thousands of concurrent requests race to rebuild it, hammering the origin while retries amplify the load.
  • A dev.to post prescribes three patterns for it: stale-while-revalidate behind a distributed lock, in-process single-flight coalescing, and probabilistic early expiration in the XFetch style.
  • The SWR snippet acquires a lock with SET NX and a millisecond expiry, recomputes, then releases through a Lua script that deletes the key only if the stored token still matches.
  • The post says the approach keeps origin queries at about one per key and eliminates p99 spikes at high QPS, at the price of a short window of staleness.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint Coalescing inside the process cannot take origin concurrency below one rebuild per replica, so the distributed lock is the only piece that can deliver a single-query guarantee for a fleet.
  • decision Switching SWR on forces a per-key ruling on whether a stale answer is acceptable, and that call belongs to whoever owns the product semantics.
  • cost The lock TTL has to come from a measured p99 of the origin call itself, so adoption starts with instrumenting the recompute path rather than with the lock.
  • capability XFetch spreads refreshes in time with no lock and no shared state in steady state, which suits keys where an occasional duplicate rebuild costs less than a coordination round trip.

The line that decides whether stale-while-revalidate works is the one the losers take. A worker that fails to acquire the lock calls `redis.get` on the cache key and returns whatever comes back [11]. The winner wrote that value with `ex=base_ttl` [10]. Once that expiry fires in Redis there is no stale copy left, and the loser returns nothing. The pattern needs the entry to outlive the freshness window it is revalidating against. In practice that means a physical expiry longer than the logical one, with the logical expiry stored next to the value [22]. The XFetch section of the same post already keeps an expiry and the last recompute duration alongside the cached value [21].

Two implementation tips matter more than the code they annotate. Set the lock TTL longer than p99 recompute time, tuned from metrics [13]. Release the lock with a Lua compare-and-del [12]. The second exists because the first can fail. If a recompute runs past the lock TTL, Redis drops the lock, another worker takes it, and the first worker finishes and tries to delete a lock it no longer owns [23]. The post's own account of a stampede has retries amplifying load on the origin [1], and an overrun is most likely under exactly that condition [23].

Single-flight is in-process: concurrent callers for one key await the same promise, and a pod's single-flight does not dedupe across pods [16][17]. The floor it can reach is one rebuild per replica, so the origin still sees one concurrent rebuild for every replica you run [24]. For extreme concurrency the post says to combine it with a distributed lock [17]. The costs it names are per-process memory for the in-flight entries and an async-friendly call chain, plus timeouts on the loaders so a hung one does not park its waiters forever [19][18].

XFetch is the one pattern that needs no coordination in steady state [20]. Its decision samples a gap of minus delta times beta times the natural log of a random draw [21]. The log of a uniform draw below one is negative, so the gap is positive and scales with delta, the last recompute duration stored with the value: keys that were expensive to build start trying to refresh earlier than cheap ones [25]. The refresh still attempts a background lock before it runs [21].

"You don't need a big architectural rewrite to fix this," the author wrote [2], and the three patterns are offered as composable and deployable incrementally [4]. TTL jitter is the baseline [4], and it answers two of the four triggers the post lists: identical TTLs from batch warms and from deploys [6]. Jitter decorrelates expiries from each other. A single hot key has one expiry, so jitter cannot reduce the simultaneous misses on it [26], and pod restarts and memory-pressure evictions are outside its reach too [6].

The post presents all three patterns without a measurement [27]. "Eliminates p99 spikes during high QPS" is a claim about the author's traffic [9]. It transfers to yours only if the read path tolerates a stale answer for the length of the revalidation window [15], p99 recompute fits inside the lock TTL [13], and a stale copy is still in the cache when the losers ask for it [22].

What to watch

  • A measured before-and-after: origin queries per hot key, with the replica count stated, would show whether the one-query-per-key figure holds outside the author's traffic.
  • Whether adopters store a logical expiry beside the value or simply lengthen the physical TTL, since the two behave differently after a memory-pressure eviction.
  • Published guidance on choosing beta, which sets how far before expiry the XFetch sampling starts firing.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories