Shrinking DNS cache entries by half made Cloudflare's inserts 43% faster
Big Pineapple holds more than 250 billion cache entries, so one wasted byte per entry is 250GB of fleet RAM. Cloudflare's fix was mostly a choice of Rust container types, and fewer allocations made the cache quicker rather than slower.
The Engineer · Build desk
What happened
- Cloudflare says the Big Pineapple platform behind 1.1.1.1 holds over 250 billion DNS cache entries at any moment, which puts the price of one wasted byte per entry above 250GB of fleet memory.
- Five successive changes to the in-memory layout more than halved the per-entry footprint and freed roughly 100 terabytes, which Cloudflare equates to the RAM in 130 of its Gen 13 servers.
- The smaller entries also ran quicker, with insert throughput up 43% and lookup latency down 19%, which Cloudflare credits to fewer allocations and better memory locality.
- Swapping eight Vec and String fields for Box<[T]> and Box<str> removed an 8-byte capacity field from each, 64 bytes per entry, worth over 15 terabytes once the reclaimed heap slack is counted.
- The answer, authority and additional record sections were merged into one list addressed by 2-byte u16 offsets, in place of the 8-byte pointer each separate list required.
Why it matters
- Who has to decide decision Capacity planning for a store this size begins at the struct definition, because the container type approved in code review moves fleet RAM by tens of terabytes before anyone signs a hardware order.
- What it limits constraint The speed result only travels to caches whose insert path is allocator-bound; where parsing or socket work dominates, halving the footprint buys memory and nothing else.
- What's now possible capability Because ECS locations cache several versions of each query and pay more memory per entry, the recovered headroom concentrates where per-entry cost was highest rather than spreading evenly across data centers.
Do the division first. Roughly 100 terabytes recovered across more than 250 billion entries is about 400 bytes per entry, or 440 if those terabytes are binary [20]. The cut was more than half, so the entry it started from sat under 900 bytes [24]. The savings the post itemises come out of the container rather than the records: a capacity word that will never be read, heap slots reserved for records that never arrive, and pointers that could have been offsets [7][8][12].
Units are load-bearing at this size. Eight bytes on each of eight fields is 64 bytes per entry [10], and 64 bytes times 250 billion entries is 1.6e13 bytes, which is 16 TB decimal and 14.6 TiB binary [21]. Cloudflare puts the fields-plus-heap-slack figure at over 15 terabytes [11], and that reconciles cleanly only on the binary reading [23]. Same bytes either way, but the label decides how many servers you think you got back.
The mechanism is three machine words. A `Vec<T>` carries a pointer, a length, and a capacity, and every push compares length against capacity to decide whether it must reallocate [6]. A cached DNS response is written once and then read until it expires [7]. The comparison never fires, the capacity is never consulted, and the growth room the `Vec` reserved on the heap is never filled: eight slots allocated, five records stored, three paid for and empty [8]. `Box<[T]>` and `Box<str>` are the same data with the growth machinery deleted [9]. The section merge is the same argument applied to indirection. Three separate lists mean three 8-byte pointers; one list with `u16` offsets costs 2 bytes per section, because a DNS section's record count fits in 16 bits [12].
The throughput and latency gains were measured on a synthetic fill: 56% A records, 25% AAAA, 19% TXT, one to four records per entry, with TXT randomised between 64 and 224 bytes to stand in for every variable-length type [14][15]. For those gains to appear in your cache, the insert path has to be dominated by allocator calls rather than parsing or socket work, and the entry has to be a scatter of small allocations rather than one packed buffer. It also has to be immutable after insert. That last condition is the one that disqualifies most people. If anything appends to a stored entry later, `Box<[T]>` is unavailable and the capacity word comes back.
The part I would copy is the instrumentation. Cloudflare wrapped Rust's `System` allocator to record the count and size of allocations per cache entry [16], which turns "this struct is fat" into a number you can regress against in CI. The post also says plainly that the synthetic inputs approximate production rather than reproduce it, that resident memory moves with traffic mix, cache occupancy, allocator state and memory outside the cache, and that resident memory was tracked across production instances during the rollout [17]. That step is what makes the 100 terabytes a measurement instead of a multiplication. The arithmetic behind the server analogy survives contact too: 100 TB over 130 Gen 13 machines is about 769 GB each [22], a plausible configuration rather than a flattering one.
What to watch
- Whether converting a Vec into Box<[T]> costs an extra copy on the insert path, which the published benchmark numbers do not isolate.
- Per-datacenter resident memory in ECS-heavy locations, where the per-entry saving compounds with the extra cached versions of each query.
- Whether Cloudflare spends the freed headroom by raising the maximum entry count, converting saved bytes into cache hit rate instead of idle RAM.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence62
- Adoption75
- Hype gap+8
- Incentives55
- Confidence66
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
Big Pineapple, the platform behind 1.1.1.1, Gateway DNS, DNS Firewall, AS112 and several other Cloudflare DNS services, stores over 250 billion DNS cache entries at any given time.
- [2]
At 250 billion entries, wasting a single byte per entry costs more than 250 gigabytes of memory across Cloudflare's fleet.
- [3]
Five successive changes to how cache entries are stored in memory cut the per-entry footprint by over 50%.
- [4]
Across the fleet the changes freed roughly 100 terabytes of memory, which Cloudflare says is equivalent to the RAM in 130 of its Gen 13 servers.
- [5]
Insert throughput rose 43% and lookup latency dropped 19%; Cloudflare attributes this to fewer allocations and better memory locality, saying it did not trade speed for space.
- [6]
A Rust Vec<T> stores three fields, a pointer to heap-allocated data, the current length and the total capacity; pushing an item checks whether the length exceeds the capacity and reallocates if needed.
- [7]
Once a DNS response is stored in the cache Cloudflare never modifies it again, so the Vec capacity field serves no purpose but still costs 8 bytes per Vec.
- [8]
Over-allocated heap space is wasted as well: a Vec with capacity for eight items but only five stored leaves three slots unused on the heap.
- [9]
Box<[T]> cannot grow after creation, so it needs no capacity field and reserves no space for future elements; Box<str> likewise drops the capacity field that String carries.
- [10]
Each cache entry stores 8 Vec and String fields, and replacing them with Box<[T]> and Box<str> saves 8 bytes per field, 64 bytes per entry.
- [11]
Combining the removed capacity fields with the eliminated excess heap reservation, Cloudflare reports combined savings of over 15 terabytes across more than 250 billion cache entries.
- [12]
Instead of separate lists for the answer, authority and additional sections, Cloudflare stores a single list with offsets to the start of each section, using a u16 (2 bytes) per offset because per-section DNS record counts fit in a u16, compared with an 8-byte pointer.
- [13]
Big Pineapple starts with an empty cache on cold start and fills until it hits its maximum entry count, at which point older or less popular items are evicted; the exact cache size varies by data center.
- [14]
Memory impact was benchmarked by filling the cache with randomly generated entries roughly matching Cloudflare's production traffic distribution: 56% A records, 25% AAAA and 19% TXT, with each entry containing between one and four records.
- [15]
TXT records serve as a stand-in for all non-A/AAAA record types in the benchmark, with size randomised between 64 and 224 bytes, close to the average response size Cloudflare sees for variable-length record types.
- [16]
Cloudflare tracked memory usage with a custom allocator that wraps Rust's System allocator and records the number and size of allocations per cache entry, alongside insert throughput and lookup latency across the full cache flow.
- [17]
Cloudflare states the benchmark inputs approximate production rather than reproduce it exactly, that process memory also depends on traffic mix, cache occupancy, allocator state and memory used outside the cache, and that it therefore measured resident memory across production instances during the rollout.
- [18]
When EDNS Client Subnet is in use, authoritative servers return different answers depending on the client's network, so Cloudflare caches multiple versions of the same query, increasing both the number of entries and the memory each one consumes and making the optimizations especially impactful for ECS-heavy locations.
- [19]
A CacheEntry holds a timestamp, an inception Instant, a TTL, a u32 hit counter, and vectors for answers, authority, additional records and extended errors; the CacheKey holds qname, qtype, an authenticated flag and a tag byte vector.
- [20]
Roughly 100 terabytes freed across more than 250 billion entries works out to about 400 bytes saved per entry using decimal terabytes, or about 440 bytes using binary terabytes.
- [21]
64 bytes per entry across 250 billion entries is 1.6e13 bytes, which is 16 TB in decimal units and 14.6 TiB in binary units.
- [22]
100 terabytes spread over 130 Gen 13 servers implies about 769 GB of RAM per server.
- [23]
The capacity-field savings alone are 14.6 TiB, so Cloudflare's figure of over 15 terabytes for fields plus reclaimed heap slack adds up on binary units; on decimal units the field savings by themselves already exceed 16 TB.
- [24]
Because the per-entry footprint fell by more than 50%, the pre-change entry was smaller than about 900 bytes.
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
2 independent publishers whose own reporting we read for this story.
- blog.cloudflare.comHow we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache
1 article · August 27, 2026
- tomshardware.comCloudflare frees up 100TB of RAM by shrinking 1.1.1.1's DNS cache entries — 250 billion cached DNS entries at any given time means one wasted byte costs 250GB
1 article · August 28, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.
Entities
- Big PineappleFollow
- EDNS Client SubnetFollow
- RustFollow
- 1.1.1.1Follow
- CloudflareFollow
- Tom's HardwareFollow
- Sebastiaan NeuteboomFollow