Cloudflare published a deep-dive on August 27 about five memory-layout optimizations to Big Pineapple, the Rust-based DNS cache behind 1.1.1.1, DNS Firewall, and their other DNS services. The headline numbers: per-entry memory dropped from 953 bytes to 420 bytes — a 56% cut — which at 250+ billion cached entries fleet-wide freed roughly 100TB of RAM. As a side effect of the leaner layout, insert throughput rose from 625K to 893K entries/sec and lookup latency dropped from 828ns to 670ns. That’s not a typo — TB, not GB, and the perf wins came free with the memory wins.

Most of us aren’t running a DNS cache handling Cloudflare’s query volume. But I read posts like this the way I read query planner internals: not because I’ll ever operate at that scale, but because the techniques are directly reusable in any hot-path Rust or C++ service holding millions of small, long-lived objects in memory — caches, session stores, in-memory indexes. I went through the writeup and mapped each technique to problems I’ve actually hit in production services.

The techniques, and where they generalize

1. Struct field reordering / packing. Rust (like C) doesn’t guarantee field order matches declaration order, but naive struct definitions still waste bytes on padding when fields of different sizes are interleaved. A struct like:

struct DnsEntry {
    ttl: u32,       // 4 bytes
    flags: bool,    // 1 byte + 3 padding
    record_type: u16, // 2 bytes + 2 padding
    expiry: u64,    // 8 bytes
}

reordered by descending size eliminates most of that padding:

struct DnsEntry {
    expiry: u64,      // 8 bytes
    ttl: u32,         // 4 bytes
    record_type: u16, // 2 bytes
    flags: bool,      // 1 byte + 1 padding
}

This alone is often a 10-20% win on structs with mixed-width fields, and it costs nothing — #[repr(Rust)] already reorders automatically in recent compilers for non-#[repr(C)] types, but plenty of teams pin #[repr(C)] for FFI or serialization reasons and pay the padding tax without realizing it. First thing I checked in our own Rust services after reading this: which structs we’ve pinned to repr(C) unnecessarily.

2. Arena / bump allocation for short-lived batches. Instead of individually heap-allocating each cache entry (and paying allocator overhead + fragmentation per allocation), Big Pineapple groups entries into arenas allocated in bulk and freed together. This is the same idea behind bumpalo in the Rust ecosystem or slab allocators in C. The generalizable lesson: if you have objects with correlated lifetimes — a batch of records that all expire together, or all requests within one connection — arena allocation turns N small malloc/free calls into one bulk allocation and one bulk free. I’ve used this pattern in a request-scoped parser before; the win was less about memory footprint and more about eliminating allocator lock contention under load.

3. String interning for repeated values. DNS records repeat domain suffixes and record types constantly. Interning — storing one canonical copy and referencing it by a small integer ID instead of duplicating the string — is old wisdom, but it’s easy to skip when you’re moving fast. If you’re storing user-facing enums, category tags, or repeated hostnames as String in a hot cache, this is almost free money. A HashMap<String, u32> intern table plus u32 references instead of String clones is a mechanical refactor with an outsized payoff at scale.

4. Bitpacking flags and small enums. Boolean flags and small enums (record type, cache state, priority) get packed into bitfields instead of full bytes/words. Rust’s bitflags crate or a manual u8 bitmask does this. Small per-entry, but multiplied by billions of entries, it adds up — this is exactly the “56% isn’t one big idea, it’s five small ones compounding” story Cloudflare is telling.

5. Fixed-size boxed slices instead of growable containers. The single biggest win Cloudflare called out (over 15TB of the 100TB total) was replacing Vec<T> and String with fixed-size boxed slices (Box<[T]>, Box<str>) for cache entries. Vec and String carry a capacity field alongside length because they’re designed to grow — but a DNS record, once cached, never grows. Paying for spare capacity on 250 billion immutable entries is pure waste. The generalizable lesson: any time you’re storing a collection you know is immutable after construction, Box<[T]> (via .into_boxed_slice()) or Box<str> (via .into_boxed_str()) drops the capacity field for free. This is the classic hot/cold and mutable/immutable data-splitting instinct from data-oriented design, applied at the type level instead of by hand-splitting structs.

The honest scoping

None of these techniques are individually novel — they’re textbook systems programming. What’s valuable about Cloudflare’s writeup is the composition and measurement: five well-known tricks, applied deliberately to one high-cardinality cache, with a real before/after number (56%, ~100TB fleet-wide) instead of a vague “we optimized memory usage.” That’s the part worth stealing for your own team: pick one hot-path in-memory structure, measure per-entry overhead with a heap profiler (heaptrack, dhat, or even std::mem::size_of sanity checks), and apply these five in order of effort — struct reordering and bitpacking cost an afternoon, arena allocation and hot/cold splitting are a real refactor. Don’t do the refactor first; measure first, or you’re optimizing a struct nobody’s holding a billion copies of.

Source: Cloudflare: How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache

Export for reading

Comments