Cloudflare frees 100 TB of RAM by slimming its 1.1.1.1 DNS cache
Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
Cloudflare's Big Pineapple platform stores over 250 billion DNS cache entries at any given time. By making five successive changes to how cache entries are stored in memory, the company cut the per-entry footprint by over 50%, freeing roughly 100 terabytes of memory across its fleet—equivalent to the RAM in 130 Gen 13 servers. The optimizations also improved performance: insert throughput rose 43% and lookup latency dropped 19%. The changes include replacing Vec with Box<[T]>, consolidating record sections, dropping redundant owner names, boxing large enum variants, and storing records in wire format.
At that scale, wasting a single byte per entry costs more than 250 gigabytes of memory across our fleet.
- lpapez
This is the right way to deliver software.
Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs.
In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.
- irdc
This is why system programming still matters.
Looks like they're missing the obvious optimisation of putting the record data right after the CacheEntry members instead of allocating memory separately though. But that might just be me as a C-programmer talking and not be all that easy in Rust.
- strenholme
With my own MaraDNS, I aggressively optimized the memory usage of blacklist entries by having a single really big malloc() to allocate the memory for the entries, then traversing that memory block for potentially blacklisted entries.
When I was using one malloc() per entry, a large blacklist took up 237 megabytes of memory. The same blacklist, once optimized to be loaded with a single malloc() call, only took up 9.5 megabytes of memory.
https://samboy.github.io/blog/entries/MaraDNS.html#BlogEntry...
- vinkelhake
These seem like some fairly standard approaches for reducing memory usage. I can't help to think that the approach of joining several distinct list into a single one in some way undercuts Rust's safety guarantees.
If you previous had three distinct Vec objects, then Rust would guarantee that you can't index out of bounds. If you now put all those objects into a single Vec and rely on offsets, then you now open the door to indexing out of range of these sub-slices without any panics.
It's a minor point, and it doesn't really invalidate the optimization, but I'm surprised the article didn't mention it.
- ManBeardPc
The Record struct contains rtype and data where RecordData is a tagged union. Aren’t those two always in sync? Not a DNS expert, just wondering if this is redundant or there is a reason both are there. Doesn’t matter anymore if they store it already serialized but I would be interested why it was this way.
- superze
How much is this in euro or do we measure money in ram now?
- bhouston
I've run into issues with using public wifi when I override my MacBook's DNS server to 1.1.1.1 or 8.8.8.8. I believe this is because captive portals require custom resolution of the name captive.apple.com. And external DNS servers will not resolve that correctly to the local gateway's authorization page.
- 1saadcodes
We're finally seeing more appreciation for this kind of engineering. Not everything needs to be solved by throwing more hardware at the problem