P99 0 ms* autocomplete for 240 million domain names
P99 0 ms* autocomplete for 240M domain names
Wirewiki's creator describes how he achieved p99 0 ms autocomplete for 240 million domain names. The trick: prefetch suggestions on keyDown and render on keyUp, giving a ~121 ms budget. The API uses an in-memory trie for popular domains and a memory-mapped block index for the long tail, with most requests answered in under 2 ms. End-to-end latency is network-bound, and while a single European server works for most users, global coverage would require geo-distributed servers.
Even at 1.6k req/s, Nginx + the API responds in 15 ms 99% of the time.
- skybrian
This autocomplete suggests domains that don't exist. You can just type garbage and it will suggest something, but then if you go there, there are no records.
It seems like one purpose of an autocomplete box is help you avoid typos, so that makes it less useful.
- chrismorgan
Using keyup makes no sense and is inconsistent with user expectations. For triggering actions (which includes normal typing), you only ever use keydown. (Well, there’s one exception for reasons unclear to me: activating a button by pressing Space. That triggers on keyup like how clicks are on release, while Enter triggers on keydown.) Keyup is limited to things where you’re constantly reacting to the state of a key, as is common in games.
This affects the functionality, too. It is in fact introducing latency by using keyup instead of keydown. Feels bad.
- ViscountPenguin
Unfortunately this approach doesn't feel that great down here in Australia, definitely a function of latency.
I think you could get a lot closer by framing this as an optimization problem, where you use the full alphabet dictionary, but add a residual prediction which aims to cover as much of the remaining domain name tree as possible weighted by popularity. This tree could then be pre-baked and stored with the same system. This would probably get you p99 0ms even in Australia.
- oersted
Why not just trigger the fetch on keyDown and show it as soon as the response arrives, as usual?
The time it takes to press a key is a reasonable target to aim at for API latency I suppose, but it is still an arbitrary target. Waiting to display until keyUp just adds more latency if your API is faster. Having it synced with keyUp doesn't make it feel more immediate to me.
- kevmo314
If you’d like to reduce the network latency further you can store each trie node as a file, naming it conveniently the prefix path to that node. Then dump the few hundred million files onto R2.
Now the traversal can be done completely via CDN lookups!