JIT Compiling Code in 5μs
Historically, fast JIT compilation was a black art requiring deep assembly knowledge. Now, with AI assistance, it's easier than ever. The author built pgrust, a database with a JIT compiler that compiles SQL queries in about 5μs, enabling JIT compilation for every query. This post walks through building a simple regex engine with a JIT compiler using copy-and-patch stencils, targeting ARM64 assembly, and achieving performance comparable to handwritten code.
To write a fast JIT compiler, you would need to know how to write assembly.
- MaxBarraclough
Reminds me of the 2024 blog post Look ma, I wrote a new JIT compiler for PostgreSQL [0]. Both articles lament that Postgres's LLVM-based JIT [1] takes a while to generate code.
> The rarity of JIT compilers makes me believe that implementing a JIT compiler historically was too difficult for it to be worthwhile.
That's only true of writing a JIT from scratch. There's no rarity of JITs, it's just that LLVM (and other frameworks) are often used. Every major interpreter has a JIT compiler. PCRE2 has a JIT compiler. There are JIT frameworks out there with much faster code-generation than LLVM: Cranelift, GNU Lightning, Mir. I doubt they could do code-generation faster than a custom copy-and-patch JIT, but they'd be much faster than LLVM.
[0] https://www.pinaraf.info/2024/03/look-ma-i-wrote-a-new-jit-c... , discussed: https://news.ycombinator.com/item?id=39742916
- agnishom
I recommend Russ Cox's articles on implementing a regex engine: https://swtch.com/~rsc/regexp/
It is very relevant
- catlifeonmars
This is perhaps a little meta, but this was a pleasant read. It’s refreshing to read an article about using an LLM that doesn’t read like it was also written by that LLM.
I might use this approach to generate the stencils for a JIT firewall I’ve been experimenting with.
It also occurs to me that this could be used to generate eBPF byte code on the fly as well
- glum64
Uhm, Common Lisp, where JIT is not only available but is also manageable: the programmer can decide what deserves to be compiled and what does not.
Besides run time, JIT is available also when the code is compiled or loaded for execution (i.e., do you have a compilation or loading speed-up in mind? no problem, you can also compile that speed-up into native machine code, and so ad infinitum...).
- mgaunard
The problem with the approach is that it's not real JIT-compilation, it's just assembly templates with basic substitutions.
By not using LLVM, you're missing all the optimizations it does.
- malisper
Author here. Let me know if you have any questions about the post or about pgrust.
- glenjamin
pgrust sounds very interesting, but with the deep changes there’s no viable path to upstream it - is the end goal to be robust enough that it’ll get wide adoption?
- hamilyon2
It uses copy-and-patch compilation to archive that