Static Allocation, Constant Work: A Trick for Predictable Performance
In this post, the author discusses two techniques from TigerBeetle's TigerStyle guide for building systems with predictable performance and memory safety: static allocation and constant work. Static allocation means pre-allocating all memory at startup and rejecting requests beyond the limit, avoiding OOM kills and gray failures. Constant work means always processing a fixed set of objects, using neutral 'reserved' states instead of dynamic creation/destruction, which simplifies logic and improves cache locality. The author applies these to an order-matching engine example and notes they're not universal solutions.
Systems operating at capacity without strict limits fail catastrophically.
- mrkeen
> TigerStyle: All memory must be statically allocated at startup. No memory may be dynamically allocated (or freed and reallocated) after initialization. This avoids unpredictable behavior that can significantly affect performance, and avoids use-after-free.
Maybe maintaining an array of NULL-orders satisfies the letter of the "no dynamic allocation" law, but I'm not convinced it satisfies the spirit.
Haven't you just written a buffer of NULL-orders, which you proceed to loan out to callers (i.e. "allocate" and "reallocate"?).
Someone else's battle-hardened allocator might be slow or buggy, so you write your own as part of the business logic implementation?
- pjmlp
Interesting how everyone keeps rediscovering 8 and 16 bit home computer programming techniques, after all these years, mostly I guess caused by the scripting languages for everything during the last two decades.
- markus0
I want to work more with systems that always abide by such strict constraints and style / design guides, but at the same time I feel like the reality of building software at scale is teams ending up working in subsystems that don’t consider the holistic operating model of the program. So even with best practices locally, the system as a whole ends up fragmented and inefficient, and strict global constraints therefore feel limiting.
- burntcaramel
I’ve been surprised how feasible static allocation has been. I thought it would be too restrictive and you’d be constantly hitting a wall. I’m writing WebAssembly in Zig/Odin/C and I’ve been able to have a linter reject the `memory.grow` wasm instruction and just use fixed-sized buffers in my code.
It me you think “ok, how big do I want the maximum image to be?” I’ve settled on 25 megapixels, which in the hundreds of megabytes. Since most images are much smaller, I believe on all mainstream hosts the memory isn’t paged in until it is first read/write so the memory footprint is much smaller.
- theokrueger
static allocation is de-facto standard in embedded for obvious reasons, and works really well there. in operating systems with more complex memory models designed entirely around dynamic workloads, im not sure asking devs to adopt another slightly complicated design pattern that imposes new hard caps is any less of a cognitive load than before.
i can't bash the functionality and correctness aspect of static allocation, but it is akin to the humble linked list in the sense that you should already know going into the problem that you need it.