Monzo's Stand-in: A Backup Banking Platform That Survives Full Cloud Outages
Monzo Stand-In
Monzo has built a completely separate backup banking infrastructure, Monzo Stand-in, to keep customers spending, transferring, and checking balances even during a full AWS outage. Running on GCP with just 18 microservices versus the primary platform's 3,000, it costs around 1% of the primary's cost. The system syncs data asynchronously, accepts eventual consistency, and uses 'Monzo Advices' to reconcile effects when the primary returns. Crucially, it runs different software to avoid the same failure affecting both platforms, and can be enabled even if the primary is completely down.
Today it doesn’t matter how many data centres you have if you run the same software in them all.
- danpalmer
I can't help but think this isn't the best idea, it raises so many questions. How does fail over work (it failed for me!), can you safely fail over, how does product think about the stack, is it funded well enough, how do you scale a system from zero to all traffic instantly, how do you effectively keep this environment warm, how do you determine which services are critical enough to be in it, how do you manage finding the bugs in two separate stacks, and so on and on and on.
It seems like it would be better to invest in a platform that is distributed across multiple failure domains. Some companies call this a cell architecture. For Monzo this might mean true multi cloud, but they already have bare metal services for bank integrations so they are already hybrid in some respects.
It's not the decision I would have made, and I'm not sure I agree with their reasoning, but I'll also admit that I have not run a bank, I don't know all their constraints, and there are some good tech folks at Monzo. That said, I am an SRE at Google so I'd like to think I'm not completely naïve here.
Edit: after reading in more detail I'm puzzled. The justification for Stand-In and not traditional scale-out is the difficulty of data consistency (agreed!), but then Stand-In is just eventually consistent anyway, and introduces sync problems in both directions. Admittedly I'm spoilt by Spanner, but I do wonder if Monzo is being bitten by past architecture choices here.
- jmkni
Just pointing out this is from February 2025, so has been a thing for a while
- janpeuker
Minor thing but I do like they use the word "Stand In" here in the payments / financial sense of Stand-in processing (STIP), which is usually used when a bank is offline to keep essential services such as public transport working.
- sdcfgy
Due to the recent outage where it didn’t work properly, this should be renamed Monzo Limp-In.
- tarun_anand
I think this is what distributed systems has a phrase for: "graceful degradation".