GPT-6 Astra Can Now Drive a Real Car
GPT-6 Astra has gained the ability to drive a car
DrivingBench puts frontier AI models behind the wheel of a Toyota Corolla, giving them control of steering, accelerator, and brakes on a fixed cone course. Each model gets up to three attempts in one continuous chat, with metrics like progress, distance, finish time, commands, and token cost. The leaderboard reveals how close today's top models are to actually driving a real car.
Can frontier models drive a real car? We give them control of a Toyota Corolla's steering, accelerator, and brakes, then evaluate them on a fixed cone course.
- jyoung8607
I'm not an expert in the LLM space, but I'm an external contributor to comma.ai's openpilot project and I'm and quite familiar with how its controls work, so I looked from that perspective. There's two questions here:
1) Could a cloud-delivered LLM figure out how to drive this route, based on those input data and given access to those output actuators? Looks like yes. Sure.
2) Could this work in the real world? Absolutely not. Three reasons: latency, latency, and latency.
openpilot's driving model updates the target curvature and acceleration at 20Hz. Every millisecond of the round trip time through every piece of its entirely-local driving stack is well-understood, extremely consistent, and tightly optimized. It has to be, otherwise you can't react to even minor bumps or wind gusts, much less rapidly-developing traffic situations.
Adding even a single speed of light RTT to a cloud service is meaningfully bad, and you'll need a whole lot more to encode and upload camera imagery to even start the time-to-LLM-response clock, and then send the response back down. By then the world around the car has moved on.
There's a reason Tesla and every other self-driving manufacturer need the compute hardware in the car.
- famouswaffles
What did they do to Astra so cracked at vision (and computer use). That ARC 3 score turned out to be no joke/fluke. That huge gap between Astra and Fable (in this case) is basically every hard vison/spatial benchmark i've seen including non-benchmarks like playing games (Portal, Factorio, RimWorld).
SpatialBench - https://x.com/spicey_lemonade/status/2096365630190698516
ZeroBench - https://zerobench.github.io/
Robot Arms - https://openai.robocurve.org/gpt-6-astra/
- valine
The bitter lesson is finally coming for the self-driving cars. The vision stack, 3D maps, lane selection grammar, occupancy networks, it’s maybe all about to give way to a single GPT looking at camera feeds and predicting the next steering wheel adjustment.
It’s mostly a latency problem at this point. The models are too big to run locally, but given that open-weight models like Qwen already exist, an open-weight, low latency equivalent to Astra can’t be too far out.
- skyberrys
I wonder if this could solve a driving problem I have. I want an automated system to slowly drive the cars from the entrance of my neighborhood to their designated parking spaces. Right now the humans do this and they go too fast, and ignore the stop signs. I think it would be safer if all cars are automatically parked instead. It seems doable, it's a very controlled environment and I want to cars to go slow. The humans can get out and walk home if they need to be there faster.
- prometheus1992
Wow! but WHY is this a benchmark?? for comparison tesla's model is approximately 10-15B parameter model (estimating from maxxing the hardware that comes with the car at 16gb ram).
- 1970-01-01
Looking forward to the juggling bananas benchmark. If Claude can only manage 5 and Astra does 6, clearly they have a better model.
- syntaxing
Surprised they didn’t try Qwen’s recently open sourced driving model https://huggingface.co/Qwen/Qwen-Drive-1.0-4B
- RomanKornev
Looking forward to the inevitable "Astra can land a plane now, with no autopilot"