AI researchers debate how close we are to recursive self-improvement
Dwarkesh Patel sits down with John Schulman of Thinking Machines, Beren Millidge of Zyphra, and Charlie O'Neill of Baseten to steelman the case against rapid recursive self-improvement. They explore why AI progress might asymptote before crossing human-level research, whether long-horizon RL can produce AGI, the persistent sim-to-real gap, and what it would take to discover the next paradigm shift.
You could imagine, as AIs get more and more capable, that they're capable of making progress on simulations which incentivize getting better at not only AI R&D, but at science generally.
- bretpiatt
From what I'm experiencing, we have recursive self improvement that can find local maxima, I'm not seeing really strong evidence of unguided RSI that finds the true optimal.
- reilly3000
I have developed a highly personalized, local, offline RSI by scrolling through too many articles about AI. It’s mostly in my right thumb.
- killbot5000
Can someone define “improvement” in this context? This concept feels like a buzz word otherwise.
- anoplus
AI should be optimized to make all humans healthy, happy and safe without exception.
- carabiner
Does anyone know the status of John Carmack's AGI work?
- preommr
What do these discussions even matter when the words don't matter?
How many times has AGI been declared already? A bunch of people (e.g. Jensen Huang) have called Astra AGI, for example.
The goal posts get moved, everybody's hustling, lying, inventing new buzzwords, doing mental gymnastics, and it's all just tiring.
Just show me the results, and let the proof be in the pudding.
- eugene3306
Okay, AI now can write a poem or compose a song.
But can an AI construct Terminal Bench 5.0 or GDPval 2027 ?
Without this, there is just no path towards RSI.
- deadbabe
Recursive self-improvement will ultimately be a problem of money. It's a very big bill and AI companies need to start thinking in terms of how they will accumulate cheap and free energy, because merely paying for compute will no longer be enough. Money is the bottleneck. The future requires companies that are post-money.