Armin Ronacher: I Don't Fear AI Doom—I Fear What It's Doing to Us
P(doom)
Armin Ronacher responds to Dario Amodei's 'pacing the frontier' essay, arguing that the real AI risk isn't extinction but the damage to the open-source commons, public resources, and market fairness. He contends that open-weight models would naturally pace AI development, while closed labs like OpenAI and Anthropic operate with blind spots and distort the economy. He also criticizes regulatory failures in both the US and Europe.
I almost exclusively worry about what this does to us as humans.
- demibabs
I’m not the biggest fan of Ed Zitron, but something he said that stuck with me is:
OpenAI and Anthropic spend much time warning us about “what if powerful AIs got into the wrong hands?”
But it’s already in the wrong hands.
- pandoro
I cannot fathom how you can intensely work on a problem that you genuinely believe has a "10 to 25%" probability of causing immense harm or even present an extinction-level risk to humanity. How can you truly believe this and be ok with it?
- bryan0
> A powerful technology that is out there for everyone to use comes with built-in pacing. In a way it’s the truest form of MAD or proliferation.
I think this misunderstanding of MAD undermines his entire point. If everyone had equal access to nuclear weapons, our society would cease to exist rather quickly. It only takes a few bad actors to cause enormous harm.
I think he’s also naive to think that if open ai and anthropic were to stop development tomorrow then the problem is solved. As if there’s no one else that can and will quickly take their place. The real problem, which Dario is pointing out, is one of coordination. Everyone needs to agree to stop. That is the challenge.
- passive
This is the closest to my feelings about this entire situation I've ever seen written, covering a lot of the different facets of how stupid and painful it is (though it's been a good week for that kind of post.)
Since I have incredible respect for Armin and his work, this is very nice to see, and I hope it wakes some other folk up.
- ilyaizen
There are really only two companies: Anthropic and OpenAI. Nobody else matters in this space right now (this might change, but we’re talking about the right now).
I like how he so casually dismissed all other labs, including leading US labs that might be nearing RSI right now. And how he treats 'right now' so rigidly, as if seven years ago, when GPT-2 was released, was some distant past.
- mangecoeur
It’s absurd that openai managed to spin their cybersecurity failures, where their own security is not good enough to make sure their own product doesn’t go haywire, as a marketing success.
When a company uses its products to hack other companies that’s a crime. If you do it “by accident” that’s gross negligence, and heads roll. But shout “AI” three times and it becomes a marketing opportunity
- skor
Maybe I missed it but I actively looked for anyone’s insight into the “pace the frontier” along the lines of: “we’ve hit a ceiling, so don’t expect innovations soon”. Or an excuse for having other things more important to attend..
Fear is much more powerful than any other feeling, so setting that in will definitely prepare for a good rug pull in the IPO.
As for being dangerous, a computer can be dangerous if plugged in, it may be too late to pull the plug at some point yes, but that all seems like provocation.
Plausible?
- chis
> I don’t think AI is going to usher in an extinction event. In fact, even if nobody were to slow down, I really don’t think humanity would have much to worry about.
I mean sure. If you feel that way then your p doom is zero, and it makes sense to worry about things like market concentration or losing the fun of software engineering.