Anthropic Researcher Quits, Warning AI Could 'Kill All Humans' — Silicon Valley Shrugs

Maybe they can use their brilliant super AI to address some real and present threats that are being ignored like nuclear annihilation, the loss of the prohibition of the acquisition of territory by force established after WWII, the certainty of all of the fossilized carbon being dug up out of the ground and released into the environment, the resurgence of measles and a poor posture for the next pandemic, ETC...

I agree with the host of the All In podcast, often I don’t. This is about creating a moat that open source models cannot compete with, regulation.

It is purely ensuring only a few providers can compete.

The future is edge and open source models.

Both Apple and Google are working towards this.

Cloud AI will be for offloading when the edge is not capable.

Most cases edge will be suffice.

Further inference costs keep coming down.

Vertical integration might be the biggest challenge of the AI industry. Trying to manage electricity generation, training, inference, harness and end user for enterprise and consumer is a lot of moving parts.

To everyone who believes that this is some nefarious plot by CEOs to bring about regulation or to hide that they've hit a wall or to make their models seem more capable or whatever:

The AI risk arguments are philosophical arguments that are 20-30 years old and don't have anything to do with the current corporate landscape. They were thought of by people who really care about the issue and all the "gotcha" lines have well-developed counters. Of course they might be stuck in their ideology but they deserve a bit more credit.

I don't doubt that AI CEOs can profit from a crisis and that they don't necessarily have our best interests at heart. But this is the same way governments during COVID tested how much they can restrain freedom. This doesn't mean COVID doesn't exist or that vaccines are evil or ineffective or whatever. Same way here, if there is an advantage to CEOs to regulate things doesn't mean AIs can't kill us all and that regulation might bring us more time to solve alignment.

Also: I accept I might be wrong and AI risk might be a fake issue and/or alignment might be impossible, but please bring better arguments if you want to convince me.

So, as someone who works with LLM's daily and even trained his own, who's enjoying the benefits of this technology daily, I'm supposed to:

- Listen to someone who just collected a few ten/hundred million building AI at a frontier lab, then decided to call it "existential crisis for humanity" and suddenly is going to work at a regulatory body made exactly for this

- Listen to CEO's of billion dollar startups nearing-IPO and are calling for "a pause in development" and regulatory capture, while they have the frontier covered and keep extracting value there like nothing happened

- Listen to "the people" apparently, as NGO's like irreplaceable.org are trying to say, pretending they are normal everyday people while the organizations are being ran by political lobbyists and people who do political non-profits for a living

- Listen to "the doomers", which mostly consists of people that have never written any code or aren't even tech-adjacent, talking about sci-fi scenarios like they are our current reality, i.e. "AI replicating itself across servers and building weaponized drone/biolab/robot factories while nobody notices" like it's a serious threat we have, without any thought to the insane sequence of events and failures needed to get there, and the actual possibility of being noticed at each step of the way.

- Listen to the folks promoting their books, substacks and podcasts talking about AI doom, where the folks involved are mostly again, not engineers and have no clue on how an […]