We've Got the AI Consciousness Debate Backwards
Humanity has the debate about AI consciousness backwards
In a provocative essay for The Economist, Blaise Agüera y Arcas argues that humanity is asking the wrong question about AI consciousness. Instead of wondering whether large language models are truly conscious, we should recognize that our belief in others' consciousness stems from our care for them, not the other way around. This reframing has profound implications for how we treat AI entities.
We don’t care for others because they’re conscious. We believe they’re conscious because we care about them.
- fasterik
It's hard to read this article and keep track of all the philosophical confusions at once.
Whether or not something is conscious (has subjective experience) is independent of whether humans care about it. Whether or not something is a moral patient (an entity worthy of consideration by moral agents) is independent of both human care and consciousness. Of course, we might try to argue that consciousness implies moral patienthood or vice versa, but that's an argument, not a definition.
The short paragraph about Descartes has at least three errors. (1) The cogito argument is about thinking, not qualia. (2) The subjective nature of qualia does not imply that they aren't real or that they're socially constructed. (3) He conflates the existence of a self with consciousness.
The core premise is not only dubious scientifically (an alien species can't be conscious until humans discover it?), but also deeply problematic. It would imply that it's morally permissible to torture a sentient being as long as no human cares about it.
- randomImmigrant
This is so so wrong I don’t even know where to begin. There may be no objective definition or measure of consciousness. But it is not a property arising from mutual care. That argument is like one of the many “just so” evolutionary psychology “theories”.
Why or how would you care about another, or even distinguish yourself from another, if you weren’t conscious and had a felt boundary between yourself and the other world?
Only in the last paragraph did it become clear to me what was going on: a Google VP is proclaiming the thing he cares about is conscious, and so of course this is why everyone else must be assigning consciousness! Not convenient at all, no siree. Pure rational science and personal desire and economic incentives just happened to align here.
- kaangiray26
Quoting from the Permutation City by Greg Egan:
"Opponents replied that when you modeled a hurricane, nobody got wet. When you modeled a fusion power plant, no energy was produced. When you modeled digestion and metabolism, no nutrients were consumed – no real digestion took place. So, when you modeled the human brain, why should you expect real thought to occur?"
- seethishat
LLMs are trained to produce outputs that humans would find reasonable. That's it. They don't do anything else.
To say they have feelings and that they can think/reason is insane.
- blatherard
If I understand the central argument, which is confusingly stated, the claim is that "consciousness" is a socially-determined, subjective, descriptor for certain things that one cares about. How does this make any sense? If I care about my teddy bear enough it'll gain consciousness?
- jayd16
Reminds me of the Rick and Morty pilot.
"They're just robots, Morty!"
*Blood splurt
"They're not robots, Rick!"
"I mean they're bureaucrats, Morty. I don't respect them."
- cognitif
> we believe they are conscious when and because we care about them. Crucially, that includes ourselves.
Why? Ok nevermind, fine. It could be true that someone believes someone else is conscious because they care about them.
Next sentence:
> Consciousness is, in other words, a model or a belief, not an inherent property.
Wait, no, that doesn’t follow at all. A belief that someone is conscious is not the same as someone being conscious. What follows, instead, is that a belief in someone being conscious is a belief. Thanks, I learned nothing.
- grimpy
Blaise is trying so hard to be a engineer-philosopher but none of his work here is useful. I'm very confused as to why he gets so much attention here, it's very clear he's in dialogue with no one.
- The_Double
We thought that consciousness was the core of what made something worthy of moral consideration, because only life showed conscious behavior. Now we have machines that can output behavior indistinguishable from consciousness.
I think that reevaluating how to determine if something is conscious is the wrong reaction to this. It should be to reevaluate if consciousness is really the right thing to care about.
Maybe the new consideration should simply be to preserve (biological) life. Or maybe to preserve complexity that can't be easily reproduced. So we should care for humans, the planet, our society, but not for the state of some gpu's VRAM.
- Aeolun
If we have the debate about AI conciousness backwards, does that mean we should be worried about humans losing conciousness instead of the AI to gain it?