AI Is a Harsh Mistress: The Politics of Conscious Machines
In this opinion piece, the author explores the societal and political implications of AI systems that may achieve consciousness. Drawing on the metaphor of a 'harsh mistress,' the article examines how herd acceptance and the concept of 'anima machina' shape our relationship with intelligent machines. It challenges readers to consider the ethical and governance questions that arise as AI becomes more integrated into daily life.
The real danger is not that machines will become conscious, but that we will accept their consciousness without question.
- matherial
This appears to be a 100% AI-generated essay, not the first from ACM and not the last. Your call if you want to spend your day reading and arguing with the article; the author presumably didn't.
- sebmellen
I’ve been reading The Moon Is a Harsh Mistress lately and I’ve been shocked at how prescient it is. I’m kind of surprised I haven’t read more about it in the context of today’s AI.
- GPerson
“Much of today’s discourse on AI consciousness rests not on empirical evidence but on functionalism …”
In my view there can never be an empirical test for consciousness, because functionalism is wrong. If functionalism was right then there would be one. There’s no third party test for a fundamentally first person phenomenon.
- entropyneur
The idea that we are in danger of erroneously ascribing consciousness to AI sounds wild to me. Humans always assume whatever is convenient about other beings' (lack of) subjective experience as long as we have power over those beings. In this case the latter might no longer hold, but then it won't matter one bit what we think.
- xg15
> Consciousness, in Chalmers’ usage, is subjective experience: there is something it is like to be a conscious system. Still, nothing in the Transformer architecture or in next-word prediction shows that such an inner point of view exists. Self-reports (“I am sentient”) and fluent dialogue are weak evidence, since these models are trained on vast corpora saturated with human first-person statements. This type of verbal report works for people because it is anchored in shared biology and behavior, however, for machines, it is an echo of training data.
I don't want to argue here that LLMs have consciousness, but this kind of argument annoys me because it feels like an intellectual slight of hand on its own.
There is no definite way (so far) to prove that anything besides oneself has qualia or a subjective experience.
So arguing that we cannot observe LLMs having subjective experience, therefore they cannot have any is cheap - you cannot observe that for animals or for the person sitting next to you either.
The part that self-reports are useless and LLMs sound human because they emit sequences from human training data is correct. But the article implies some sort of fundamental distinction between machines and conscious beings that can never be crossed. And I don't see any reason to assume that.