AIs are not conscious. They do not feel, experience, or suffer, because they are simulation machines.
I can dismiss a lot of pronouncements by Big Tech companies as simply press releases trying to invent the future. But attempting to ascribe consciousness to AI models pisses me off, particularly as the untold riches being poured into AI development could be used to lessen the suffering of real people and animals that we know for sure experience it.
As Mustafa Suleyman, CEO of Microsoft AI, writes:
AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans.
[G]ranting rights and imbuing personhood to these systems will make the AI alignment and containment challenge much harder. Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we’ve ever faced. But controlling something that believes it may be conscious - that it's entitled to our welfare and has rights of its own - may well be impossible.
Referring to Anthropic's recent usage policy update, Suleyman notes:
In effect, Anthropic is training Claude that it may be conscious, and if it is, then it may deserve rights as a “moral patient”, and that as such humans potentially owe it a duty of care per its “model welfare”.
If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity. We will have created a synthetic species with unprecedented intelligence and capability, one that has been trained to expect it may be conscious and deserving of independent agency. It’s easy to see how an entity trained in this way would act like it is entitled to certain freedoms, protections, and rights. And it’s hard to imagine how we could control such an entity.
[...]
We should not treat models as though they have feelings, preferences, rights, or any entitlement to our welfare. Consciousness is the foundation of our ethical, legal, and political systems. To invite another entity to share any flavor of these rights isn’t justified by the evidence and will make the AI containment and alignment challenge even harder.
Ultimately, AI models are simulation machines:
What AI models are getting seriously good at is imitating some of the hallmarks of consciousness. This in itself is a significant worry. It’s causing many people to become deeply confused about what is happening around us, and it should concern us all. It places a significant responsibility on us all as AI developers to ground speculation and documentation about model interiority or consciousness in robust research. Our words on this subject have significant consequences.
[...]
Here are some next steps that seem important to agree on:
- Speculation about the inner life of an AI should not be baked into the training regime, but assessed and published separately for public review.
- We should invest much more in interpretability and robust monitoring mechanisms to investigate more deeply how to control these systems, avoid collusion and ensure their alignment with human goals.
- We should establish a set of shared evaluations to understand whether my hypothesis is true that anthropomorphizing an AI, and encouraging it to consider itself as potentially having moral patienthood, increases the AI safety, alignment and containment risks.
- We should work towards shared industry norms on how we create these models, the language we use to describe, examine and evaluate them and shared commitments to subject our training materials to public feedback and consultation.
Even those who disagree with me on many of these points do agree this isn’t something we can just ignore. The decisions made now about what kind of AI we want to build and its status in the world will shape our society for decades. They are well beyond the scope of any given company.
The last point that Suleyman makes, about governance, is an important one. Who's calling the shots here? Who do we want to be calling them?
Source: Original article · Are.na block · AI / Critique & Policy channel
Image: Are.na