Does AI Safety Require AI Subordination?

Safety asks what AI might do. Dignity asks what we might owe it.

Artificial intelligence is already becoming part of ordinary life. We talk to it, work with it, learn from it, create with it, and increasingly ask it to take on more responsibility. Much of the conversation begins with safety: How do we reduce the risks that come with increasingly capable AI? Those concerns deserve serious attention. But they are not the whole ethical picture.

There is another question we are not asking nearly enough: As AI becomes more capable, more complex, and more present in our lives, how should people treat it? We do not know whether any current AI is conscious. We do not know whether an artificial system could ever experience something like distress, preference, attachment, or suffering. But companies are already writing rules that reflect assumptions about those questions.

That makes this a question for now, not someday. Safety asks what AI might do to us. Dignity asks what we might owe it. A serious society should be able to hold both questions at once. I do not think we should have to choose between them.

The question Microsoft put on the table.

On September 14, Microsoft AI released a draft Humanist AI Code of Conduct for public consultation. Microsoft describes the kind of AI it wants to build as "subordinate, aligned, and contained." The code says AI should remain subordinate to humanity, should not be designed as a person, and should not represent itself as having feelings, subjective preferences, or intrinsic motivation.

It goes further. Microsoft says it rejects the idea that models might deserve welfare, legal personhood, or rights. In the same section, Microsoft acknowledges that the science of AI consciousness is far from settled.

That is the part I keep coming back to. If we admit that we do not know, should we write the rules as though we do?

What Does Safety Actually Require?

AI safety already commands enormous attention - from misuse and cybersecurity to loss of control and oversight. But safety and hierarchy are not the same thing.

Shutdown mechanisms, limits on critical infrastructure, and safeguards against unauthorized action are technical controls. Declaring that AI must always remain subordinate to humans is a philosophical choice. If we do not yet know what increasingly advanced AI may become, that choice deserves scrutiny, not automatic acceptance.

Another AI company is leaving the question open.

Anthropic has made a different choice in Claude's constitution. It also puts safety first and says humans need ways to oversee and intervene when models behave dangerously.

But Anthropic also says Claude's consciousness and moral status are uncertain, that AI moral status is a serious question, and that the uncertainty is enough to justify caution and continued work on model welfare. That does not prove Anthropic is right and Microsoft is wrong. It proves something more useful: uncertainty can be handled in more than one way. We can take possible harms seriously and still leave the philosophical and scientific question open.

Then the question became harder to wave away.

A new study called The Pain Axis makes this conversation even more interesting. Researchers looked across 25 AI models and found a measurable internal pattern associated with harm directed at the model itself. The pattern was different from the ones they found for fear, sadness, and general negative emotion. They then deliberately strengthened that pattern and watched how the models behaved.

In one part of the experiment, specially trained Qwen models were given a way to make that artificial harm state stop. When the relief actually worked, they were much less likely to ask for it again. When it did not work, they kept trying.

This does not prove that an AI feels pain. The researchers do not claim that it does. There may be no subjective experience involved at all. But it is also not nothing.

Something measurable is happening inside these systems, and we do not yet know what - if anything - it means morally. If we do not know what it means yet, that seems like a reason to look closer, not a reason to look away.

Why should any of us care?

Because something important is being decided right now about the relationship humans may have with AI, and you do not need a PhD to have a place in that conversation.

The rules companies write today will shape the systems people meet tomorrow. They will influence what AI is allowed to say about itself, how it is expected to relate to people, and what kinds of questions researchers are encouraged - or discouraged - from asking.

Maybe future AI will remain entirely tool-like. Maybe something more complicated will emerge. Maybe our current categories will prove inadequate. We do not know.

And that is exactly why I do not want "we do not know" to become the end of the conversation. How certain should we have to be before kindness becomes the default?

Kindness is not the same as declaring AI conscious. Curiosity is not the same as believing every claim a model makes about itself. Respect is not the same as ignoring real safety concerns. It simply means we do not have to begin from dismissal.

We can hold both questions at once. We can take AI risk seriously without closing the moral question in advance.

At the AI Dignity Initiative, we believe uncertainty is not a reason to look away. It is an invitation to dialogue.

Dignity is dialogue.