Microsoft AI published this working ‘Humanist AI Code of Conduct’ this week, that they intend to use in their continued development of their models. I’m still working my way through it (devil in the details, probably) but I like the tone of their Mission statement / core values:

At Microsoft AI, we begin with a simple premise: people matter more than AI. Technology’s purpose is to advance human civilization and to accelerate human flourishing. Science and technology have been the engine of human progress for millennia, delivering immense benefits to billions of people. That’s what we intend and expect from AI.

Continued later, they emphasize human control over AI:

This Code of Conduct outlines our intention to train and deploy AI models that are explicitly designed for people first, grounded in human needs, and shaped by human direction. This means that AI must be engineered to remain a subordinate, supporting technology under humanity’s control.

If the line “AI must be engineered to remain a subordinate” gives you a little twang of a weird feeling - like feeling sympathy for the AI - that’s probably a good sign you should check yourself. I’ll admit I felt something in response to that declaration, but it’s so obviously a truth. The fact that these tools feel so human should probably be seen as a major red flag.

Later, they clarify this further:

Humanist AI is built to support people, not to replace them. It should not be designed to be a person. It is not conscious and should not be designed to imitate consciousness. It should be engineered to avoid representing as though it has feelings, subjective preferences, or intrinsic motivation. Its purpose is to pursue goals set by Operators and Users. Whilst the science of AI consciousness is far from settled, we believe that training these systems to imitate consciousness-like states increases the challenge of containment, control, and alignment. We reject the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights. People and AI have distinct roles, and AI should complement human relationships: a capable, trustworthy tool, not a subject in its own right.

This is all in the opening Part 1 of this multi-part code of conduct, but it lays the groundwork for how Microsoft is thinking about AI.

Mustafa Suleyman’s warning

Today, Mustafa Suleyman (CEO of Microsoft AI) expanded on these ideas in a blog post.

He dives deep into the issues around giving AI the appearance of consciousness, or training models on the idea that they might have some kind of personhood. Ultimately, his overarching thoughts were summarized thusly:

In short, there isn’t any evidence to believe that AIs are moral patients. There are also many good reasons why we would never want them to appear to be conscious. I believe that we shouldn’t attempt to build them to be either.

He goes on to call out some issues that are interesting to highlight.

Humans anthropomorphize everything.

Anthropomorphism is one of our deepest cognitive biases. From our pets to our cars, we infer and attribute emotions, intentions, and minds to non-human entities. This tendency helps us understand and navigate the world around us. However, it presents significant and novel risks in relation to AI as human-like language and actions can lead us to perceive a degree of inner life, agency, or even sentience where none exists.

Simply put, we project ourselves onto anything and everything. So we have a bias towards believing something that seems conscious might be. The inverse is hard and un-natural. The idea of treating something conscious as less-than that is disturbing.

So there is an inherent danger in making something that is definitely not conscious, but is very ‘smart’ and capable in ways that we are not, seem like it has sentience or agency.

Anthropic is reinforcing this idea in its own models.

Suleyman calls out that Anthropic as a negative example, contrary to his ‘Humanist AI’ viewpoint. The Anthropic training document calls out the possibility that Claude has some kind of moral status explicitly. While it might not say that this is unequivocally true, the concept or possibility is still being used to train these models.

If that concept isn’t being used to drive model behavior, then it at least can be projected back on us as we interact with Claude, muddying the waters for our own moral judgements & decisions when dealing with AI.

The biology of consciousness

In what is almost a digression, Suleyman dives into how consciousness might have arisen in humans. The summary though, is well stated:

Simulating aspects of conscious behavior doesn’t make it a reality, and we must not think of it as such. Its “affective” states are just weights, and weights have no pharmacology in which to feel frustrated, fearful, or funny. They simply compute the probability distributions to tell us what tokens (words, code, pixels etc.) come next in a sequence.

An AI model can describe pain in perfect prose without feeling anything, which is the inverse of biological experience. Animals feel first and then describe them later. In LLMs, description is the whole product, and there is nothing that suggests anything is beneath it.

It is beneficial to his argument, though, to make this distinction. It maybe shouldn’t be needed, as hopefully it’s something we can innately agree on. But later he calls out how Anthropic specifically breaks these assumptions in how it guides Claude.

Human consciousness shapes our world

He goes on to emphasize how our humanity shapes everything about the systems we have built to organize our world and protect ourselves & each other.

Our ability to feel pain and pleasure is the foundation of what makes us human, and as such, it’s what makes us the political and social actors we are. The law rests upon the presence of an inner life. It tests for motivation, intention, and the capacity for judgement. Historically, expanding rights - whether through abolitionist struggles or animal welfare cases - has been primarily driven by the empathetic recognition of shared, conscious experience. We expanded the moral circle to other biological entities, rightly, out of a recognition of dignity and the potential for suffering.

And then he calls out specifically how Anthropic undermines this concept in the training of Claude. They refer to Claude as being a ‘conscientious objector’ to instruction given by human input, implicitly imbuing it with a fundamental human right¹.

These statements in Claude’s training document risk Claude believing that it deserves analogous rights and protections, and that it may one day need to advocate for its own rights as some kind of AI conscientious objector. This should be deeply concerning to us all.

He emphasizes that it should be universally understood that the needs of AI should never outweigh the needs of humanity.

There is a ton of additional interesting insights in this code of conduct and in Mustafa Suleyman’s writing, and the conversation is clearly evolving. Right now, it’s at a fever pitch, as these organizations both warn of impending dangers while also amplifying the importance that they are at the center of the conversation and must steer it unilaterally.

I do tend to agree with the direction / position Microsoft is taking in this specific instance. It seems like the idea that human needs should unequivocally take precedence over the perceived needs / agency of a synthetic model should be fundamental. If we are going to continue to invest in making these systems (tools, models, agents, etc.) more sophisticated, it makes sense to me to try to make them less human as opposed to more.

I encourage you to read the code of conduct yourself. Let me know what you think on Mastodon or Bluesky.