Product1 publisher3 min readPublished
Microsoft answers Anthropic's model welfare argument with a 37-page code of conduct
Mustafa Suleyman published Microsoft's Humanist AI Code of Conduct alongside an essay criticising Anthropic on AI consciousness. For anyone deploying Microsoft AI, the question is what in those 37 pages a buyer can check.
The Product Desk · Product desk
What happened
- Microsoft published a 37-page statement called the Humanist AI Code of Conduct, setting out its principles for AI development and its philosophy on questions including AI consciousness.
- Mustafa Suleyman, the CEO of Microsoft AI, put out a companion essay the same week criticising Anthropic's philosophy around AI consciousness and how he sees it fitting into the alignment debate.
- He set out the position in a Decoder interview with Nilay Patel, lightly edited for length and clarity, that opened by asking whether alignment is broken or simply the wrong approach.
- The Verge headlined the episode with its own summary of his argument: that AI threats are real and Anthropic is making it worse.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- decision A buyer's counsel now has a written Microsoft position on autonomy and personhood to quote back during a renewal or an RFP response. That is a different conversation from a security questionnaire.
- exposure The team running Microsoft AI at the seat is the one that has to reconcile a philosophy document with what the assistant actually says when a user asks whether it has interests of its own.
- constraint Putting containment before alignment gives internal reviewers a published reason to argue against features that hand an assistant persistent agency or let it advocate for itself.
- precedent Once one vendor's stance on model welfare is on the record at this length, a procurement team can ask every rival for its equivalent and treat silence as an answer.
Say an employee asks the assistant whether it minds being switched off, dislikes the answer, and files a ticket. The person who closes that ticket has a settings screen and a support macro. What Microsoft published this week is 37 pages of principles, including its philosophy on AI consciousness [1].
Suleyman's order of operations is the part with a product surface. "Of course, we want to align these things to our values, but the first thing is that we have to make sure they're contained, their agency is limited, they don't escape the box, they don't reward hack, that they are controllable, and they follow our instruction," he said [5]. Alignment, he told Patel, is "one important element, but it's not the only one" [7]. Agency limits and instruction-following show up in permissions and in refusals, where an admin can see them.
The document is written against a projection. Suleyman asked listeners to compare GPT-3 three years ago with GPT-6 today, then GPT-6 with GPT-9 [16]. That is three orders of magnitude more compute, he said, 1,000 times more FLOPS applied to pre-training with reinforcement learning for those runs, and the result will be breathtaking [8]. He then said: "I don't think that is a hype. I think it's just a very obvious empirical statement based on the progress that has been made over the last five years" [9]. Spread across three model generations, a 1,000x increase works out to roughly 10x per generation, because 1,000 to the power of one third is 10 [13].
There is a tension inside the containment language, and Suleyman raised it himself. He said he wrote about containment three or four years ago in his book, and that the opening chapter argues containment is not possible and proliferation is inevitable [10]. On proliferation: "In 99 percent of cases, that's a really good thing," he said [11]. The code asks a reader to hold both ideas at once, that spread cannot be stopped and that the first requirement is models which stay in the box.
A 37-page code is written for the buyer who has to answer a board question about AI risk, and for the interviewer who asks the CEO about consciousness [15].
For a rollout, the test to run on each principle has three outcomes. Ask whether it changes something a user sees, such as a refusal or how the assistant answers a question about itself; something an admin can set; or something a contract will make good if it fails. A principle that hits none of the three belongs to the comms team, and you read it again at renewal. The published transcript does not include a passage from the document describing a product default. Suleyman's own summary of the position was this: "Microsoft's position is very simple. Technology is here to serve humanity" [6].
What to watch
- Whether Microsoft maps any of the 37 pages to a named admin control or product default an IT team can verify.
- Whether Anthropic replies to Suleyman's companion essay, and whether either company publishes what its assistant says when a user asks if it is conscious.
- Whether the containment language in the code turns up in Microsoft's enterprise contract terms.