Satya Nadella wants AI agents designed as if the model were already compromised

In our view, Nadella's asks add up to one pattern, the one security engineers already apply to untrusted code. The model proposes an action. A separate orchestrator decides whether to run it, runs it, and records the result. Splitting the model from the system that coordinates its work ^[3] and moving controls outside the model ^[4] put the trust in that orchestrator.

That changes where today's guardrails can live. A system prompt telling an agent not to touch production data is a control inside the model. Once the design starts from a compromised model ^[2], that instruction is part of what was compromised. Audit logs follow the same rule. Letting the agent write its own audit trail meets the letter of recording every significant action ^[5] and none of the intent. The record has to come from the harness, at the tool-call boundary, written to storage the agent's credentials cannot modify.

Nadella's "emergency brake" ^[7] is the costly control. An authorised person can stop a task mid-run ^[1] only at a point where the orchestrator holds control. In the usual loop the model emits a tool call and the harness executes it, so that point is the step boundary. Once a request has gone to an external API, it is gone. The pause stops the next one. If an agent holds its own API keys, starts background jobs, or reaches tools by a path the harness does not mediate, the brake has nothing to act on.

Honouring the brake therefore takes a single choke point for side effects. It also takes checkpointed task state, so a paused run can be resumed or dropped without half-applied changes. Teams that already route every tool call through their own orchestrator are most of the way there. Agents that call tools directly need a refactor first.

Whether any of this becomes a requirement on Microsoft's platforms is a separate question. The evidence is one post on X ^[6], and dev.to's write-up calls its contents recommendations, not implementation details ^[11]. Both reports place it after major AI companies acknowledged more incidents in which they appeared to lose control of their models ^[8]. It also followed a plan for more careful AI development that Anthropic CEO Dario Amodei published on September 12 ^[10].

We think the design is right for any agent with production access, whoever ends up mandating it. We would build the choke point first and the external log second, then the brake, because the brake depends on both.