A green build only proves your agent was consistent with itself

Salesforce's engineering team has published an account of what went wrong when it used AI coding agents to build an AI-powered mobile application designer: the agents wrote good code, and the team could not explain why it should be trusted [s1c5]. Its diagnosis is uncomfortable for anyone currently measuring agent output by merge rate, because it holds that the checks doing the measuring cannot see the failure mode.

The scenario the post describes is deliberately unremarkable. A large pull request arrives, the build passes, every automated test is green, code review finds no major issues, and the change is merged [s1c1]. Later the team finds that the agent quietly misinterpreted an ambiguous requirement early in the implementation, and every decision after that was internally consistent with the misreading [s1c2]. The tests validated the implementation and the reviewer approved the diff [s1c3]. Nothing in the process ever proved the interpretation was correct before the code reached production [s1c4]. Each of those gates takes the diff as its input, which is why a full sweep of green signals establishes agreement between the code and the agent's reading of the task, and says nothing about agreement between that reading and the requirement [s1d1].

The mechanism is that many paths lead from a requirement to the same visible result. An agent may add a utility that already exists, honour a framework convention while breaking a repository-specific one, or resolve an ambiguity without disclosing that it made a choice [s1c6]. Code can compile and still increase technical debt [s1c7]. That leaves the final reviewer reconstructing what happened before the diff existed: which assumptions were made, whether existing components were searched for, whether the tests encode intended behaviour or merely implemented behaviour [s1c8]. As agents produce larger changes, generation outpaces human review capacity [s1c9], and Salesforce's conclusion is that trust which cannot be reconstructed at the end has to start earlier [s1c10].

The fix is procedural rather than clever. Before implementation, the team required a specification stating required behaviour, measures of success, assumptions, unresolved questions and failure conditions [s1c11]. That document became the contract: the plan had to show how it fit the repository, the tests had to encode its success criteria, and reviewers could assess the code against the same definition of success [s1c12]. Salesforce calls this an agentic Spec-Driven Development workflow, structured as four phases with a gate that must pass at each [s1c13], [s1c14].

The interesting part is the second-order failure. Making the contract explicit surfaced so much uncertainty that the first version routed nearly every unresolved question to a person: whether a component existed, which utility performed an operation, which naming convention applied, which command ran the tests, why an earlier implementation was structured as it was [s1c15]. The human became a search service for the agents [s1c16]. Salesforce's split is between lookup, which has an evidence-based answer recoverable from code, docs, tests or history, and judgment, which requires a decision that evidence cannot supply, such as which user experience is preferable or whether a compatibility break is acceptable [s1c17]. Agents resolve the first; humans decide the second [s1c18].

Watch the loose end the post names itself: once an agent answers its own lookup questions, you need a way to know it found something real [s1c19]. A specification gate that agents populate with their own unverified findings recreates the original problem one layer up.