Build1 publisher2 min readPublished
An afternoon of oneOf branches replaced the paragraph carrying a tool contract
Two models read the same tool description and disagreed about one field name. The fix moved the shape into the JSON Schema for the nine pattern kinds that account for 85 percent of emissions, and left the rest loose.
The Engineer · Build desk
What happened
- Two models given the same prompt, tool description and request returned different object shapes for the same pattern argument, and the downstream patcher rejected the second one with a 422.
- The tool schema for that pattern argument required only a string kind, and the real shape lived in the tool description as a paragraph of prose with examples.
- Nine common kinds became discriminated oneOf branches with real required lists, so EntityPattern now demands kind, entityName and definition instead of describing them.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Proprietary field names sit outside every model's training data, so the class of failure the author describes cannot be bought away by upgrading to a larger model.
- cost Strict coverage of every kind was priced at a week plus ongoing maintenance, and the partial version cost an afternoon by leaving the rare kinds to fail at runtime instead of at the schema layer.
- decision Teams that test a flat schema on one model are choosing where the contract lives, and the schema keywords that would move it are already supported by the two providers the post names.
The two branches only work as a pair. EntityPattern pins `kind` to the const `entity` and requires `entityName` and `definition` [13]. The fallback, OtherPattern, accepts any `kind` that is not one of `entity`, `requires`, `toggleable`, `nav`, `writeHandler`, `queryHandler`, `hook`, `notification` or `metric`, and requires nothing else [14]. So the bad output, `kind: "entity"` sitting next to `name` and `fields`, misses the required list on the first branch and is excluded from the second by name [27].
That exclusion list is also the maintenance bill. The nine kind names are written twice, once as const discriminators and once inside the `not`/`enum`, so a tenth strict branch means editing both places [26].
The post's explanation for the original divergence is exposure and not capability. `entityName` and `definition.fields` are the framework's own names, so a model that has seen the shape retrieves it, and a model that has not guesses from the description, where `{ name, fields }` is the better guess [8]. The author writes that you cannot fix an exposure gap by paying for a bigger model [9]. The runs described do not isolate that variable: they changed the schema and measured the output, so training exposure and the tightened required lists move together.
Three fixtures, run live twice, cost about $0.18 in total [22]. That is six calls at roughly three cents each [24]. Two of three passed before the change and three of three after, and the one that flipped was the rename-entity fixture that had been emitting `name` and `fields` [22][23]. Six calls for eighteen cents is a cheap experiment and a small one. "The evidence is thinner than I'd like, and it points the right way," the author wrote [21].
Two conditions have to hold before that result transfers. Your provider has to honor `oneOf` and `allOf`/`if`/`then` in tool input schemas, which the post says both Anthropic and OpenAI do [19]. And your discriminator distribution has to be concentrated enough that a handful of branches covers most traffic: here nine of about twenty kinds carry roughly 85 percent of emissions [10], which leaves about 15 percent going through the permissive branch to the runtime patcher [25][15].
The per-operation rules use the same machinery, `allOf` with `if`/`then`, so a `replace` carries `id` and `pattern`, an `add` carries `pattern`, and a `remove` carries `id` [17][18]. The post says most people skip these keywords because the flat version works well enough on whichever model they tested with [20].
What to watch
- Whether a fixture set larger than three reproduces the flip, and whether any strict branch starts rejecting output the runtime patcher would have accepted.
- Whether Anthropic or OpenAI change their handling of oneOf and allOf/if/then in tool input schemas.
- Whether the fallback branch's nine-name exclusion list drifts out of sync the first time a tenth strict kind is added.