format="json" is steering, not a contract: what it takes to put a local model behind an API

The trap is in the API surface. The same `format` parameter takes either a string or a schema object, and the two do entirely different work. The string `"json"` is steering: the model is pushed toward emitting a valid object, and nothing enforces field names, types or required keys, so `{"result": "..."}` is a legal answer to a request for `severity` and `summary` ^[3]. The schema object is enforcement ^[4]. Both calls return without complaint, and the response gives you no way to tell which of the two you got.

Enforcement here is mechanical, which is why it is worth the migration. Ollama zeroes the probability of any token that would violate the schema at every generation step, so a markdown fence is not discouraged, it is unreachable ^[4]. No prompt wording does that. On Ollama 0.3.0 or newer the Python client passes the schema through with Pydantic's `model_json_schema()` ^[5], and the dev.to walkthrough reports that for straightforward schemas on a 7B-or-larger model this alone ends the parse failures ^[19].

The mask has a narrow remit, though. The same piece lists three things that still bite, and only one of them sits inside that remit: small models fumbling nested and optional fields ^[20]. Of the other two, one is a deployment fact, since an older Ollama or any endpoint offering only loose JSON mode never applies the mask at all ^[6]. The other is a job no decoder constraint can do. Where the text says the salary is competitive and your schema demands an integer, the model supplies an integer ^[8]. Two of the three named failure modes are therefore outside what constrained decoding can reach ^[18].

That second one is the expensive failure, because it survives every later layer. A fabricated integer satisfies the grammar that produced it and satisfies the validator that inspects it, since both only ever asked what type the value was ^[17]. The feedback retry does not catch it either: the retry is triggered by a validation error, and there is no validation error to feed back ^[1].

The retry ladder is where the budget breaks. `instructor` gives it to you in configuration, with `max_retries=2` and a `timeout` the source flags as total across retries rather than per call, precisely because local models are slow ^[9]. At the example's 30 seconds, that is three model calls in 30 seconds, roughly ten apiece ^[16], on hardware chosen because you own it. Rung two of the ladder is theoretical unless that number goes up.

On the defensive side, `json_repair` earns its place as the `json.loads` replacement, in preference to the regex that strips fences until the night it does not ^[7][2]. Its default ordering is the safe one: stdlib parse first, repair parser only on failure, so valid input passes through untouched ^[10]. `skip_json_loads=True` removes that fast path, and the library's own docs warn against pointing it at input you expect to be valid, because the repair parser can reshape valid JSON ^[11]. Where you cannot vouch for the provenance of the bytes, `strict=True`, which raises instead of repairing, is the setting that tells you the truth ^[14].

And `instructor` does not replace the mask. It talks to the OpenAI-compatible endpoint, which still depends on the backend honouring the schema ^[12]. Constrain, defend, validate, retry are four jobs, not four names for one ^[13]. The retries are what you run when the constraint is not there.