The confidence field is emitted by the same forward pass as the tool call it grades. The proposed decision object carries tool, args, confidence and reasoning together, and the rule governing the number is a line in the system prompt: set confidence below 0.8 if you are unsure about the data or the tool [9]. That is a request to the model, not a check on it. The orchestrator then branches on confidence below 0.8 and holds execution for a fallback or a human [10]. The branch is only as sharp as the calibration behind the number, and the post reports no calibration for it [14]. Log the score next to the outcome for a week before you let it gate anything.
The other half of the design needs nothing from the model. The pre-conditions in the interrupt pattern inspect the tool, not the generation: read-only executes automatically, state-modifying requires confirmation, and only the third branch consults the confidence score [11]. Read-only versus mutating is set when the tool is registered, so it can be checked before any inference happens. That is the part I would ship first, because a property of the registry still holds when the model is confidently wrong.
The hand-off is worth fixing at the same layer. In the described break, the loop returns tool output to the model as ground truth, so a 500 body becomes evidence and the model reasons past it, retries with a modified query, or writes an answer out of the error text [5]. An executor that raises on a non-2xx and halts the run never puts that text in the transcript at all. You lose the occasional legitimate recovery in exchange for a stack trace you can grep.
The 98/2 split [2] is a claim about someone else's traffic mix. Read the 2% as per step and apply it to the five-step refund walkthrough [7]: 1 - 0.98^5 = 0.0961, so roughly one run in ten touches at least one bad step [13]. That arithmetic assumes independent steps, which the same post rejects when it argues that errors compound through the chain [6]. For the 2% itself to transfer to your stack, you would need argument validation as strict as theirs and tools that fail loudly rather than returning a 200 with an error blob inside.
My ordering in this context: the schema-level gate on writes first, a hard halt on tool errors second, and confidence scoring last, instrumented, and only after the number has earned its threshold.
