A 0.7 confidence gate lets 10 to 20 percent of wrong Whisper tokens through in noise

A token confidence score is the model's own estimate of itself. Calibration is what makes that estimate usable as a gate, and the post defines it plainly: a model is calibrated if, among everything it labels 70 percent confident, roughly 70 percent turns out correct ^[12]. The study cited puts a number on the gap for Whisper under noisy conditions, where 10 to 20 percent of tokens were wrong while scoring above 0.7 ^[13].

The direction of the failure is what breaks a threshold design. The same study found that overconfident errors increase specifically as signal-to-noise ratio drops ^[14]. Tune the gate on clean recordings and you measure its best false-accept rate in the one condition that never occurs in an exam room.

For that 10-to-20-percent band to transfer to your pipeline, you need the same model family, noise at a comparable level, and a token score computed the same way. MedScribe, the pipeline described in the post, runs offline FasterWhisper for transcription with LLM-based medical term extraction on top ^[9], so the model family matches. The post reports overconfidence as a documented property of deep networks in general, across vision, structured prediction and language tasks ^[15], so the direction is not specific to speech.

The clinical numbers come with their own transfer conditions. Emergency department notes produced with speech recognition averaged 1.3 errors each, with 15 percent of those errors judged clinically significant ^[4]. Multiply through and that is 0.195 clinically significant errors per note, about one in every five notes ^[16]. Those are ED notes, so the rate carries to an outpatient pipeline only if the vocabulary and the room sound similar. A separate comparison in the post found physician notes produced with speech recognition carried four times the error rate of notes produced without it ^[5].

The single-token version of the problem is the discharge dose: 8 units became 80, a tenfold dose ^[1][17], from a system that heard "eighty" for "eight" and transcribed it with total confidence ^[2]. The post's author says it is a documented patient safety incident reported through ISMP's medication error tracking ^[3]. Another transcription in the post recorded a patient's allergy as "XOPENEX", a bronchodilator, when the clinician had said "sulfa" ^[6].

The post also documents what came first. The early 2025 version of MedScribe was Whisper wrapped in a UI, transcribe and display ^[10], and the author says watching the pipeline mangle "atorva statin" changed the question from transcription accuracy to when the model should refuse to touch a medical term at all ^[11].

Before speech recognition, human transcriptionists caught the contextually impossible, a report mentioning ovaries for a male patient among them, and flagged it before it reached the chart ^[7]. What replaced that checkpoint, in some records, is a standing disclaimer warning that "wrong-word or sound-a-like substitutions may have occurred" and asking readers to catch them manually ^[8].

On the architectural claim, the material stops short. The version of the post available ends on the general overconfidence finding without stating the rule MedScribe uses to decide which terms it will not edit ^[18]. Word error rate on clean held-out audio will not surface the problem. I would expect any workable gate to need a signal the model does not produce itself, because the score degrades under exactly the condition it is meant to detect ^[14].