Probes on a 27B open model match direct probes of a 397B model on deception

The reader never touches the author's activations. It is prefilled with text the author generated, and a linear probe is trained on the reader's own hidden states while it processes that text ^[10][14]. The post's intuition is a distillation chain. Kimi is distilled from Fable, so Kimi's internal state while reading Fable's output may say something about Fable's ^[14]. The probe therefore measures how the reader represents a transcript. Whether that tracks the author is what the post calls the faithfulness gap: author probe performance minus reader probe performance ^[7].

For auditors, the input requirement decides whether this is usable at all. Surrogate Fidelity, the closest prior work, needs logits from the closed model, and those are not commonly available ^[15]. Hermeneutics needs only text. Pre-deployment evaluators typically get API access with reasoning ^[13]. A reader can run on that.

The 0.004 figure is an average. It compares the Qwen3.5-27B reader with authors from 30B to 397B ^[3][4], on deception probing reproduced from the UK AISI's Did You Lie paper ^[6]. The largest author has about 14.7 times the reader's parameters ^[1]. The Qwen3.5-9B reader's gap was 0.026, 6.5 times larger ^[5][2]. The post defines the gap as author minus reader but reports the 9B result as -0.026 and calls it a fall-off ^[5][7]. I treat both figures as magnitudes.

For the number to transfer, a closed model has to be as legible to the reader as the open-weight authors were. That cannot be checked directly. The team baselined on open-weight pairs because closed-weight internals have no ground truth ^[8]. In the closed-model experiments, probes trained on off-policy generations from three frontier models were compared with probes trained on the reader's own on-policy generations ^[10]. That comparison shows what reading another model's text costs the probe. It cannot place the reader against an author-side probe for Claude Opus 4.6 or GPT-5.4, because those weights are not shared ^[9][17].

Distillation narrows the gap. According to the post, distilling a model organism author into a reader recovered about 75% of the author-reader difference for deception probing ^[11]. That share could be computed only because the organism itself could be probed ^[7][11].

On cost, the post says probes beat LLM judges on dollar cost per evaluation in some cases, "showing potential as a competitive choice once they are similarly performant" ^[12]. The available portion of the post does not include the per-evaluation costs.

This is about 1.5 months of work ^[16]. The team defined a gap and measured it on open-weight pairs before applying the method to closed ones, and that is the correct order. A footnote says hermeneutics is about interpreting texts whose authors could not be asked directly ^[18]. These authors can be asked; the case for interpretability is that it does not depend on models telling the truth ^[17]. I think a reader probe is a defensible second signal beside an LLM judge for an API-only evaluator, and the authors pitch it the same way, as a complement to blackbox monitoring ^[13]. I would want each closed-model result published with the open-weight gap for its nearest comparable pair before anyone calls it an internal view.