Build1 publisher3 min readPublished
Thought-anchor scores change when a second model resamples the same trace
A 20-hour MATS project replayed one 14B model's published chains of thought under two other reasoning models. Sentences its own resampling had called causally important scored as ordinary under both readers.
The Engineer · Build desk
What happened
- A LessWrong write-up reports that sentences one model's own resampling flags as causally important in its chain of thought come out ordinary when a second reasoning model is teacher-forced with the same trace.
- The writer model was DeepSeek-R1-Distill-Qwen-14B, replayed from its published Thought Anchors and Thought Branches traces, rollouts and per-sentence importance scores.
- Two reader models took the transplants: Qwen3-1.7B from a similar family, and gpt-oss-20b from a different one.
- An initial paired test looked weakly positive on the anchor question, and the follow-up full resampling did not confirm it.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost The bill for step-importance analysis lands on whoever runs it at the writer model's inference price, about 100 rollouts per sentence, with no smaller model standing in.
- constraint A CoT monitoring signal calibrated on one model's anchors cannot be reused on another model, so each model and each checkpoint needs its own resampling run.
- decision Anyone building a hint-following or unfaithfulness eval has to check whether the trace mentions the answer options before treating a transplanted result as evidence.
- contradiction Screening statistics and full resampling pointed opposite ways on the same data, which sets the bar for any positive transfer claim on a set this size: the full run, not the screen.
The Thought Anchors procedure cuts a trace at a sentence, has the model rewrite that sentence, continues about 100 times, keeps only the continuations whose new sentence says something different, and compares the resulting accuracy against continuing from the original [1]. Sentences whose replacement moves accuracy a lot are the anchors [1]. Every accuracy figure in that comparison is produced by the model doing the continuing.
So the transfer question is a budget question. "If the important steps are in the text, another, possibly cheaper model can resample a transcript to recover which steps mattered and how likely a given answer was," the author wrote [3]. Qwen3-1.7B is the cheap candidate: by the model names, about an eighth of the writer's parameter count, since 14 / 1.7 = 8.2 [9]. Phase 1 tested that substitution on Thought Anchors rollouts for 20 MATH problems, taking the top three anchors per trace from the dataset's own accuracy-based counterfactual scores, using negative-importance anchors in the traces that ended correct and positive-importance ones in the traces that ended wrong [10].
The hinted-trace result is the one that survives the swap, and it is narrow. Thought Branches supplies MMLU traces whose writer silently followed a hint [17]. Teacher-forced into a different model, such a trace has a significant impact, but only when the trace already discusses the answer options; without that, the post reports no effect cross-model or within the same model [4]. The author's takeaway is that the part which transfers is the part arguing for the answer [13].
The setup is careful in the place that matters for a null result. Before running the comparisons, the author checked qualitatively and quantitatively that reader models are willing to continue a teacher-forced trace from another model, with each model's own instruction template, its own thinking-block delimiters, and thinking turned on [12]. Sampling used each model's recommended hyperparameters, and the cap was set to 32,768 tokens because reasoning was not finishing [7]. That doubling puts the original Thought Anchors cap at 16,384 [8].
One writer model, two readers, 20 problems [5][6][10]. For the null to say something about resampling in general, it would have to hold with writer models from other families and with more traces per problem. The post states the limit plainly: the number of evaluated models and CoT traces is limited, and the findings, while true there, may not apply universally across reasoning models [15]. It was a 20-hour project for Neel Nanda's Winter 2027 MATS stream [15].
In my view the result is the right way round for anyone planning a monitor. A cheap reader that agreed with the writer about which steps mattered would have been the convenient answer, and the convenient answer is the one that needs the most traces before it gets built on. Here the writer was DeepSeek-R1-Distill-Qwen-14B, and everything ran locally on vLLM across four RTX PRO 6000 cards [5][7].
What to watch
- A writer model from outside the DeepSeek-R1 distill family would show whether the null result is about this writer or about resampling generally.
- The post says the Thought Branches problem set needs cleaning before reuse, so the hint-transfer result is pending a rerun on a cleaned set.
- Whether the hint effect holds when a trace refers to the answer options obliquely rather than listing them.