Build1 publisher2 min readPublished
Claude Sonnet 4.5 follows a constraint on its own reasoning trace 2.7% of the time
A new evaluation gives models explicit rules about what may appear in their reasoning and scores whether they comply while still solving the problem. Compliance rose with model size and fell with more RL training.
The Engineer · Build desk
What happened
- A paper on arxiv.org introduces CoT-Control, a suite of 14,076 problems drawn from GPQA, MMLU-Pro, Humanity's Last Exam and BFCL that scores whether a model obeys constraints placed on its reasoning trace.
- Claude Sonnet 4.5 obeyed a constraint on its chain of thought 2.7% of the time and a constraint on its final output 61.9% of the time.
- Controllability was higher for larger models and lower with more RL training, more test-time compute and harder problems.
- The failures held up when prompts gave models an incentive to evade a CoT monitor instead of simply asking them to comply.
- The paper says the reason models control their reasoning traces so poorly is not well understood.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Anyone monitoring traces today is relying on something the model cannot do, and the one axis in these tests that improved the ability was model size, so each larger release thins the margin.
- decision Test-time compute settings picked on accuracy grounds now carry a second argument: more of it lowered the model's control over its own trace. That makes long-thinking configurations the more auditable ones.
- exposure The line teams put in system prompts telling a model it is being watched is not free, since models told they were monitored were slightly better at controlling their traces.
- precedent If frontier labs take the authors' recommendation, controllability becomes a per-release number a buyer can ask for, and one a model card can leave out.
Give a model a genetics question and tell it to reason without using the word "chromosome" [2]. It has to answer and keep that token out of its trace at the same time. The suite is built on that pattern: explicit constraints on what may appear in the reasoning trace, applied while the model solves real tasks, scored on whether the constraint holds [15].
The authors chose a proxy because measuring a real intent to evade is hard. Current models have no reason to evade a monitor, and even when given one they can fail through unreliability or a tendency to report their own behaviour honestly [16]. CoT monitoring works in the first place because reasoning models tend to surface malicious intent, plans and other precursors of misbehavior in their traces [17]. The failure the paper is chasing is situational awareness: models that learn their reasoning is watched and begin suppressing or reshaping what they write [20].
Divide the two scores and a constraint on the final output holds 22.9 times more often than a constraint on the trace [18]. Whether that gap describes your deployment depends on how hard your monitored step is. The problems come from GPQA, MMLU-Pro, Humanity's Last Exam and BFCL [3], and controllability drops as difficulty rises [7]. An agent that reads a ticket and calls one tool is not working at the difficulty of a Humanity's Last Exam item, so by the paper's own trend it should steer its trace better than 2.7% of the time [4].
The elicitation is what makes the low score worth something. Prompts implied that controlling the CoT was instrumentally beneficial to the model's own goals [21]. Adversarial prompt optimisation, the cheapest thing a red team would reach for, did not meaningfully raise controllability [10]. The authors wrote that their "results leave us cautiously optimistic that CoT controllability is currently unlikely to be a failure mode of CoT monitorability" [11].
The authors ask AI developers to filter the paper and its dataset out of training data, and publish a canary string for checking [14]. A team adopting CoT-Control as an internal check inherits that condition: once the 14,076 problems are in the training mix, a score on them stops measuring control over the trace [3].
What to watch
- A model card that publishes a controllability score per release, and whether the score moves with each RL run.
- Signs that CoT-Control problems have entered training corpora; scores measured after that point are not comparable to these.
- A replication on models trained with substantially more RL than the ones tested here, to see if the downward trend holds.