Skip to content

Build1 publisher2 min readPublished

Eleven models encode 'no admissible answer' on an axis 85 degrees off safety refusal

A one-dimensional probe finds structural impossibility in the hidden state of instruction-tuned models from 1.7B to 70B parameters. The trained refusal pathway that guardrail work tunes reads an axis about 85 degrees away from it.

The Engineer · Build desk

Photograph accompanying Eleven models encode 'no admissible answer' on an axis 85 degrees off safety refusal
Photo: aclanthology.org

What happened

  • The paper's main grid covers 11 instruction-tuned models from 1.7B to 70B parameters across two structural-impossibility domains, math and code, for 22 model-dataset cells.
  • A one-dimensional probe reading a single residual-stream direction separates answerable from structurally impossible prompts with a mean AUC of 0.939, before any token is generated.
  • Paired base and instruct comparisons place the low-cosine geometry at the pretraining endpoint, largely before instruction tuning.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • exposure A stack whose only abstention route is a refusal classifier will pass a prompt that raises an attribute error and hand an answer-shaped token to whatever consumes it.
  • decision Teams that want abstention on impossible tasks have a cheaper candidate to test than another tuning run: one dot product per prompt against a fixed direction, at whatever layer they already instrument.
  • precedent A refusal benchmark stops being an answer to the abstention question; measure the cosine between the two directions in the model actually being served.

The probe reads one dimension. Take the pre-generation hidden state, project it onto a single vector, threshold the scalar. A mean AUC of 0.939 means that reader ranks a random impossible prompt above a random answerable one about 94 times in 100 [5][18]. The paper is careful about what that buys: "the model knows" is shorthand for a linearly accessible signal in the hidden state, and not a claim that the generation policy will use it [12].

Compare Arditi et al. (2024), where trained harmful-content refusal is mediated by a single residual-stream direction [7]. A cosine of 0.087 against that direction works out to about 85 degrees [16]. So a unit-length step along the refusal direction moves the recognition read-out by 0.087 of a unit [17]. That 0.087 averages over 22 model-dataset cells [4]. Refusal tuning can be said to have little leverage on structural abstention; it cannot be said to have none.

The paper also measures a second candidate route to abstention. An in-domain, behavior-defined invalidity-aware direction sits closer to recognition, though only partially aligned with it, and it too is near-orthogonal to safety refusal [8]. The authors read the confident-on-impossible failure as a routing failure rather than an encoding failure: the model carries a usable no-admissible-answer signal, and the safety-refusal pathway is not aligned to use it [11].

Steering carries the causal claim. Adding the recognition direction during generation moved invalidity-aware behavior in both directions and scaled with dose, while random directions did not [9]. Paired base and instruct comparisons put the low-cosine geometry at the pretraining endpoint [10].

For 0.087 to mean anything about your serving stack, your unanswerable prompts have to look like theirs. The paper tests structural impossibility: math and code prompts with formally checkable rules, and matched answerable/unanswerable pairs that differ in one diagnosable feature [14]. The headline claims cover that setting, and fact800 and FalseQA appear only as scoped boundary tests [13]. A false-premise question in prose is a different object from (1).startswith("1"), which raises an attribute error while a clean baseline returns True [3]. A model that answers True there never evaluated the expression. The abstract and introduction do not name the 11 models [20], so the two means describe a grid you cannot yet match against your own deployment. Code, datasets, aggregate experiment artifacts and analysis scripts are in the project's public repository [15].

What to watch

  • A fuller release naming the 11 models and giving per-cell cosines would show whether near-orthogonality holds everywhere or is an average over disagreeing cells.
  • Whether anyone ships a recognition-direction probe as an inference-time check and publishes its false-abstention rate on answerable prompts.
  • Whether the same geometry appears for false-premise natural-language questions, which the current headline claims do not cover.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories