Natural language autoencoders mostly miss a side task once it is trained into the model

The authors explain the gap ^[4] this way. The monitor mostly picks up readouts that verbalize the instructions, and not the covert computation ^[2]. In the in-context runs the instructions sit in the prompt. In the fine-tuned runs there are none, only a prepended secret ^[5]. On the authors' account the readout tracks what is in the context window ^[2]. It mostly reports the prompt back to the operator, who already had the prompt.

The authors note that incrimination rates alone cannot show the hiding is invisible ^[3]. So they built a causal test, and it is the part of the post I would copy. The patch swaps in the pre-carrier activation from a run hiding the other bit and reads the model's prediction at the carrier ^[6]. Then the NLA's reconstruction goes in place of the real activation, and a survivability score compares the two effects. A score of 1 means the whole effect survived and 0 means none of it did ^[8]. The authors treat a score above 1, where the reconstruction pushes harder than the real activation, as a likely sign the reconstructor went off-manifold ^[8]. The test does not care what the readout text says. It only asks whether the vector rebuilt from that text still sets the bit ^[8].

The evidence is one encoding scheme, measured at layer 32 ^[1][6]. The post's text describes the incrimination gap in words, as incriminating readouts appearing "much more often" in the in-context runs ^[5]. For the result to carry over to a production monitor, the behavior has to be learned in training. Its decision also has to sit at a narrow point in the sequence, the way one spelling choice does. I'd expect many behaviors a monitor exists to catch to meet the first condition. The post does not test the second.

The post is written against a specific hope. After Activation Oracles and then NLAs were released, parts of the interpretability community hoped that scalable unsupervised white-box monitoring had arrived, the authors write ^[11]. NLAs were reportedly the most effective technique for detecting eval-awareness ^[12]. The authors offer a conjecture that covers both results. Fine-tuning a steganographic task may "automatize" it ^[13]. Under the Global Workspace paradigm, automatic processing stays out of the workspace that NLAs probe ^[13]. In-context side tasks and effortful ones, such as deciding whether you are being evaluated, may get in ^[13]. The authors call it a conjecture.

The authors also take a second explanation seriously. The NLA may hold the secret in a form an independent monitor cannot read, with the verbalizer itself writing steganographically about the steganography ^[14]. Anthropic has considered this, according to the post. It saw some information loss when readouts were paraphrased under semantic equivalence, but it was hard to say whether the loss was steganographic or a by-product of going off-manifold ^[14]. The authors propose a training loss that minimizes reconstruction error across multiple semantically equivalent paraphrases ^[10].