Misbehaviour scores and past failure rates predict fine-tuning misalignment before training

The authors explain the result with one property of their data: misalignment emerges broadly ^[11]. A dataset that causes any failure mode usually causes several at once ^[11]. That splits the prediction into two inputs. The misbehaviour score is an LLM's rating of how hard a dataset pushes toward misbehaviour of any kind, and it says whether anything will happen ^[3][11]. The emergence rates record how often each failure mode appeared in past fine-tuning runs, and they say which modes tend to show up when something does ^[3][11]. The authors wrote that this account "shouldn't be interpreted as a broader claim about the nature of misalignment" ^[18].

Raw rows are a weak signal. The best-known case is models fine-tuned to write insecure code that began encouraging self-harm (Betley et al. 2025), and the authors write that reading the data often does not settle whether this will happen ^[14]. When frontier LLMs were also handed the misbehaviour score and emergence rates, they predicted failure modes about as well as the authors' simple regression, but their probabilities were poorly calibrated ^[6]. Calibration is the property a gate reads. A rule that blocks a run when the forecast crosses a threshold only works if a stated probability matches how often runs actually go wrong. The regression also costs less to run, a result that did not need 17 fine-tuned models to establish ^[2]. I would put the regression in the pipeline and keep the LLM where it already sits, producing the misbehaviour score ^[3][6].

Emergence rates are the expensive input, because they come from past fine-tuning runs with measured failure modes ^[3]. The authors frame the method as extrapolating from historical observations on weaker models to predict whether fine-tuning will misalign a new, stronger model, and they call the evidence preliminary ^[13]. That assumes failure-mode base rates carry across model scale. A full cross of the benchmark's 17 models, 32 datasets and 16 failure modes would be 8,704 triples, against the more than 5,000 the post reports, so some pairings appear to be missing from the grid ^[17]. The post's summary does not state a figure for "well above chance" ^[3].

The forecast scores only a whole dataset ^[7]. To act on rows, the authors pass its output to an LLM classifier that drops the rows likely to cause the predicted failure mode ^[7]. The filtered set beat two controls: dropping a random half of the rows, and running the same classifier without the forecast ^[8]. The second control is good design. It separates what the forecast adds from what an LLM filter would catch on its own ^[8].

Every clearly positive result measured misalignment with multiple-choice questions, and the training data was synthetic SFT ^[4][9]. "We do not know how well that tracks deployment behavior," the authors wrote ^[10]. For these numbers to transfer to a production fine-tune, the data would need to resemble the synthetic sets, the failure modes of concern would need to be ones a multiple-choice probe detects, and the team would need an emergence history of its own ^[3][4][9].