Topic

Alignment midtraining

A training phase between pretraining and post-training in which a model continues pretraining on documents about how it should behave, such as a developer's constitution or behavioural specification.

Current clusters

build1 publisher

Competing finetuning overrode a midtraining run 3,800 times its size

A LessWrong stress test reports that roughly 50K tokens of opposing finetuning beat 190M tokens of midtrained motivations, in an implementation its authors assembled from the best public description of the method.

Publishers:lesswrong.com

Reality

Evidence38
Adoption22
Hype gap+20
Incentives55
Confidence45