The gap between the two arms that both contain model prose is the number that actually decides things in that table, not the ratio to chance. Treatment sits at 0.175, the arm labelled prior-control at 0.129, the random basis at 0.044 [14]. Subtract: 0.046 of separation, which is 0.002 more than the entire chance floor [2]. Put the control's lift over chance against the treatment's and 65 percent of the above-floor agreement is already sitting in the control arm [5]. The bootstrap intervals touch as well, with the treatment reaching down to 0.153 and the control up to 0.162, an overlap of 0.009 [3]. Against chance the effect is just under four times the floor [1]; against the model's own output it is a gap the intervals will not separate.
That is one direction a baseline lies in. The louder failure went the other way. The first version of the random arm harvested its vocabulary out of the control arm's own readings, on the reasonable ground that you want model English and not dictionary English [9]. Seven readings do not carry much vocabulary. The random texts came out as near-duplicates of each other, and the cosine floor rose to 0.391 while the model readings it was meant to sit under were at 0.232 [10]. The floor exceeded the thing it was measuring by 0.159, a factor of 1.68 [4]. The comment now in the code puts it better than a paraphrase would: a floor that climbs the less you sample it is a sampling artifact, not a floor [11].
Note the direction of that error: baseline 0.391 against treatment 0.232 reads as no effect, the kind of result a team would ship as a clean null. The floor's origin, just seven readings, never appears in the output [12]. The artifact has no field for that.
The fix is the part worth copying. The alphabet now comes from a request that never mentions any message, "list 240 common English words, mix lengths 1 to 12", so it is still the model's vocabulary but cannot inherit the convergence it exists to measure, and the run prints which source it used and records the fallback in the saved artifact [13].
What transfers to your pipeline is that design, not the 0.175. Positional agreement only works because every text inside an arm shares a length profile, so slot 4 means the same thing in two readings [7]. Judge scores and majority votes have no slot alignment, and the writeup already reports that the embedding cosine was worthless on this data [8]. The rule that does transfer is the sampling one: separate processes, no shared context, because one sampled list of five guesses is one reading and not five [6]. If your self-consistency harness parses five answers out of a single completion, your N is 1. The author's own framing is that anyone running such a pipeline already has both baseline failures in their numbers [16], and with two arms the first message alone would have been written up as recovered signal [15].
The dev.to author reports that a second message reversed the effect and calls that reversal the most useful result in the run [4]. The supplied text stops mid-sentence, while describing what a two-arm design would have concluded, before any table for that second message appears [17].
