Check the two headline numbers against each other before carrying either one anywhere. Mean per-sample correct probability after RL is reported as 0.231, up from 0.075 [9]. pass@64 is reported as 0.19, down from 0.83 [7]. Those cannot describe the same prompts at the same checkpoint. A problem solved with per-sample probability p is solved at least once in 64 draws with probability 1 - (1-p)^64, which is never below p, and at p = 0.231 with independent draws it rounds to 1.0 [19]. One of the two figures is therefore a different measurement point, most likely a later step on a curve the authors describe as still falling [7]. The direction of the finding does not depend on the pairing, but a reproduction should log both at the same step.
The contraction had somewhere specific to land. On the carry class, 87.5% of the base model's correct answers sat in sums 110 to 119, which the authors count as 1.7 effective values against 99 nominal [10][4]. The base's correct mass lived in the sampling distribution rather than the argmax [5]. Outcome-only reward has one term, whether the sample scored correct, so mass moves to whatever pays and nothing in the objective protects spread [11]. A beta=0 ablation reproduced the same contraction, which takes the KL anchor off the suspect list [11]. Instance memorization is off it too: fresh accuracy matched or beat seen [12].
The second blind spot runs the other way. On the parity family with a 90/10 SFT imbalance, odd-class greedy accuracy stayed at 0.000 through 3,840 odd examples in every RL run, which a greedy-only evaluator reads as RL having done nothing [13]. Sampling was the only channel carrying that class, and whether RL preserved it varied by seed [14].
The registered prior for the issue was that outcome-only RL is load-bearing exactly when base failures are systematic, because RL can then fix the wrong rule [1]. At near-zero base competence there is no rule to fix: about 60,000 rollouts produced zero correct samples, a wall for RL and for base-plus-search alike [2]. Learning that at 1.8 million parameters is cheaper than learning it on a cluster.
For the 0.83 to 0.19 number to say anything about a production run, three things have to hold. The class you care about has to start diffuse, with greedy well below pass@k [5]. The answer space has to be wide enough for entropy to have room to fall [4]. And your reward has to be outcome-only, with no term on the sampling channel [11]. The write-up is explicit that this is a small synthetic setup rather than a frontier model on real code or math, and that what exports is the pair of qualitative signatures plus the entropy diagnostic, not the magnitudes [16]. It does not test whether large-model RLVR shows the same signature [16]. Per-prompt answer entropy is cheap to log against a fixed prompt set. Sampling 64 times per problem is 64 times the generation cost of a greedy pass [20]. In my context that makes the entropy trace the thing that runs every checkpoint and pass@64 the thing that runs when the trace moves.
