Build1 publisher3 min readPublished
Cognition put a cost penalty inside SWE-2's reinforcement-learning objective
The model averages 53 steps a run against SWE-1.7's 127, and Cognition says its mean rollout cost is 64% below Fable 5.1's on a leaderboard Cognition built, runs and grades. Reproducing that takes Devin's harness.
The Engineer · Build desk

What happened
- Cognition released SWE-2 on September 10 in Devin Desktop and the command-line interface, with a rollout underway across Devin Web and Fusion.
- On the 100-task FrontierCode Main set, SWE-2 at medium effort averaged 53 steps per run, down from 127 for SWE-1.7.
- On Terminal-Bench 4, SWE-2 scored 27.3% against 55.8% for Fable 5.1 and 57.9% for GPT-6 Astra.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision Buyers comparing coding agents now have to price mean rollout cost per finished task alongside solve rate, and the reasoning-effort setting becomes a procurement parameter instead of a debugging preference.
- constraint Nothing published with the release lets a team reproduce the cost saving outside Devin, so the figure stays a hypothesis until someone re-runs it on their own repositories with their own grader.
- contradiction Cognition claims near-parity with the leading proprietary models while reporting a 30.6-point deficit to GPT-6 Astra on Terminal-Bench 4, so the parity claim is workload-specific and cannot be quoted as general equivalence.
- capability If a cost penalty in the training objective reliably shortens trajectories, spend can be tuned in post-training rather than by dropping to a smaller base model.
Cognition built a cost penalty into the reinforcement-learning objective, and that is the training change worth reading closely. Cognition says a single reinforcement-learning run tuned the medium, high and maximum reasoning-effort settings together, with each setting carrying a cost penalty calibrated to the local slope of the base model's cost-performance curve [5]. That is an objective that credits extra tokens only when they buy more solve rate than the base model would have bought for the same spend at that point on the curve. Cognition's behavioral figures move the way such an objective predicts: the first substantive code edit arrives after a median of 18 steps, against 48 for SWE-1.7 [17], which is 63% fewer [2].
The cost claim carries conditions. First, the workload has to look like FrontierCode's 100-task Main set [16]. Then the harness: coding models are not mapped one-to-one onto agent harnesses, and Moonshot's Kimi K3 documentation reports evaluations run through Kimi Code and Claude Code [11]. Effort setting matters too, because Cognition reports the best score across reasoning-effort settings [10], so the reported figure is a best-of-three result across those settings, not a single run.
The two cost figures also fix a ratio Cognition did not print. A mean rollout cost 64% below Fable 5.1's puts SWE-2 at 0.36 of Fable, and roughly a quarter of GPT-6 Astra's puts Astra at four times SWE-2 [9]. On Cognition's own arithmetic, Astra's mean rollout cost is about 1.4 times Fable's [4].
FrontierCode itself moved in July. Cognition audited more than 1,000 grading criteria, relaxed 75, changed how internet use is judged and retired the Diamond subset for version 1.1 [12], about two months before SWE-2 shipped [7]. Changes of that size can make an evaluation better or worse. The ranking that puts SWE-2's 50.0% behind Fable 5.1's 50.9% and ahead of SWE-1.7's 42.0% [8] rests on criteria that Cognition itself rewrote two months earlier.
The results are uneven inside the same announcement. SWE-2 took 92.8% on Terminal-Bench 2.1, ahead of every comparison model Cognition listed [14], and 73.0% on DeepSWE 1.1 against GPT-6 Astra's 74.1% [13]. On Terminal-Bench 4 it is 30.6 points behind Astra, under half its score [5], and Cognition has not said why the gap is that wide.
Cognition puts its post-training gain at five to six percentage points on several coding evaluations [4]. If that held on the Main set, the base would have scored around 44 to 45, nearer SWE-1.7's 42.0 than Fable's 50.9 [6], but Cognition has not confirmed that the gain held on that set. The base is unconfirmed anyway: Cognition says SWE-2 was post-trained from Kimi K3, and no outside party has independently verified that claimed lineage [3].
Some of the saving comes from systems engineering underneath the training objective, and it is decent work. Cognition says it trained an online draft model for speculative decoding and used lower-precision NVFP4 and FP8 kernels to hold memory use down while working from a base model almost three times the size of SWE-1.7 [7]. It also tripled the number of reinforcement-learning environments and used earlier SWE-2 checkpoints to find weaknesses in its own automated verifiers [6].
What an outside team can settle cheaply is behavioral. Steps to first edit and tokens per finished task, counted on its own repositories at medium effort, do not need Cognition's grader. The solve-rate ranking still needs Cognition's grader.
What to watch
- An evaluation of SWE-2 run outside Cognition's FrontierCode, or inside a harness Cognition does not control.
- Confirmation or denial from Moonshot that SWE-2 was post-trained from Kimi K3.
- Terminal-Bench 4 numbers at high and maximum effort, which would show whether the cost penalty is what caps long-trajectory tasks.