Leadership1 publisher2 min readPublished
Rule-based elision before summarization led on efficiency among context-management strategies
Researchers held a coding harness's execution loop fixed and varied planning, tools and context management across four models, and found that what each component is worth depends on how strong the model already is.
The Board Room · Leadership desk

What happened
- A study of coding-agent harness design held the execution loop fixed and compared 176 matched settings across four models on SWE-Bench Verified and Terminal-Bench 2.1.
- Context management became more valuable as the context-window budget tightened, and most of its benefit came from preventing context-overflow failures rather than from better reasoning.
- Making elided content recoverable added machinery the models rarely used and produced no accuracy gain.
- Planning worked as an accuracy scaffold for weaker models and as a cost saver for stronger ones, with little change in accuracy either way.
Compiled by The Board RoomSomething wrong?How this is made
Why it matters
- cost A bash-only interface is the cheaper option only where the model is already bash-capable; for a model with weaker bash proficiency the saving is paid for in success rate, so the interface choice is priced per model.
- constraint Because each component's value is indexed to model capability and context budget, every model swap reopens the planning and action-space choices.
- decision A team currently building recoverable-context machinery has a reason to check first whether its model ever calls it, since the study found no accuracy return on that work.
- precedent Publishing per-component ablations against a fixed loop sets an expectation for how harness claims get reported, and makes whole-system comparisons harder to present as evidence about any one component.
Trajectory analysis is where the three components separate. Context management extends execution trajectories without substantially altering agent behavior [14], planning changes where trajectories stop [15], and the action space changes the granularity at which code is written [16]. Most of context management's value comes from preventing context-overflow failures, which otherwise end a run before the agent gets to code modification and verification [4][6].
The elision result is narrower than the ranking suggests. Staging rule-based elision ahead of LLM-based summarization was the strongest of the five strategies on overall efficiency [7], and the paper describes it as maintaining "mean success similar to the other managed strategies while controlling peak context and reducing reliance on summarization calls" [8]. Equal accuracy for fewer summarization calls is a cost finding. The more elaborate variant, in which elided content can be recovered, "adds machinery that models rarely use and yields no accuracy gain", the authors write [9].
The planning result splits by model in a way that matters for anyone reading a single benchmark number. For weaker models, planning keeps the trajectory alive long enough to attempt an edit, raising success at additional cost; for stronger models it mainly removes redundant post-edit verification, lowering cost [11]. The authors frame the contribution as informing model- and budget-aware harness design [19], which is a fair description of a set of findings that change sign depending on the model in the loop.
Five context-management strategies at four context-window budgets on four models comes to 80 settings, leaving 96 for the targeted planning and action-space ablations [21]. The paper page gives the abstract and a findings summary; it does not name the four models or report absolute success rates and costs [20]. The posted summary states the gap the study is aimed at: "Most evaluations compare complete agent systems, making it difficult to separate the contributions of individual harness components" [18].
Four unnamed models on two benchmarks is a thin base for point estimates. The direction holds across all four: weak-model and strong-model configurations diverge, and they diverge mostly in cost [11][12].
For a team already running an agent, the ordering in the paper is the usable part: overflow prevention is what context management mostly buys back [4], and the recoverable variant bought no accuracy at all [9].
What to watch
- Replication of the elision-before-summarization ordering on benchmarks other than SWE-Bench Verified and Terminal-Bench 2.1.
- As context windows widen, the accuracy case for context management shrinks: the paper already reports that benefit diminishing as the window expands.
- Whether harness-native follow-on work such as openJiuwen adopts the same fixed-loop, component-level evaluation protocol.