Build1 distinct publisher3 min readPublished
The Copilot team put a token-shortening utility through its agentic benchmarks, watched task cost rise even as individual responses shrank, and shipped a compressor that only touches install, build, test and progress output.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
The load-bearing piece of this design is the allowlist, not the compression. GitHub sorted tool output by command class after reading its own benchmark runs, which showed that install, build, test and lint output tends to carry repetitive noise while source-like output and arbitrary command results tend to carry the thing the agent actually needs next [6]. That is a bet about where information density lives in a shell session, and it was derived from traces rather than from first principles [6].
The failure that produced the allowlist is more instructive than the policy. An early version compressed `git diff`. Benchmark tasks then showed agents reopening the original output to recover what had been stripped, and the filter was removed; GitHub says early versions were aggressive enough to make the model repeat work and reduce task success [10]. The RTK evaluation had already produced the same pattern from the outside, with shorter individual responses and tasks that on average used more tokens and took longer [5]. GitHub's conclusion is that tokens per tool call is the wrong objective, and that an efficiency change has to be measured from the user's request through the final result [11].
A compressor that mostly declines to compress is an odd thing to ship, and the post says as much: it is conservative because that is what the evaluations supported, not because conservatism was the goal [12].
The instrumentation is the part I would copy first. Compressed output keeps a direct recovery path to the complete original [8], and the team tracks whether the agent opened that saved original, reran commands, repeated exploration, narrowed its searches, or took additional turns [9]. Frequent recovery means the compressor removed something valuable [9]. So the safety valve doubles as the regression metric, which is cheap to add to any harness that already logs tool calls.
What the post does not give you is a magnitude. Across the four shipped changes and the RTK evaluation, there is no percentage, no token count, and no latency figure, only direction [1]. Offline agentic benchmarks picked the candidates and controlled online experiments validated the promising ones before shipping [2], but without the deltas you cannot size the win against the maintenance cost of a command-class allowlist.
For the RTK result to transfer to your setup, three things have to hold: your model recovers the same way when text goes missing, your build and test commands are verbose in the same proportion, and rerunning a command in your harness is cheap enough that the model prefers it to reasoning around the gap. GitHub fences the claim itself, saying it applies to the integration and workloads tested rather than to every RTK configuration or to output compression in general [11]. The examples are from Copilot CLI, with the Copilot app and Copilot code review sharing the same harness [3].
The other three changes get one line each: strip formatting that adds nothing, shorten instructions without changing useful behaviour, and hand back completed background work without a second retrieval [1]. That last one is the same principle pointed at a round trip instead of a payload.
Ranked by verification strength, evidence, and original report placement.
GitHub's post examines four changes in GitHub Copilot: preserve useful context while reducing repetitive output; remove formatting that adds no value to the task; shorten instructions without changing useful behavior; deliver completed background work without an extra retrieval step.
Possible changes were evaluated offline using agentic coding benchmarks, and the most promising changes were then validated through controlled online experiments before shipping.
The examples in the post come from GitHub Copilot CLI; multiple other Copilot products, such as the GitHub Copilot app and Copilot code review, use the same underlying harness and also become more efficient through these improvements.
RTK (Rust Token Killer) is a utility that shortens shell output before an agent reads it; GitHub evaluated its effect on GitHub Copilot using its agentic coding benchmarks.
In GitHub's harness and benchmark configuration, RTK shortened some responses, but when the omitted text mattered the model sometimes reopened the original output or reran the command to recover what it needed; those recovery steps added turns and carried more context forward, and on average the task used more tokens and took longer.
Analysis of benchmark runs showed that install, build, test, and lint output often contains repetitive noise, while source-like output and arbitrary command results are more likely to contain the information an agent needs; that analysis informed a selective output compressor, informed in part by RTK and similar approaches.
Distinct publishers with included, body-backed reporting in this cluster.
1 article · September 2, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
product
GitHub will charge for Copilot seats before developers can use them1 distinct publisher
build
Grok 4.6 lands in Copilot two days after launch, and the model picker becomes a procurement problem1 distinct publisher
build
Copilot's meter changed on June 1, and half your seats are still priced in the old unit1 distinct publisher
build
Three hours of prompt training gated the 28% PR gain in a 70-engineer rollout1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Detailed, first-party, unreproducible
One post by the team that wrote the code is the entire record. It is unusually candid about method — offline benchmark screening, then a controlled online experiment — and candid about a reversal, the dropped git diff filter. But GitHub's benchmark suite and repository set are never named, the RTK comparison is a single vendor's configuration, and the only hard figures attach to the smallest of the four changes.
Shipped, scale undisclosed
These are live changes, not proposals: the compression policy and the view-tool cleanup are in the harness, the line-number removal graduated from offline runs to real Copilot CLI users, and GitHub says the same harness sits under the Copilot app and Copilot code review. What is missing is size — no user counts, no rollout dates, no share of sessions in which compression actually fires.
Underclaimed for a vendor post
The headline finding here is that GitHub's own compressor is deliberately timid because that is all its evaluations justified — not a sentence written to impress. GitHub fences the RTK result to the configuration it tested, publishes the failure that forced git diff back onto the preserve list, and reports its one clean win as a single-digit percentage. If anything the write-up undersells its most portable lesson by leaving the compressor's savings unquantified.
Author, subject, and scorekeeper
GitHub is describing its own product using its own unpublished harness, and the conclusion it reaches — judge efficiency across the whole task, not per tool call — happens to be the metric on which an integrated vendor looks best. Notice who absorbs the cost of that argument: RTK, a third-party utility, is the cautionary example, benchmarked by the platform it plugs into, with no configuration details and no reply.
Coherent but uncorroborated
The mechanism is concrete enough to be wrong in public: over-compress and the agent goes back for the original, which shows up as extra turns. Details like pulling the git diff filter are not the kind of thing teams invent for a blog post. Still, everything traces to one first-party account with a single published percentage and no external replication, which caps how far this can be trusted as a general result.