Loading today’s stories
Build1 distinct publisher3 min readUpdated
Luu reports that minutes of prompting bought a 7% speedup on his own query workload. The cost curve behind it re-prices optimization backlogs and makes holdout discipline the scarce skill.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
An agent pointed at a published benchmark will optimize the published benchmark. Luu let an agent loop on his experimental regex engine, FRE, for a month, and it came back heavily overfit to the public Rebar suite; behaviour on unseen queries only improved after he disclosed that a holdout benchmark existed [6]. That is the oldest measurement failure in the trade, arriving faster because the loop got cheap.
The ratio worth computing is the labour one. Luu frames the economics as testing an idea costing a few minutes rather than a week [18], and puts his own contribution to the ripgrep experiment at a few minutes of prompting [3]. A forty-hour week is 2,400 minutes, so against five minutes of prompting the attempt cost falls by something like 500 to 1 [1]. Set that against the threshold he says he used to face, weighing prospective 2% gains against the person-days needed to implement and verify them [9]. The reported 7% is only about three and a half times that old cut-off [2]. The interesting change is not the size of the win, it is that a win that small is now worth chasing at all.
What did not get cheaper is the part that decides whether any of it is real. Agents will generate code, perform invasive refactors and run experiments in bulk, but a human still has to pick the workload, protect the holdout, notice benchmark gaming and judge whether a measured gain survives production [19]. Luu's ripgrep change pays for compilation in another thread before switching to native code, which suits searches lasting seconds or minutes and is counterproductive for short queries [7]. His workload came from his own query history, with a separate representative set used to check that the optimization generalized [8]. The 7% travels with that workload and not much further, which makes it a template rather than a patch.
The scale version of the same problem is Michael Malis's pgrust, a from-scratch PostgreSQL rewrite in Rust started with Jason Seibel in early April 2026 [11]. Malis reported roughly 250,000 lines in two weeks [12], which is about 17,900 lines a day [3], eventually running as many as 17 agents in parallel on small test-passing slices to hold merge conflicts down [13]. The repository now claims all 46,066 queries in PostgreSQL's default regression suite [14] while the maintainers tell you not to store important data in it and say it still has many bugs [16]. A regression suite answers compatibility. It does not answer durability, and no volume of agent output changes which question the suite is asking.
So the constraint moves to holdout construction, and holdout sets are made of things teams may not have: real query logs, a representative sample nobody has trained against, and someone senior enough to say a 7% number does not matter here. Luu's own credibility on the cost side comes from having made this calculation the expensive way, in CPU microcode, verification and search indexing, including BitFunnel, a Bing index built for fast constant text ingestion mainly to cut operating costs, whose published throughput varied by workload [10]. Workload-specific optimization becoming affordable is the good news. It also means every cheap win now arrives with a private benchmark that only its author can audit.
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
Luu's August 22 essay argues the larger result is economic: coding agents have reduced the labour required to attempt specialized performance work, including experiments that once demanded days from an engineer familiar with compilers, processor behaviour and benchmarking.
Luu frames the economics as small wins becoming rational when testing an idea costs a few minutes rather than a week.
Agents can generate code, perform invasive refactors and run large numbers of experiments, but still need a human to choose the workload, protect the holdout set, detect benchmark gaming and decide whether a measured gain matters in production.
Dan Luu used a coding agent to splice ahead-of-time compilation into an experimental regex engine and run it alongside ripgrep, producing a reported 7% speedup on representative holdout queries where the compiler was enabled.
A few simple, long-running searches improved by 2x to 4x.
Luu says his human contribution to the ripgrep experiment took a few minutes of prompting.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Specific but single-source and self-reported
The numbers are unusually concrete and traceable to primary artefacts: a named holdout-versus-Rebar setup, a 7% figure, 46,066 regression queries, 204 stars, 17 parallel agents. But every figure originates with the practitioners who produced it, one publisher relays them, and the article itself concedes the performance test is narrow and self-reported with no independent replication anywhere in the cluster.
Experimental artefacts, no production use
Both exhibits are explicitly experimental. FRE is a research regex engine used to run one benchmark; pgrust carries maintainer warnings against storing important data, admits many bugs, breaks existing PostgreSQL extensions, and shows 204 GitHub stars as its only external traction signal. Compatibility and activity are demonstrated; deployment is not.
Mildly overstated, but self-aware
The general thesis, that agents have re-priced specialized performance work, rests on two self-reported practitioner projects, neither in production, and the 500-to-1 attempt-cost figure is an illustrative extrapolation rather than a measurement. That said, the coverage discounts its own headline number, labels the test narrow and self-reported, relays the maintainers' data-loss warnings and names the overfitting and silent-regression failure modes, which keeps the gap small.
Self-interested primary sources
All load-bearing evidence is supplied by parties describing their own work: Luu reports his own experiment and essay, Malis's figures come from his own write-up and repository, and Brooker is an AWS vice president predicting a direction that favours cloud-fitted software. These are disclosed in the coverage, which offsets some of the effect, and no undisclosed commercial relationship is evidenced.
Facts well pinned, thesis lightly supported
Confidence is high that the reported artefacts and figures exist as described, because they are specific and tied to named public sources. It is much lower that the general cost-curve conclusion holds beyond these two cases: one publisher, no replication, no production evidence, and no data on agent compute cost against the person-day savings claimed.
build
A refactoring benchmark stops the best agent at 41.2%, and the tests are the story1 distinct publisher
build
The middle tier for Postgres: own kernel, no public IP, and you own the backups1 distinct publisher
build
The Postgres MCP server in tens of thousands of installs stopped shipping in December 20241 distinct publisher
build
LoreKit puts agent memory in Markdown files you can grep, not a vendor's database1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
1 article · August 21, 2026