Loading today’s stories
Build1 distinct publisher3 min readPublished
Selection here turned on two things no leaderboard reports: whether a model flips Yang Wood to Yin Wood in English output, and whether it lets you switch reasoning off. Price only broke the remaining ties.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
The 5.8x is the figure that will travel, so take it apart first. The published price map lists the flagship at 2.50 in and 7.50 out per million tokens, against the mid-tier's 0.40 and 1.60 [12]. Input is 6.25x [1]. Output is 4.69x [2]. A single per-call ratio of 5.8 sits between those two, and the mix that produces it is recoverable: solve for the input share and roughly 91% of billed tokens are going up, not coming back [3]. That fits a chart app, where a computed chart and a long system prompt go in and a few hundred words come out [1]. Run a chattier workload, output-heavy, and the same two rows give you something nearer 4.7x. The ratio is a fact about the author's prompt as much as about the two models.
The exclusion was decided by a flag, not by that ratio. The preview snapshot rejects the request to disable thinking outright, with a 400 saying the parameter is restricted to True [5]. On one streamed request the author logged 2.1 seconds to the first reasoning token and 10.5 seconds to the first character a reader could see, with 286 reasoning events in between [6]. That is 8.4 seconds [4], about 34 events per second [5], of billed output that never reaches the page. The author reports the resulting prose was indistinguishable from the mid-tier's [c7b], and the reasoning tokens bill on top of the list-price gap [7].
The routing file is where the eval stops being an opinion. Failures are classified into three moves: 401 and 403 are fatal, because a different model will not fix your key; 5xx retries the same route; a 400 or 404 goes to the next route only when the body matches model, not-found, or InvalidParameter, and is otherwise fatal [15]. Notice that the flagship's own rejection string sits inside that regex, so a hard parameter error walks down the chain instead of surfacing to the caller [5][15]. The 429 branch is the one the author flags as the trap: throttling should retry the same route, quota exhaustion should move on, and the only way to tell them apart is sniffing the response body, which is provider-specific [16].
None of the ranking transfers, because no candidate is named anywhere in the writeup; every model appears as a role label [17]. What transfers is the ordering of the gates. An element flip is categorically wrong, the way a compiler that flips a bit is wrong [4], and a closed vocabulary means invented jargon is a liability rather than a style note [3]. The flagship still sits at the end of the paid chain [13]: excluded from the workhorse slot, kept for the request nothing else will serve. And because a route is an endpoint plus a key plus a model [14], the chain's last entry is the primary model on a backup account, since an exhausted balance kills every model on that key at once [14]. One loose end in the config as printed: the free chain points at a route called legacy-plus that has no row in the price map [18], which is the kind of drift an eval file collects between planning and launch.
Ranked by verification strength, evidence, and original report placement.
Before launch the author benchmarked eight candidate LLMs on the app's actual workload, then built the routing layer around what the benchmark found.
On one streamed request to the flagship the author recorded 286 reasoning events before the first character of the answer, 2.1s to the first reasoning token and 10.5s to the first character a user can read.
On list price the flagship costs 5.8x the mid-tier model per call, and the reasoning tokens bill on top of that.
The author could not tell the flagship's prose apart from the mid-tier's, and excluded the flagship.
Two well-known open-weight models had quietly been delisted from the provider's international endpoint between planning and testing, which the author frames as a model choice being a dependency with an EOL you do not control.
The app generates personalized readings for BaZi (Chinese "Four Pillars") birth charts, and every reading is an LLM call.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 28, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
Retry budgets are sized against outages, not jitter: 45 seconds lost all sixteen clips1 distinct publisher
build
Allow-list the closed set, block-list the open one: 193 thin geo pages, one gate1 distinct publisher
build
A GAN beauty filter is a device budget allocation, not a feature toggle1 distinct publisher
build
Agent memory rots by accumulation, and the missing primitive is a supersession key1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
The arithmetic closes; the sourcing can't be checked
Run the published price table against the stated call shape and the meter readings dev.to prints fall out to the cent — $0.0021 paid, $0.0005 free, and a 5.8x flagship multiple that only works because roughly 90% of billed tokens are input. That internal consistency is rare and it earns trust in the numbers. What it cannot do is substitute for identification: with no vendor, model version, corpus size or pass rate given, the rankings are testimony, and the write-up contradicts itself once, calling the previous-generation fallback cheaper than a model its own table prices below it.
One app's routing table, plus its meter
The footprint is a single production app by a single developer, and dev.to is the only place any of it appears. What lifts this above a blog sketch is that the routing layer is described as live and instrumented — usage events per call, cost attributed to the model that actually served rather than the one intended — and the cost figures are presented as meter readings, not estimates. No other team, deployment or replication turns up anywhere in our coverage.
The link overpromises what the writing delivers
The prose is disciplined — exclusions are justified, the cost multiple is arithmetic rather than rhetoric, and the author volunteers that the winning model was chosen partly by taste. The overshoot lives at the edges: the link dev.to still publishes this under advertises "16x the cost for nothing" against a text that says 5.8x, and "the results generalize to any LLM in a niche domain" is a large claim built on one corpus, one grader and one streamed latency sample. Anonymising every model also flatters the finding, since nobody can bring a counter-benchmark to it.
Selling the method, not a model
The author is writing about their own commercial product on a personal developer blog, and the free tier they describe as marketing spend is part of that product — so this works as credibility-building whether or not it was meant to. Cutting the other way: no vendor, provider or model is named, which removes almost every reason to shade a benchmark, and no sponsorship or provider relationship is disclosed or implied. The incentive here is reputational rather than transactional.
Trust the method, hold the ranking loosely
We are confident about what this is: a well-instrumented account whose numbers survive being recalculated. We are much less confident about what it concludes, because a single self-published source with anonymised candidates, one contradicted price claim and an unpriced route in the shipped config cannot be checked from outside. Method: solid enough to copy. Verdicts on specific models: unverifiable by construction.