Invest1 publisher3 min readPublished
A $14.34 router matched Opus-5's score on LiteLLM's 21-task benchmark
LiteLLM's own Terminal-Bench run puts a gpt-5.4-mini classifier routing across Haiku, Sonnet and Opus at $14.34 against Opus-5's $19.74 for the same 16 solved tasks. The saving works out at 26 cents a task.
The Investor · Invest desk

What happened
- A router with a gpt-5.4-mini classifier reading only the current user message solved 16 of 21 Terminal-Bench 2.0 tasks for $14.34, the same count Claude Opus-5 solved for $19.74.
- Giving a 4o-mini classifier the last three user messages instead of one lifted its solve rate from 66.7% to 76.2% and its bill from $12.86 to $18.58, a 44% cost increase for 14% relative quality.
- The heuristic router, which makes no classifier call at all, reached 87% of Opus quality at 56% of the cost and had the lowest median wall clock of any of the eight arms.
Compiled by The InvestorSomething wrong?How this is made
Why it matters
- constraint At 26 cents saved a task, a buyer needs roughly 3.9 million tasks a year for routing to be worth $1m, so this is a lever for gateways and large agent fleets and close to noise for a team running a few thousand jobs.
- decision Anyone budgeting classifier spend now has a priced comparison inside one run: paying for a better classifier model got the same 16 solved tasks for $4.24 less than paying for a wider classifier context window.
- exposure The only figures on the table belong to the vendor selling the router, published on its own documentation site with a design-partner solicitation attached, so procurement teams modelling savings are modelling one party's spend logs.
The saving is $5.40 [1]. Spread across 21 tasks that is about 26 cents a task [2], and the classifier call that produced it cost at most $0.34, which is 2.4% of the winning arm's bill [8][11]. Per solved task, the router paid $0.90 and Opus-5 on its own paid $1.23 [4]. Turning 26 cents a task into $1m of annual saving takes about 3.9 million tasks [3].
The context result is narrower than it sounds. All three context arms use one 4o-mini classifier and one tier map, so the only variable is how much of the conversation the classifier reads before it picks a tier [15]; the answering model's context window is not what moved. Reading the last three user messages instead of one raised Opus's share of requests from 34% to 45%, and LiteLLM attributes most of the extra cost to that escalation [9]. The bill went from $12.86 to $18.58 [3]. Swapping in gpt-5.4-mini on a single message bought the same 16 tasks for $14.34, or $4.24 less [6]. Median time per task rose from 6.8 minutes to 8.3 [4]. The arm that also read assistant replies sent 5% of requests to Haiku and scored worst of the three [6].
The arm I would look at hardest has no classifier in it. Its 87% of Opus quality at 56% of the cost [7] works out to roughly $11.05 and about 14 of 21 tasks [7][8], which is $0.79 per solved task, cheaper than the routed or the unrouted Opus figure [9]. The two extra tasks the gpt-5.4-mini router picks up cost $3.29 between them, $1.65 each [10]. LiteLLM reports cost as a per-arm total across all 21 tasks, taken from its gateway spend logs [14].
The scoring is soft at this size. One task out of 21 moves a solve rate by 4.8 points [12], and errors ran between zero and four per arm, excluded from the denominator only where a task did not complete for that arm [11]. Every tier is Anthropic: Haiku for SIMPLE, Sonnet for MEDIUM, Opus for COMPLEX and REASONING, with Sonnet as the fallback [10]. All eight arms ran the same 21 tasks with prompt caching on, which is what LiteLLM means by like for like [12][1], and the company published the run on its own documentation site alongside a call for design partners [13].
The router cannot beat Opus, because its top tier is Opus, reached on between 34% and 45% of requests [9]. So what is on offer is a price effect on a fixed ceiling. The claim most likely to break is the one about the single message: Terminal-Bench tasks arrive as self-contained specifications, so one user turn tells a good classifier what it is dealing with, and on a workload where the first message does not describe the job, the arm that paid $18.58 for three turns of context is the one that should win [3][15].
What to watch
- Third-party numbers posted to LiteLLM's discussion #32168 by teams running the router on their own production traffic.
- Any price change across claude-haiku-4-5, claude-sonnet-5 and claude-opus-5, since the whole saving is the spread between those three tiers.
- Whether LiteLLM publishes task-level results per arm, so the failures of the heuristic and classifier routers can be compared directly.