Databricks bets the cheapest AI coding saving is a routing decision, not a downgrade

Databricks has moved a feature called Smart Routing into beta in its Unity AI Gateway, which classifies each coding task and selects a model for it instead of leaving that decision to the developer ^[1][2]. The argument beneath the launch matters more than the feature: the company says the largest available saving in AI coding is matching tasks to models, and that hard spend caps buy savings by suppressing output ^[3].

The behavioural premise is the load-bearing part. Databricks counts 33 new models released in 2026 alone ^[4], and says that with that many models and harnesses on offer, coding agent users hit choice overload and respond by setting the most capable model at the highest effort level and moving on ^[5]. Its earlier benchmarking against its own codebase found that models cluster into capability tiers, and that much everyday work, such as flipping a flag, a single-file edit or a well-scoped bug fix, did not require the most expensive models ^[6]. The default is therefore over-provisioned by habit rather than by need. Databricks says using lower-cost models alone can save more than 50 percent, but calls doing that manually daunting for users ^[7].

The reported results are self-graded. On internal coding workloads, Databricks says Smart Routing outperformed any single model at 65 percent of the cost per task of a leading model such as Opus 5 ^[8], which is the same 35 percent saving it reports against its internal benchmark ^[9][14]. That benchmark is its own and, by its account, no labs have had access to it ^[9]. On public coding benchmarks it reports 56 percent cost savings ^[10] and says it matched Opus 5 on performance at less than half the cost ^[11].

Mechanically, the router runs a cheaper, low-latency model over the task description and labels a handful of semantic fields: which part of the system changes, what code evidence the prompt carries (a snippet, a traceback, or nothing explicit), how it appears to be failing, how localised the fix looks, and what kind of project it belongs to, from which it derives a task-type family and a language family ^[12]. It then defaults to a medium-sized model and escalates or delegates down from there ^[13]. Databricks says it deliberately kept the extractor small because using a frontier model would tax every request, including the simple ones the system exists to save money on ^[15]. It also chose task-aware routing specifically to preserve cache efficiency ^[16]. Smart Routing runs inside Claude Code and Codex ^[17], and Databricks says its Omnigent meta-harness extends the same policy across coding harnesses as well as models ^[18].

Set against that, The New Stack argues the fastest way to reduce the cost of an AI application is often not switching to a cheaper model but asking why the model is processing so many tokens in the first place ^[19], and points to repeated system prompts, unbounded conversation history, oversized retrieval pipelines, verbose tool outputs fed back whole, and redundant sequential model calls as the usual sources of waste ^[20]. It also notes that architectural optimisation frequently delivers more than migrating to a smaller model ^[21], and that more context can dilute relevant evidence and increase hallucinations ^[22]. Routing and token hygiene are different levers; the cache-efficiency constraint suggests Databricks is aware of the second one ^[16].

What to watch: whether the savings survive contact with codebases that are not Databricks' own, given that the router currently applies a single policy uniformly to every task ^[23], and what the design partners Databricks references report ^[24]. Also worth watching is the misroute cost. A task sent down a tier and then rerun is paid for twice, and the material does not report a retry or misroute rate.