Routing's Real Reference Number Is 45 Billion Tokens A Day, Not 40% Of Queries

The figure to write down from AT&T's routing disclosure is 45 billion tokens a day, the average volume its gateway has to place ^[1]. A share-of-queries number for open models is not in the supplied material: neither AT&T's post nor the LiteLLM router documentation states what fraction of traffic is served by open weights, so a 40% split is unverified here ^[15].

What is documented is the mechanism. AT&T's AI Gateway uses cache-aware routing to match each task to the cheapest adequate model, and it can change models partway through a multi-turn session ^[2]. At each turn it weighs speed and cost against expected output quality before dispatching the prompt ^[3]. The company's stated result is AI cost reduction of as much as 90%, already saving millions ^[4], on the premise that only a small percentage of its tasks need frontier-class models ^[5].

The open-model figures AT&T does give are training inputs, not serving shares: OTel 2.0 was post-trained on over 400 billion tokens on AMD GPUs ^[6], inside a program that processed over 1 trillion tokens for training via Managed Compute on Microsoft Foundry, using GSMA data ^[7].

For anyone building the same thing, LiteLLM's complexity router shows what the decision actually turns on. It sorts requests into SIMPLE, MEDIUM, COMPLEX and REASONING tiers ^[8]; the default heuristic scorer scores seven dimensions with zero API calls in sub-millisecond time ^[9], with tier boundaries at 0.15, 0.35 and 0.60 and token thresholds of 15 and 400 ^[11], and two or more reasoning markers force the REASONING tier regardless of score ^[10]. Pools can be Thompson-sampled when adaptive routing is on ^[13]. Notably, session affinity, the setting that pins a session to its first-turn model to preserve prompt cache, ships off by default ^[12] - the opposite of the cache-aware behaviour AT&T claims as its cost lever ^[2].

Worth watching: whether AT&T ever publishes the tier histogram behind the 90% claim ^[4], and how much of the 45 billion daily tokens OTel 2.0 absorbs ^[1][6].