Uber's AI cost per session fell 18 points further than its cost per request

Take the two Uber percentages together. Cost per request is down 34%, so a request costs 0.66 of what it did. Cost per session is down 52%, so a session costs 0.48. Divide the second by the first: the average session now contains about 27% fewer requests than it did ^[1]. Requests per session is a property of the harness and the prompt loop. The inference holds only if the mix of work inside a session stayed roughly constant. If Uber's developers also changed what they ask for, part of that 27% is changed behaviour ^[1].

Two of the reported levers point straight at that ratio. Uber triggers automatic compaction above 400K tokens, even for models with 1M context windows ^[8], and it caches prompts when work runs through its own harness, Minions ^[9]. A third is a default: Uber found that defaulting to Medium effort gives the best cost-to-output ratio with advanced models ^[7]. Subagents get cheaper models because their tasks do not need the most expensive ones ^[6]. Compaction at 400K on a 1M-context model is a decision that the remaining 600K tokens are not worth their price ^[5].

The substitution saving is a range. The Pragmatic Engineer puts open models at 2-20x less than frontier ones ^[11], which is between 50% and 5% of frontier cost ^[2]. Pinterest's published figure sits near the cheap end. On last month's earnings call, CEO William Ready said "With open models, we are achieving cost per transaction at less than 8% of the cost of comparable closed proprietary models" ^[12]. Under 8% is better than 12x cheaper ^[3]. Ready also stated the condition attached to it: "When we leverage open source models, such as with Pinterest Assistant, we are seeing superior performance for our use cases when compared to closed third-party models because we are able to post-train open models on our highly unique data" ^[13]. To get 8%, you need the unique data, an environment to post-train in, and a use case narrow enough to post-train for.

AT&T, with 100,000 employees, cut its AI bill by 56% while measuring a 2% decrease in output quality after moving workloads to open models ^[15][16]. The substitution is partial. According to The Information, AT&T's software developers still rely on cutting-edge models for complex tasks, and Austin said he has found that open source models are "just as good or better" than older models sold by the likes of Anthropic and OpenAI ^[18][17].

The May list of approaches companies were trying included per-developer monthly AI usage limits ^[1]. That lever does not appear among the ones reported for Uber. What the Uber account supports is narrower: cost per unit down by a third to a half, and total spend flat since March while token use and session counts kept rising ^[4][5]. Uber has published an engineering blog post detailing dozens of optimizations to reduce token cost without noticeable change in the quality of code its agents generate ^[10].