Glean turns token cost into a scoreboard number. Buyers should insist on running the test

Per-query token cost is a ratio, and a ratio is only as honest as both of its inputs. Glean's figure comes from Glean's own testing ^[3], and the writeup names no task set and no model pairing, and does not say who did the preferring behind the 3.6 number ^[4]. That matters because Glean itself ships an auto-routing control letting administrators aim work at efficient, balanced or frontier models according to the price-performance tradeoff they want ^[8]. A test with an efficient model on one side and a frontier model on the other would produce a wide token gap and say nothing about context architecture.

The architecture claim is separable, and testable on its own. Glean's stated mechanism is that keeping permission-aware company context available to the model avoids spending tokens on repeated retrieval calls that rebuild that context at runtime ^[5]. LegalZoom's David Lee, quoted in the announcement, said recreating the same depth of context on other platforms would take a huge number of tokens ^[11]. That is the vendor thesis in a customer's mouth with no figure attached, which is how most context-cost arguments currently arrive.

There is also a denominator problem. Tau is meant to plan multistep work, carry it out for the user and check its own output ^[6], and Glean pitches it as a reduction in what it calls botsitting, the steering and file-hunting a person does around a model ^[7]. Work that used to be supervised step by step becomes work an agent runs alone, and agents running alone issue more queries. A 5.2x advantage per query and a 5.2x smaller invoice are different claims ^[3], and only the second one shows up in a finance report. Glean has at least shipped the instrumentation for telling them apart: centralized usage visibility and administrative controls over AI spending are generally available, while the workspace they would measure is not ^[9][12].

Glean's chief product officer, Emrecan Dogan, argues that models are increasingly interchangeable while understanding of a given enterprise is not ^[14]. Taken seriously, that is an invitation to hold the model constant and vary only the context layer, which is precisely the experiment the published number does not describe.

The reproducible version is not expensive to specify: the buyer's own corpus, a fixed task list, comparable model tiers on both sides, tokens counted per completed task rather than per query. Glean chose to compete on cost in public ^[3], which is also an agreement to be audited on it. The seven unreleased items in this announcement, Tau included ^[13], are the part of the pitch no audit can reach yet.