Once a per-developer token count is visible, it stops describing behavior and starts producing it. That is the part of the lines-of-code episode worth carrying forward: the metric did not merely fail to capture quality, it paid for verbosity, and redundant or needlessly complex code outscored clean code [4]. Token totals have the same shape. Consumption is set by how many prompts go out and how much context rides with each one [2], and both rise when a request is scoped badly and the model is made to reconstruct the same background across a drifting conversation [5]. Hold the delivered result constant and the ranking inverts, with the noisier workflow on top [1].
The metric is available in the first place because somebody else needed a unit. Token pricing is a rational way for a provider to sell compute; the failure sits on the buyer's side, lifting a supplier's billing unit into an internal productivity scorecard [7]. A review cycle built on that unit rewards precisely the quantity the vendor invoices [2].
The devops.com argument is not the lazy one that token spend is waste. Extended reasoning, deeper agentic trajectories and multi-attempt scaffolding all consume tokens, and on hard problems they meaningfully improve outcomes, including finding more vulnerabilities [8]. Equally, the same spend across two configurations, models or prompting strategies can land nowhere, with inefficient scaffolding and poor context management driving consumption without proportional gain [9]. The raw count therefore moves in the same direction in both cases and cannot tell them apart. The separation only appears when tokens sit in a denominator under something that was delivered [3].
That is the useful half, and it arrives with one worked instrument: BountyBench, which benchmarks models against real-world bug bounty programs and reports cost-per-finding across models and configurations [10]. The internal version the piece recommends is remediation value per token spent, measured as whether AI use is materially reducing security debt, accelerating patching or improving remediation quality [11], with the objective restated as minimizing the cost of a real finding rather than maximizing usage [12]. All of that depends on a countable outcome validated by somebody other than the person claiming credit. Security has one. Most engineering work does not, and that gap is where the measurement problem actually sits [4].
The window for choosing a denominator is narrower than it looks. The environments now arriving have developers coordinating specialized agents for coding, testing, analysis, documentation and validation, and in those, token consumption is as disconnected from business value as CPU utilization is from software quality [6]. A scorecard built on consumption today will be scoring harness design tomorrow, under an engineer's name.
