An AI chip insider argues inference should get cheap enough that teams stop counting tokens

A token cap starts as a finance decision and ends up as a product one. Someone writes the limit, and someone else answers on Friday for the feature that hit it.

The author of the SiliconAngle piece is explicit about whose margins the argument cuts against. "That may sound odd coming from someone in the AI semiconductor business," the author wrote ^[2]. The description of the current market is blunt: "AI is currently priced, marketed and deployed as a luxury product" ^[5]. On the fear that cheap inference shrinks the business, the piece says: "This isn't a race to the bottom. It's the dawn of a larger market" ^[11].

The price gap in the chocolate comparison is about 27 to 1, which is $27 divided by $0.99 ^[9]. That ratio describes a consumer good with a century of volume data behind it, and the piece uses it as an argument about direction ^[8]. It does not include inference price data or a date by which costs fall ^[16], and its statement that Microsoft reportedly limits AI usage arrives unsourced ^[7].

So sort the features you have capped or shelved against two questions. Does the value of the feature grow with call volume? And is per-call cost the thing actually stopping it, or is latency, accuracy or user trust the real blocker?

Volume-scaling and cost-blocked is the only quadrant where this argument changes a plan. Build those so the limit is a configuration value with an owner, and the piece's own list of candidates is ambient intelligence, autonomous systems and always-on assistants ^[10]. Volume-scaling but blocked by quality means cheaper tokens let you ship a weak feature more often. For everything that does not scale with volume, falling cost turns up as margin on what already ships; the piece argues each efficiency gain reduces operating costs for applications already in production ^[15].

Generated lines of code, requests per second and benchmark scores are useful engineering metrics and not business outcomes, according to the piece ^[12]. The product-side version of that mistake is calls per user, which tells you what the cap is doing and nothing about whether anyone got value. Two numbers do answer it: whether the people who hit the ceiling came back the following week, and how long a new user waits for one useful result.

Until a vendor publishes pricing a buyer can plan two years against, the cap stays. The choice available now is whether it sits in a config file with a named owner and a review date, or inside the design where nobody can move it.