Topic

Agentic inference

Serving pattern in which a model runs tens or hundreds of tool-using turns within one session, producing long contexts, heavy KV-cache reuse and bursts of short-lived sub-agents.

Current clusters

invest1 publisher

Rubin shows 7x tokens per megawatt and over 2x profit per gigawatt versus Blackwell, SemiAnalysis says

SemiAnalysis says its AgentX benchmark clocked Nvidia's Vera Rubin NVL72 at up to seven times Blackwell's token throughput per megawatt on pre-release software, and puts the profit gain at over two times per gigawatt.

Publishers:newsletter.semianalysis.com

Reality

Evidence46
Adoption29
Hype gap+16
Incentives67
Confidence41