Nvidia puts a decode chip in the rack, and inference planning gets a second variable

Nvidia's own comparison puts the part four times ahead of rival platforms on latency-sensitive work ^[7], which sets the baseline it beat at roughly 850 tokens per second ^[12]. That is a checkable number. The other claim in the same announcement, that multistep agentic tasks finish in minutes instead of hours ^[8], is not: a 4x factor applied to a three-hour job leaves 45 minutes ^[13]. Either the tasks in question sat close to an hour rather than six, or the saving comes from something other than decode throughput. The 4x is the figure worth holding Nvidia to.

The second gap is that the source never says whether the measured rate is one stream or the whole rack, and that distinction is the product. Per stream it works out to about 0.29 milliseconds a token, so a 10,000-token reasoning chain clears in roughly three seconds ^[15], which is the number an agent loop budget is built from. Spread across the accelerators a full rack can hold, the same figure would be about 13 tokens per second each ^[14], and nobody would publish that as a record. Read it as single-stream, then ask what the batch size and the concurrency were.

Then the planning problem, which is the part that lands on operators this quarter. The rack pairs dozens of Vera Rubin GPUs doing context ingestion with a much larger population of decode parts ^[4][5], and neither the intended ratio for a given workload mix nor whether a buyer can vary it is stated. An agent that reads a 100,000-token context and emits 200 tokens has a different balance than one that plans in long chains from a short prompt. Guess wrong and you have idle silicon on one side of the interconnect. Huang's phrasing, workload-optimized AI factory configurations for the era of agentic AI ^[10], is honest about what is being sold: a configuration, not a chip. Configurations have to be sized, and the sizing evidence here is one model at one context length ^[6]. Worth noting too that the announcement's own text calls the rack's accelerators LP30 while the product is Groq 3 LPX ^[17], so treat the configuration detail as provisional.

The clearest fact in the release is the $20 billion paid in December for Groq's technology, along with founder Jonathan Ross and president Sunny Madra ^[9]. Nvidia decided that decode-specialised silicon was worth buying rather than waiting to build, and took the people who had already shipped it. Nebius CTO Danila Shtan makes the same argument from the buyer's side, that generation is the phase that decides how responsive a system feels ^[11].

What is absent: price, power draw, and how much of this Nebius is installing ^[16]. Nebius says it will run the chips in its Token Factory platform ^[3]. Until it publishes a rate per million output tokens, the responsiveness claim is a specification rather than an operating cost.