Skip to content

Build1 publisher3 min readPublished

Inception's 1,107 tokens per second needs a batch size before it enters your capacity plan

Mercury 2.5's diffusion decoder refines many token positions at once, which is why its headline throughput moves with batch size, output length and the reasoning setting Inception ships as an adjustable knob.

The Engineer · Build desk

Photograph accompanying Inception's 1,107 tokens per second needs a batch size before it enters your capacity plan
Photo: techcrunch.com

What happened

  • Inception launched Mercury 2.5 on September 8th, saying its diffusion language model generates 1,107 tokens per second on widely available Nvidia GPUs.
  • The API ships a 260,000-token context window, adjustable reasoning, parallel tool calls and schema-aligned JSON output.
  • Inception also reports a 40% intelligence gain over Mercury 2 without attaching the figure to a clearly named independent benchmark in the launch prose.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint Without a batch size and an output length, 1,107 tokens per second cannot go into a capacity plan, because it sets no floor for what your own traffic will see.
  • decision Anyone costing an agent loop at $0.15 per million output tokens is choosing to budget against a rate that steps up five-fold whenever the discount lapses.
  • contradiction Inception's own co-founder has published on the quality-speed trade-off, yet this release quantifies the speed side to four digits and leaves the quality gain without a named benchmark.
  • capability If parallel denoising holds up under production traffic, loops that call a model dozens of times per answer get a serving option that does not pay for output length one token at a time.

Diffusion decoding does not commit to token one before producing token two. It starts from a rough output and refines many positions at once over repeated denoising steps [6]. Throughput therefore scales with how many positions get refined per step and how many steps run, which is the information a tokens-per-second headline leaves out.

Inception's own material names the variables: batch size, output length, hardware configuration, reasoning-token accounting and network overhead all move a tokens-per-second result, and the release does not disclose enough of them to compare 1,107 cleanly against rival claims [9]. One of those variables is also a product feature. Mercury 2.5 ships adjustable reasoning [2], so the same endpoint can return different tokens-per-second figures depending on where a caller sets that knob and whether reasoning tokens land in the numerator.

For 1,107 to transfer to your queue, several conditions have to hold together: your batch size matches the measured one, your output lengths match, your GPU is the same widely available Nvidia part [1], your reasoning setting matches, and you measure server-side with network time excluded. Miss on batch size alone and the number describes someone else's demo.

The price is checkable now. List is $0.20 per million input tokens and $0.75 per million output [3], and the 80 percent launch discount puts those at $0.04 and $0.15 [4]. The discount preserves the shape: output costs 3.75 times input at both list and discounted rates [13]. Filling the 260,000-token context once [2] costs $0.052 at list and about $0.010 discounted [16]. Sustained generation at the claimed rate for an hour is roughly 4 million output tokens, which bills about $0.60 discounted and about $2.99 at list [17].

That gap is the budgeting problem. List is five times the discounted rate on both meters [14], and Inception has not said when the discount ends [5]. A loop costed at $0.15 per million output tokens is a loop costed against an unannounced expiry.

The quality side is thinner than the speed side. Inception reports a 40 percent increase in "intelligence" over Mercury 2, and the launch prose does not attach that figure to a clearly named independent benchmark [8]. Kuleshov's published guide to diffusion language models covers the quality-speed trade-off directly [12], which is the trade a serving architecture of this kind has to negotiate. Both halves are load-bearing for the pitch. Only one half arrives with a number, and that number arrives without its denominator.

None of this makes the engineering weak. Commercializing a decoding architecture is harder work than wrapping another provider's model in application software [20], and Ermon left the purely academic route because proving the architecture at commercial scale needed more compute, engineering and capital than a university group could supply [11]. The $50 million seed led by Menlo Ventures in November 2025, with Nvidia's NVentures among the participants [10], buys the GPU access that proof requires.

What decides adoption for a router that calls a model dozens of times before returning one answer [7] is tokens per second at your batch size and your reasoning setting. That figure is not derivable from what Inception has disclosed, so the only number in this launch a buyer can verify independently is the discounted price, whose end date is unannounced [5].

What to watch

  • An announced end date for the 80% launch discount, or a change to the $0.20 and $0.75 list rates.
  • Disclosure of the batch size, output length and GPU model behind 1,107 tokens per second, or an independent measurement of it.
  • A named benchmark behind the claimed 40% intelligence gain over Mercury 2.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories