Time per output token comes out of a division [10]. The numerator is the bytes the card has to read to run one pass: the full weights, 16 GB in the post's example, plus the notes and the word just produced [9][8]. The denominator is how fast the card can read its own memory [11]. Core count does not appear anywhere in that expression [18]. Fit a card with more arithmetic and the same memory read rate, and each decode lap lands at the same moment it did before, which is exactly the case the dev.to post flags as a faster GPU making no difference at all [16].
The toy example makes the volume visible. "The capital is Paris" is four laps [14]. Each lap moves the whole 16 GB of weights out of VRAM and past the cores, which glance at it and do a small amount of maths [9]. Four laps at 16 GB is 64 GB of reads to write four words [15]. Most of the card is watching.
Prefill runs the other way round. The whole prompt goes through in one pass with thousands of cores firing together [3], and the length of the pause is set by how much text you handed over: six words vanishes, a hundred-page contract is tens of thousands of tokens and the pause is real [6]. That makes a published TTFT a statement about somebody's prompt length. For the number to transfer to you, your prompt token count has to match theirs; otherwise the table has told you about their document [19].
The reason each decode lap stays cheap is the notes. Every word produced is appended to the KV cache, so lap three does not re-read your prompt plus the first two words, and if it did, every single word would carry its own pause [13]. Those notes are written into the same VRAM that already holds the weights [4][20]. So a long context costs capacity that the weights have claimed, and it adds bytes that every subsequent lap has to read [8].
That is the whole argument for reading a card as two specs rather than one. In my context, mostly long inputs and short structured outputs, prefill dominates the wall clock and arithmetic throughput is the thing I would pay for. Flip the traffic to short prompts and long answers and the same purchase buys idle cores, because the loop is then bounded by memory reads for its entire duration [11]. The ratio of prompt tokens to output tokens in your own logs decides which of those two situations you are in, and no spec sheet knows it.
