One early test puts a 120B model at 63 tokens per second on an RTX Spark laptop

The throughput figures come from a single DEV Community roundup, and it relays them secondhand. The post does not say which labs ran the tests, what quantization or runtime they used, or which memory configuration each laptop carried. Neither it nor NVIDIA's launch page includes an API price to compare against ^[2].

Start with decode speed. The 35B figure runs from 58 to 89 tokens per second, so the fast end is about 1.5 times the slow end ^[15]. The 120B result of 63 tokens per second ^[12] falls inside that range, though the model is about 3.4 times larger ^[16]. We'd expect decode speed to track the weights read per token. A 3.4x size gap with near-equal speed points to a mixture-of-experts model or heavier quantization on the larger one.

For the 63 to transfer to your workload, you need the same model at the same quantization and a prompt about as short as the test's.

Prompt processing is the other half. Microsoft's own benchmark, a vendor figure, shows 170 seconds before the first word on a 128K-token prompt ^[5]. If 128K means 128,000 tokens, that implies about 750 tokens per second of prompt processing ^[10]. The 170 seconds and the 63 tokens per second may come from different models, so we have not combined them.

NVIDIA's own launch page sends readers to Artificial Analysis for independent testing of local AI performance on RTX Spark ^[6]. The five review excerpts the page quotes are descriptive, and none states a tokens-per-second figure ^[8]. TrustedReviews, UK wrote that "the performance absolutely stunned me" ^[7].