d-Matrix's CTO labels Raptor's 1,000-tokens-per-second figure a simulation

"Some of our results come from real hardware, some come from modeling, and some are still targets," Bhoja wrote in answers to LDS ^[5]. He splits d-Matrix's evidence into measurements on early memory silicon, simulations of a future Raptor system, and results from the existing Corsair product ^[17]. Only the first involves Raptor's own silicon ^[1]. I think this sorting is good practice for a chip still short of tape-out. A buyer's cost model should keep the three apart.

Raptor stacks memory and compute in one package. The goal is to cut the cost of moving data while supplying the bandwidth fast token generation needs ^[7]. Divide the 0.37 picojoule figure into the 2 to 3 picojoules per bit Bhoja cites for HBM4 and you get 5.4 to 8.1 times less energy per bit moved ^[3]. That matches the company's own five-to-eight range ^[9]. For the ratio to reach the power bill intact, data movement would have to be nearly all of a rack's draw. LDS points out that compute, other memory activity and networking also consume energy ^[18].

The speed chart models a 72-card system serving Kimi K3 with a one-million-token context ^[10]. Its sizing assumes 4-bit weights and an 8-bit KV cache ^[11]. Going from a decode batch of eight to one of 32 cuts per-user speed by 21 percent ^[1]. Multiplied out, aggregate output climbs from about 7,900 to about 25,100 tokens per second, assuming every user in the batch decodes at the charted rate ^[2]. A service sold on latency cares about the first number. One sold on throughput cares about the second. For either figure to transfer, the buyer's model has to meet its quality targets at 4-bit weights and run near the charted concurrency. The simulator also has to match a chip that has not yet taped out ^[6].

The prefill-decode split adds costs that a per-user rate leaves out. Moving the KV cache from GPU to accelerator takes time and energy. Each accelerator's memory caps how many users it can serve at once ^[15]. The operator also has to size two hardware pools, and idle capacity in either one still costs money ^[16]. Bhoja's advice is to count only requests that meet speed and quality requirements, and to put idle hardware and data transfers in the bill ^[4].

The September 10 announcement puts initial availability of the NVIDIA-integrated Raptor rack in Q4 2027 ^[13]. Bhoja said d-Matrix has not measured end-to-end speed or power on a full Raptor system ^[3]. His last test for any proposal is which NVIDIA dependencies it keeps ^[4].