A 32GB card holds 17GB of four-bit weights with 15GB left over [2][4][1]. That remainder is the budget for KV cache, activations and whatever the runner reserves for itself, and it is what sets the context you can actually serve.
llama.cpp will allocate the full 262K window with this model on a 5090, which is the only part of that path that is quick [7]. Processing speeds at long context on the card are dire, and time-to-first-token stretches to roughly 30 minutes, which Tom's Hardware reads as something being broken rather than merely slow [7][8]. The card moves 1.8 TB/s [4]. A half-hour prefill points to something other than bandwidth; the report's verdict is that llama.cpp is the wrong model runner for this hardware right now [9].
vLLM works on one 5090 in a pinch, and charges for the load in host memory [13]. Bringing the model up the first time on a 64GB rig required another 64GB of swap, so 128GB of addressable memory to load it [11][3]. That is about seven and a half times the size of the four-bit weight file, although vLLM ran the maintainers' NVFP4 quantisation, whose size the report does not give [12][4]. The other charge is context: the published single-5090 recipe caps you at 32K, roughly 12 percent of what llama.cpp would happily reserve [13][2].
Two conditions would have to hold for either figure to describe your machine. The 32K ceiling is a property of the maintainers' base recipe for one card rather than a measured wall, and Tom's Hardware reports the limit as the recipe's without reporting an attempt to push past it with hand-written launch arguments [13]. The 30-minute number came from the latest llama.cpp build at test time, fetched and compiled from GitHub against an Unsloth quantisation [6]. Prefill paths that look broken tend to get fixed, and when that one is, the number stops describing anything.
The structural reason this model is hard on small memory pools is that it is dense. Tom's Hardware notes that mixture-of-experts models tend to be friendlier to lower-end hardware such as the DGX Spark and Strix Halo, because a small active-parameter count means less data movement during inference, which the report offers as a tendency rather than a measurement [16].
Multi-token prediction is where the memory arithmetic bites a second time. Tom's Hardware tested MTP on and off where it could, and found that not every runner could fit it in the VRAM available on some platforms; on a single card there is not enough memory to enable it at all [19][15]. The model's own throughput feature is the first casualty of the same headroom shortage that caps the window.
The report frames the stakes around agents, which remove the limits of human perception from the latency question [17]. On that framing a 30-minute prefill is disqualifying and a 32K window is a constraint on prompt design. Until one of the two runners changes, a single 5090 offers the full context length on a runner Tom's Hardware calls wrong for the card, or a supported deployment at an eighth of that window.
