Liquid AI's 279.5M drafter turns a 3.13x decode gain into 2.62x end to end

Liquid AI says the vision drafter can use the same process as its text-model drafters because image patches and text tokens have both been projected into a shared representation by the time they reach the layers it reads ^[4]. Granted that, the drafter never has to look at pixels. It reads hidden states from selected target layers, proposes a block of candidate tokens, and the target model checks the whole block together. Tokens that pass are emitted without the target generating each one serially ^[3].

Liquid trained at block size nine and recommends eight or nine at inference depending on hardware ^[5][6], so one verification pass retires at most nine tokens. The drafter costs something. At 8.9% of the target's parameters ^[5], nine drafter steps cost roughly 0.8 of a target pass if time tracks parameter count, putting a block cycle at about 1.8 target passes ^[3]. A 2.30x to 3.13x decode speedup then implies something like 4.1 to 5.6 accepted tokens per block ^[4]. That number is an estimate built on a bandwidth-proportional cost assumption, not a measured acceptance rate. (The 8.9% also implies a target of about 3.14 billion parameters, so the 3B in the name is rounded down ^[5].)

The gap between the decode figures and the end-to-end figures can be solved for a time split. Take the M5 Max pair and assume the 3.13x decode number and the 2.62x end-to-end number came from the same run. Decode then accounted for about 91% of baseline latency, since 0.91/3.13 + 0.09 = 0.38, and 1/0.38 is 2.62 ^[1]. Run the same solve on the llama.cpp floor, 1.57x decoding against 1.30x end to end. Decode falls to about 64% of baseline, leaving roughly a third of the wall clock in image encoding, prefill and overhead ^[2]. Liquid reports ranges over six test categories and does not say which category produced which number ^[14], so the two splits bracket the range.

The 3.13x came from an Apple M5 Max running MLX-VLM ^[8]; the same model on an M3 Ultra under llama.cpp tops out at 2.14x decoding and 1.77x end to end ^[9]. Getting the top number means matching the platform, the precision and the workload. Long generations help, because every token after the first is decode. Multi-turn conversation was one of the six categories ^[14]. Screen understanding and object grounding, two of the tasks the base model was built for ^[13], can answer in a few dozen tokens over a full-resolution image. Those runs spend most of their time in the steps the drafter does not touch ^[11].

Liquid's earlier LFM2.5 checkpoints were tuned for 4-bit deployment ^[15], and quantizing both models changes the cost ratio the estimate above depends on. Liquid did not test quantized deployments ^[12], and on device that is the case that matters. Training and ablations ran on AMD hardware, across 10 epochs of vision-language supervised fine-tuning data weighted toward expected use cases ^[7], while the published speedups come from Apple and Nvidia parts ^[8][10].