Bartowski broke tensors one at a time to find where GGUF bits belong

The solver is the piece that changes what an operator does. It reads the model's shape, the measured sensitivity priors, GGML block-size information and a target quantization type, then writes a `--tensor-type-file` for llama.cpp, assigning a type to every quantizable tensor instead of accepting the runtime's built-in heuristic ^[13]. The generator, the solver and the frozen sensitivity data are published in Bartowski's quantization configuration repository ^[14].

Maintenance is why it exists. Mixture-of-experts models with more than eight experts brought small `shexp` tensors that proved unusually sensitive, and Bartowski had been carrying a modified `llama-quant.cpp` to handle them ^[4]. New families and competing layout methods convinced him another pile of special cases would become the same problem ^[4].

Bartowski said he ran roughly 96 hours of experiments on a Framework Desktop, producing and evaluating more than 1,000 quantizations, with the test scripts built with Claude Code ^[5]. That is four days ^[18], and about ten quantizations an hour, one every six minutes ^[19]. The priors ship frozen ^[14]. Any team that wants the ranking remeasured for its own model family pays for those four days itself.

The central run degraded one tensor at a time: nearly everything stored at `q8_0`, one tensor crushed to `q2_k`, and the resulting change as the measure of that tensor's importance ^[6]. Scoring was KL divergence against the bf16 reference, computed with `llama-perplexity` on wikitext-2-raw at a context length of 512, where lower KLD means the compressed model stayed closer to the reference ^[9]. The whole ranking rests on that setup. The sweeps used Qwen3.5 at 0.8B and 4B, with the patterns then checked against Gemma 4, IBM's Granite 4.2, MiniCPM5, Ling, Muse and DeepSeek-V2-Lite ^[8]. If your workload is code at long context, the hierarchy is a prior you inherited.

Token embeddings dominated the scale, at about eight times the damage of the worst individual weight tensor in the 0.8B model and 16 times in the 4B ^[10]. Parameters grew fivefold between those two models while the ratio doubled ^[20]. Early and late layers followed the familiar U-shaped curve, and attention value and output projections, feed-forward up projections and state-space-model output tensors also ranked highly ^[11]. Feed-forward gate tensors generally did not earn extra bits ^[11]. Embeddings do not have to stay at `q8_0` either: Bartowski found `q4_k` restored much of the lost performance, leaving the solver room to spend bits elsewhere when the target file is small ^[12].

He also tried the inverse, upgrading one tensor inside an otherwise low-precision model, and said those results contained too much noise to drive the method ^[7].

What the material does not contain is a head-to-head at fixed file size: layout map against llama.cpp's heuristic, KLD reported for both. Runtimewire frames the payoff as a possibility, writing that better bit allocation could preserve more model behavior at the same memory limit, though broad independent benchmarks are still needed ^[17]. The sensitivity ranking is the measured part, and it has not been independently reproduced ^[16].

For anyone pulling quants, the checkable part is the suffix. Under the new scheme `_S` means at least 90% of a model's body tensors remain at the named quantization type ^[15], so the filename asserts something you can test against the tensor list in the file you downloaded. The scheme sets a lower floor for `_M` ^[21].