Benchmark
MMLU-Pro is a multiple-choice benchmark covering many academic and professional subjects, a harder successor to MMLU for evaluating LLM reasoning.
VIDRAFT released a 111GB, 4-bit build of its 180B-parameter Darwin model that it says runs on a laptop with 8GB of VRAM and 32GB of RAM. Air-gapped teams still need a measured token rate on that laptop before they can price it against a server.
Reality
Researchers led by Lingjiao Chen of Stanford found the AI model with the lower list price cost more in 106 of 336 head-to-head comparisons. For anyone setting an AI budget, the finding means per-task cost has to be measured on their own workloads before a model is chosen.
Reality
VIDRAFT says POCKET-Darwin-180B, a 111 GB 4-bit GGUF build of its 180B mixture-of-experts model, runs in llama.cpp on about $1,400 of consumer hardware. The accuracy evidence so far is one MMLU-Pro comparison that VIDRAFT reports itself.
Reality
LiveNerf reruns 78 calibrated questions against Claude Opus 5.5 every day, with frozen prompts and a pinned Claude Code CLI. That gives teams building on the model a dated launch baseline to test a suspected regression against.
Reality
Moonshot AI's Kimi Linear reports 75% less KV cache and 6.3x faster decoding than MLA at 1M tokens, using three linear layers per attention layer. The speedup falls to parity at 4k tokens, so the saving goes to traffic that runs at hundreds of thousands of tokens.
Reality
A new evaluation gives models explicit rules about what may appear in their reasoning and scores whether they comply while still solving the problem. Compliance rose with model size and fell with more RL training.
Reality
AWS's open harness records $0.0021 per correct AIME answer for gpt-5.6-luna after an 80 percent Bedrock price cut. The figure depends on running luna with reasoning disabled while mini runs at its defaults.
Reality
Ai2 fit a multidimensional item response model to 100 models across 16 benchmarks. The per-question estimates say parts of its own safety suite are grading general reasoning rather than refusal behaviour.
Reality