Liquid AI's LFM2.5-24B-A2B outscores Qwen3.6-35B-A3B on LiveClaw in Liquid's own evaluation
Liquid AI says its LFM2.5-24B-A2B scores 48.09 on LiveClaw against 38.19 for the larger Qwen3.6-35B-A3B. The 184 tok/s it reports on an Apple M5 Max is a decode rate, so agent latency depends on how many tokens the model spends thinking.
The Engineer · Build desk
Key figures: LFM2.5-24B-A2B scores 48.09 on LiveClaw and Qwen3.6-35B-A3B scores 38.19, in Liquid AI's evaluation. Liquid says Qwen3.6-35B-A3B has about 50% more total and active parameters than LFM2.5-24B-A2B.
| Figure | Value | Note | Claim |
|---|---|---|---|
| LFM2.5-24B-A2B LiveClaw score, per Liquid | 48.09 score | 5 | |
| Qwen3.6-35B-A3B, LiveClaw | 38.19 score | 5 | |
| Qwen3.6 parameters | 50% more, about | 7 |
What happened
- Larger models keep the edge on knowledge benchmarks such as GPQA Diamond and HLE, and on coding benchmarks such as SWE-Bench Verified (OpenCode) and SciCode.
- On a Ryzen AI Max+ 395 the model decodes 114 tokens/s, which Liquid says is up to 1.7x the next-best compared MoE while using the least memory.
- It ships with day-one support in llama.cpp, MLX, vLLM, SGLang and ONNX.
Why it matters
- cost Every answer starts with thinking tokens: 1,000 of them take about 5.4 seconds on the M5 Max and about 8.8 seconds on the Ryzen chip, before any tool call returns.
- constraint Knowledge-heavy and coding work stays with larger models, so an on-device agent needs retrieval or search in its loop, and each lookup adds another round of thinking.
- decision Anyone weighing a swap from a Qwen-class model should time their own agent loop end to end, which the day-one runtime support makes cheap to do.
The LiveClaw gap is 9.9 points, or 1.26 times the Qwen score [15]. It is the figure Liquid puts in its summary bullets [5]. Liquid ran the evaluations itself: four models, 16 benchmarks, served with vLLM at each benchmark's recommended generation parameters [8]. It describes Qwen3.6-35B-A3B as having about 50% more total and active parameters [7]. Leading or matching on five of eight tool-use and agentic benchmarks leaves three where it does not [13]. On AutomationBench, Liquid says, it scores more than three times the next-best model [2]. A ratio that size depends heavily on how low the next-best score is.
Sparsity explains the speed, according to Liquid: with only 2B parameters active per token, each reasoning token is cheap [10]. That cuts compute per token but leaves the full weight set to load. Each MoE block routes a token to 4 of 64 experts [4], and different tokens can pick different experts, so all 24B parameters [12] have to be resident. In bf16, the dtype in Liquid's quickstart [14], that is 48 GB of weights before any KV cache [17]. The post does not say what precision the 184 tok/s run used [1], and its claim of the least memory among the compared models comes without a figure [18].
The quickstart sets temperature to 0.1 and max_new_tokens to 512 [14]. The model always writes a chain of thought before it answers [10], so the 512 cap would cover the thinking as well as the answer. We'd expect any agent harness to raise it.
In our view the evidence supports two vendor-run results: a lead on tool-use benchmarks and a decode rate on two chips [1][18]. Liquid trained the model with agentic reinforcement learning inside harnesses such as OpenClaw, Hermes Agent, Pi and OpenCode [11]. Larger models still lead on SWE-Bench Verified (OpenCode) [9], so training in that harness did not produce a lead there. For the LiveClaw gap to transfer to your workload, your tasks and tool schemas would have to resemble the benchmark's.
What to watch
- Third-party runs of LiveClaw, AutomationBench and BFCLv4 on LFM2.5-24B-A2B, which would test whether Liquid's own numbers hold.
- Liquid disclosing the precision, prompt length and context behind the 184 tok/s figure, plus a memory number for the llama.cpp and MLX builds.
- End-to-end task times for the model inside Hermes Agent or OpenCode on the M5 Max, with thinking tokens counted.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence40
- Adoption
- Insufficient
- Hype gap+30
- Incentives70
- Confidence58
Perspective Coverage
3 publishers- Builder
- Builder 50%
- Operator
- Operator 25%
- Investor
- Investor 25%
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
LFM2.5-24B-A2B decodes 184 tokens/s on an Apple M5 Max, according to Liquid.
- [2]
Liquid says LFM2.5-24B-A2B leads or matches the competition on five of eight tool-use and agentic benchmarks (AutomationBench, BFCLv4, ToolSandbox, LiveClaw and tau3-Bench Banking), scores more than three times the next-best model on AutomationBench, and leads all three instruction-following benchmarks.
- [3]
The context window grows from 32K to 131K tokens compared with the predecessor LFM2-24B-A2B.
- [4]
The model keeps its predecessor's architecture: 40 layers, 64 experts per MoE block with top-4 routing, and a hybrid of gated short convolutions and GQA.
- [5]
Liquid AI's post says LFM2.5-24B-A2B scores 48.09 on LiveClaw versus 38.19 for Qwen3.6-35B-A3B, and lists this as a summary bullet.
- [6]
The model ships with day-one support in llama.cpp, MLX, vLLM, SGLang and ONNX.
- [7]
Liquid describes Qwen3.6-35B-A3B as a model with about 50% more total and active parameters than LFM2.5-24B-A2B.
- [8]
Liquid evaluated LFM2.5-24B-A2B against Qwen3.6-35B-A3B, Gemma-4-26B-A4B-it and Nemotron-3.5-Lightning on 16 benchmarks, with all models run on vLLM using each benchmark's recommended generation parameters.
- [9]
Liquid says larger models keep an edge on knowledge benchmarks (GPQA Diamond, HLE, AA-Omniscience) and on coding benchmarks (SWE-Bench Verified (OpenCode), SciCode), and advises pairing the model with retrieval or web search for knowledge-heavy workloads.
- [10]
LFM2.5-24B-A2B is a reasoning-only model that always produces an explicit chain of thought before its final answer; Liquid says that with only 2B active parameters per token, each reasoning token is cheap.
- [11]
Post-training includes agentic reinforcement learning, which trains the model inside agent harnesses such as OpenClaw, Hermes Agent, Pi and OpenCode.
- [12]
The model has 24B total parameters and 2B active per token, as its name LFM2.5-24B-A2B indicates.
- [13]
Five of eight tool-use and agentic benchmarks led or matched leaves three where LFM2.5-24B-A2B neither leads nor matches.
- [14]
Liquid's transformers quickstart loads the model with dtype bfloat16 and generates with temperature=0.1, top_k=50, repetition_penalty=1.05 and max_new_tokens=512.
- [15]
The LiveClaw gap is 9.9 points, or about 1.26 times the Qwen3.6-35B-A3B score.
- [16]
1,000 reasoning tokens take about 5.4 seconds at 184 tok/s and about 8.8 seconds at 114 tok/s.
- [17]
At bfloat16 (2 bytes per parameter), 24B parameters occupy about 48 GB of weights, before KV cache.
- [18]
LFM2.5-24B-A2B decodes 114 tokens/s on a Ryzen AI Max+ 395, which Liquid says is up to 1.7x faster than the next-best compared MoE, while using the least memory; the post gives no memory figure.
Sources
3 independent publishers whose own reporting we read for this story.
- cryptobriefing.comLiquid AI releases LFM2.5-24B-A2B, a 24B-parameter model built to run on your laptop
1 article · October 13, 2026
- dev.toLiquid AI ships open-weight 24B agent model for local use
1 article · October 13, 2026
- huggingface.coRun fast, private agents on-device with LFM2.5-24B-A2B
1 article · October 13, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.
Topics
Entities
- Qwen3.6 35B-A3BFollow
- LFM2.5-24B-A2BFollow
- Hugging FaceFollow
- Liquid AIFollow
- LiveClawFollow