Build1 publisher3 min readPublished
GLM 5.3's low effort setting misses five of 33 tasks its default gets right
Z.ai's GLM 5.3 rejects every way of switching thinking off and defaults to max effort, where 33 of 33 checkable tasks came back right at $0.00468 per correct answer against $0.01173 from GLM 5.2.
The Engineer · Build desk
What happened
- GLM 5.3 and GLM 5.3 Flash return HTTP 400 with error code 1210 for thinking: disabled, enable_thinking: false, and reasoning_effort of none, minimal, medium or xhigh; omitting the parameter gives max.
- Across 11 checkable tasks run three times each, GLM 5.3 at max got 33 of 33 right at $0.00468 per correct answer, against $0.01173 for GLM 5.2 at the same setting.
- GLM 5.3 accepts the thinking_budget parameter and ignores it: every value from 0 to 1,024 produced about 101 reasoning tokens on the same question.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision Anyone who ran GLM 5.2 with thinking disabled as the cheap path has to choose a reasoning floor and pair it with a way of catching the errors that low leaves in place.
- constraint Per-request spend can no longer be capped by a thinking token budget, so the only levers left are the effort tier and max_tokens.
- contradiction DeepSeek V4.1 Flash scored 33 of 33 at every effort setting in the same batch, so effort governing accuracy is a property of this model and not of reasoning parameters in general.
- exposure Callers that rely on strict json_schema enforcement get it ignored on both GLM 5.3 builds, leaving json_object as the structured-output path that returned correct values.
Per correct answer, low is still the cheaper setting. GLM 5.3 at low costs 63% less per correct answer than the default [8], which puts it near $0.0017 against $0.00468 [1]. Low missed five of 33. At low, GLM 5.3 returned 216 twice for a grid-path count that is 132 once one cell is blocked, and it missed the coin combinations, the knapsack and the base conversion once each [11]. Those four failure modes account for the entire gap between 28 and 33 [7].
The saving comes out of token volume. Input is still $1.40 per million tokens and output $4.40, unchanged from GLM 5.2 [1], and Z.ai's guide says GLM 5.3 "uses the same base model as GLM-5.2, with all improvements driven by post-training" [17]. Reasoning tokens are billed as output [20]. GLM 5.3's median run was 538 reasoning tokens against GLM 5.2's 1,129, with a longest run of 7,733 against 21,917 [7]. Roughly halving the median thinking is what takes $0.01173 per correct answer down to $0.00468 [6][4].
The test set was 11 problems with one checkable number as the answer, among them sums of primes, grid paths, a knapsack and the remainder of 7 to the power 222 divided by 1000, each run three times [24][5]. The accuracy figures therefore rest on 33 runs, so one flipped run moves them by about three points [2]. Cost per correct answer is total spend divided by correct answers at list price [23], and that division only exists where you can grade the output. On work you cannot grade, low buys the 63% cut and the 15% error rate goes unobserved [5].
GLM 5.3 Flash reached 31 of 33 at a tenth of the price [10], and at max GLM 5.3 costs 3.4x DeepSeek V4.1 Flash per correct answer while GLM 5.3 Flash costs under a third of it [14]. That puts GLM 5.3 at max above ten times the per-answer cost of Z.ai's own Flash model [3].
GLM 5.3 accepts thinking_budget and ignores it: every value from 0 to 1,024 produced about 101 reasoning tokens on the same question [15], and those tokens are billed [20]. The disabling paths at least fail loudly, with HTTP 400 and error 1210, "this model always thinks, turning thinking off is not supported; use low, high or max" [2][3].
The same batch asked five questions about invented entities, including a share price for Verantis Dynamics and the melting point of Oridium-7, where the only correct answer is "I do not know", and forcing thinking on did not fix the fabrication: the write-up's verdict is "No, even with thinking forced on." [25]
The recommendation in the write-up is to keep the default for anything with arithmetic or several steps and use low only where a wrong answer is cheap to catch [22]. In my view that is right for any pipeline that cannot check its own output, and losing the thinking-off path costs little: with thinking off, GLM 5.2 got 10 of the same 33 right [21].
What to watch
- Whether the per-correct-answer ranking holds on the agent and coding work Z.ai's model card points to, which this batch did not test.
- Whether Z.ai reinstates a medium tier or a no-thinking mode, both of which currently return error 1210.
- Whether strict json_schema starts being honoured, since json_object is the structured-output path that worked.