Frontier models stall in the mid-70s on the triage work Thinking Machines wants to automate

Reading was never the constraint the lab set out to remove. The post is explicit that the work sits in the small repeated judgments carried over the reading, and that those are the judgments investors get stuck articulating even though the tasks themselves are trivial to them ^[14][19]. The examples carry the weight: an article touching Greenland reads as noise given its context while one on China tariffs reads as signal, and both are geopolitics bolted to finance ^[18]. Interestingness is defined against a particular seat, the macroeconomic investor who does not care about a small IPO however financially relevant it plainly is ^[17]. A prompt can only carry the part of that an expert can spell out, which is the stated reason for going to fine-tuning instead ^[13].

Five points sounds like polish until you count it in documents: mid-70s accuracy leaves roughly one document in four labelled wrong, 80% leaves one in five, so the stretch that no amount of prompt work could cross is a fifth of the surviving errors ^[3]. The price of that fifth is annotation by high-quality human hands ^[15], and the hands belong to the same investors whose attention the system exists to free. The post does not say how many labelled documents it took, nor how many analyst hours, and it reports on a subset of data cleared for public release ^[12]. Hours in against hours out is the ratio a buyer would want before signing anything.

The cost line is where I would push hardest. GPT 5.4 priced 43% above 5.2 for a marginal accuracy gain ^[9] means that if a single point moves, 75 to 76, the cost per correctly labelled document rises about 41% ^[2], and the lab's own reading is that newer models are not improving quickly at this task, per dollar least of all ^[10].

This is probably wrong, but the more interesting version of the mid-70s plateau is that it describes the labels rather than the models. Automatic prompt optimisation added nothing on top of the expert-written instructions ^[7], which is what you would expect if the residual disagreement is inside the label set. No inter-annotator agreement figure appears, and no absolute accuracy number or price for the proprietary model either, only that it beat everything tested on information accuracy and recall at a fraction of the cost ^[4][11]. If two macro investors split on "relevant but uninteresting" one time in six, then 80% is a description of a committee and a fine-tune that clears it has learned one desk's habits.

Which may be the commercial point. Differentiated intelligence, models tuned to specific organisational needs ^[16], puts the asset in a client's labelled corpus rather than in the weights, and it means research time is going into a narrow relevance classifier fitted to one firm's taste instead of a horizontal product sold off a menu. Two readings would break that. A frontier release that clears 80 on these six tasks ^[2] from a plain prompt makes the corpus worth close to nothing, and a per-firm labelling cost small enough to disclose turns it into a cheap onboarding step rather than a moat. Until one of those numbers is published, the demonstrated finding is that judgment is still cheaper to rent than to specify.