A leaderboard row is arithmetic over dimension scores. That is the only reason the format carries any authority: anyone can redo the sum. When the sum does not reproduce, the row has stopped being a claim about the vendor and become a claim about the harness. On the recomputation published in the dev.to piece, Bench'd's 80.0 sits over dimensions averaging 26.7, roughly a factor of three out [3][20], and its 60.0 sits over dimensions that are all zero [3]. The 100.0 appears beside a reliability score of 4.0, which the author reads as the fingerprint of an adapter echoing expected answers back to the scorer [c3b]. A design that averages a content score with a reliability score, rather than letting reliability veto the total, will rank that echo first every time.
The next thing you reach for is the manifest, and none of the three leaders has one anyone can download [6]. The receipt is next: the methodology page says "You don't need to trust us. Every receipt can be independently verified," and hands you a curl against benchd.dev [7]. That host answers NXDOMAIN from 1.1.1.1, from 8.8.8.8, and from the author's own resolver, while benchd.ai resolves normally [8]. The independent-verification path terminates at a hostname nobody stood up, and establishing that took four seconds [c8b].
The third fallback is signatures. The trust page defines authenticity as a signature from a key published on that page, and the key that signed Bench'd's shipped manifests matches neither published fingerprint [9].
The most useful figure on the board is the one Bench'd put there as a sanity check. It runs GPT-4o-mini with no memory attached as a control, and on 2026-08-29 that control finished third in the Conversational Memory track [14]. Two ranked memory systems are therefore indistinguishable from having no memory, on the axis the track is named after.
The author's own reporting is the format worth copying: RE-call at 69.0 LongMemEval and 71.6 LoCoMo, unmodified scoring path, $6.64 of spend, on 2026-08-23 [17], published next to a no-memory control of 57.6 [18]. Subtract and you get 11.4 points [19]. The rank does not transfer to your workload; the delta might, if your traffic resembles LongMemEval's, and that is at least a question you can put to it.
The upstream defect is more mundane. One project's published 96.6% recall came back as 87.0% when rescored with the benchmark's own scorer, because it reported recall_any where the spec says recall_all [15]. That is 9.6 points from one word in a field name [21]. The ground truth underneath is not clean either: per the audit by @dial481 that the piece cites, LoCoMo carries roughly 99 wrong or misattributed answers, and a judge accepted about 63% of deliberately wrong responses [16].
So the test for a row in any of these tables is a manifest you can fetch, dimensions that reproduce the total, and a control in the same table. Bench'd's top three clear neither of the first two, and where its control does appear it beats two of the systems ranked above it [14].
