Build1 distinct publisher3 min readPublished
Debashish Ghosal built two LLM verification systems on opposite bets, code and structure, and both did what their designs promised. The blind spot they shared sat in whichever layer each one had decided to trust.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
Here is what actually happens when PlannerCritic's critic flags a problem. The critic returns a finding with a severity label. The system checks that finding's family against a frozenset holding `unsafe_sequencing`, `weak_rollback`, `unverified_dependencies` and `feasibility` [4]. If the family is outside the set, a `blocker` is downgraded to a warning [5]. The frozenset is the safety policy. The model's severity field is a suggestion box [5].
That is good engineering, and the sweep counts read like it working [11]. Add the three structural families and you get 497 blockers over 183 goals, about 2.7 per goal [19]. For that number to tell you anything about your own planner, you would need his goal corpus, his plan schema and his gate code at v0.2.2 [10]. It is a property of that sweep, not a property of plans.
The asymmetry is in the instrumentation. The critic is the component under suspicion, so it gets meters: `label_flip_rate`, `evidence_drift_rate` [14]. The gates are the component being trusted, so they got build-time tests and no runtime monitoring [16]. The safety contract lives in the gates [2]. So the contract sits in the one layer with no live signal [14].
A reader named Artjoms Stukans put the consequence in the comments: "If one blocker class stops firing after some refactor, your numbers only look better. 226 becomes 40 and that reads like plans got safer" [12]. That is a drop of 186, an 82 percent reduction [20], and it is entirely welcome news as long as you assume the gate ran.
The second system already had the answer to that shape of question. The convergence score is an aggregate, and it deliberately measures movement rather than agreement, so two models that agree without engaging score low [9]. The capitulation detector does not read the aggregate at all. It reads the transcript and asks whether one side conceded everything in round one without a single rebuttal [15]. That is a check on the record of the process rather than on its summary statistic, and it is the part of this work I would steal first.
Ghosal's port back into PlannerCritic is the Gate Canary: ten fixture pairs, one per gate class, each with a known-good plan [17]. The source text stops mid-sentence there, so the other half of each pair is a guess I am not going to make.
The narrower lesson is the one worth keeping. Both projects bet that the right structure produces reliability, and by the author's own account the structure held [18]. Structure decides which component is allowed to be wrong. It does not produce evidence that the trusted component is still doing its job [14][16]. If you concentrate a contract in one layer, the probe has to sit in that layer too, and it has to fail loudly when the layer goes quiet.
Ranked by verification strength, evidence, and original report placement.
The convergence score measures movement rather than agreement: two models that agree without engaging get a low score, and two that genuinely challenge each other get a high one.
AdversarialDebate's capitulation detector reads each debate transcript for the shape of failure instead of trusting the aggregate convergence score, asking whether one side conceded everything in round one without a single rebuttal.
The author's fix is the Gate Canary: ten fixture pairs, one per gate class, each with a known-good plan; the supplied source text is truncated mid-sentence at this point.
The author says he thought the answer was architecture and was half right: the architecture worked and the systems are reliable, but both had the same blind spot, which took months to see because it made the metrics look better rather than worse.
Debashish Ghosal, writing on dev.to, says he built two systems this year that attack the same problem from opposite directions: PlannerCritic and AdversarialDebate.
PlannerCritic puts an LLM critic in a loop with a deterministic gate layer; the gates own the safety contract and the critic is advisory.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 29, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
The refusal came back labelled "infeasible": reading a planner/critic jailbreak test1 distinct publisher
build
SSE in Go breaks twice before your handler runs: an illegal header, then a 30-second timeout1 distinct publisher
build
An empty array is a claim about your query: verify identifiers before you trust the metric1 distinct publisher
build
Rate limit your MCP servers, because a retrying agent turns one error into a billing incident1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
One builder's own instruments
Every figure that matters — 226 blockers over 183 goals, 0.982 convergence over 41 debates, 65% capitulations — was produced and reported by the same person, on dev.to, with no reproduction path published. The specificity is genuine and falsifiable: named finding families, a named CLI command, an exit code, version numbers. But falsifiable is not verified, and the argument's sharpest evidence is an anecdote in the comments about a Kubernetes ReplicaSet that nobody can check either.
Two hobby repos and one stranger
Both projects are at 0.2.x, both benchmarks were run by their author, and the only person outside the project who appears anywhere in this story is Artjoms Stukans in the comments — whose remark, notably, changed the software. No installs, no third-party deployment, no users cited. A shipped v0.2.3 is real movement; it is not uptake.
The rhetoric argues against its own numbers
Rare direction of travel: the piece exists to discredit its author's best-looking metrics, retiring a 0.982 convergence result as "almost useless for adversarial review" and treating a healthy blocker count as a thing that could mean the opposite. That earns the slightly-understated read. The one unpaid claim is the quiet aside that both systems are reliable — asserted on the strength of the same instruments the essay has just shown can flatter a broken layer.
Author, subject and measurer are one person
Ghosal writes about Ghosal's projects on a developer platform that publishes what he submits, and he is the sole source of every count in the story — the classic setup for a favourable ledger. What pulls the reading down is the content: he spends the piece admitting a blind spot he carried for months and retiring his own best result, and the remedy he promotes costs nothing to check. There is no product to sell here and no third party paying for the framing.
Sound reasoning, unaudited numbers, cut text
The mechanism is easy to believe on logic alone — a control that stops rejecting things produces fewer rejections — and needs no trust in anyone's data. Trusting the specific figures is a different matter, since one unaudited harness produced all of them, and the text available to us breaks off mid-word on the debate-side fix, leaving that remedy named but undescribed.