person
Software developer who maintains the CauterRule agent-rule extraction project and publishes engineering write-ups on dev.to.
No evidence-backed relationships are recorded.
Debashish Ghosal got HivePlane's agent certification to repeat across three runs by swapping in deterministic agents and a mocked judge. The stable verdict covers the control plane's plumbing, and whether an LLM-driven agent answers correctly is now outside the certificate.
Reality
A dev.to field test of 157 planning traces argues teams are hardening tools, memory and orchestration while single-pass decomposition ships plans missing one ordering constraint.
Reality
agent-tooltrust ran 206 live model calls instead of 2,490 and reports identical coverage, on the argument that the only thing a real LLM adds is proof that each framework adapter can surface all four verdicts.
Reality
A two-model code review produced rebuttals and a clean verdict while the raw logs showed no changed positions and no new evidence. The rebuild enforces independence by withholding each verdict until both are committed.
Reality
The expensive part of software work has moved from writing code to proving it. An engineer building agent systems puts the deny outside the model where no prompt can reach it, and says plainly which defects his deterministic checks still miss.
Reality
One developer's 170-goal sweep sorts almost every planning blocker into three families a linter can name, two of them plain graph properties. Upgrading the planner to GPT-4o left the same pattern behind.
Reality
Precision 1.00 looks like a rule that works, until you notice the candidate fired on exactly one trajectory out of 210 and matched because "step_1" is a substring of every step identifier in the corpus.
Reality
Across 4,150 calls and four analyzer sizes, every proposal landed in the same corner of the prompt, and the edit the failure data pointed at never got written. Search strategy sets the ceiling here.
Reality
A permutation gate rejected an edit that fixed four tasks and broke one, then rejected weaker edits after the corpus grew to 40, because detection depends on how many tasks an edit moves rather than how many you own.
Reality
AgentSelfEdit's promotion gate compared the p-value against the confidence level instead of alpha, widening its acceptance window nineteenfold. Its author reports 31 fixes in one session, nine of which had been faking a working system.
Reality
Measured five times on the same plan, one safety critic returned five different verdicts and still never approved a seeded defect. It holds because the code downstream never reads the model's severity label.
Reality
It shipped inside a commit whose message promised the opposite, an automated reviewer found it in under four minutes, and the repair now keeps the list of deciding fields frozen in the consumer instead.
Reality
Debashish Ghosal built two LLM verification systems on opposite bets, code and structure, and both did what their designs promised. The blind spot they shared sat in whichever layer each one had decided to trust.
Reality
PlannerCritic's author tried to inject his own engine. The blocks arrived as feasibility verdicts rather than safety strings, which is a design worth copying and a limit worth reading closely.
Reality