FORGE's simulator passed for real agents because both emit the same events

Live view and replay in FORGE are the same code. Replay simply stops reading the log partway through ^[4]. Both views are one computation over the events, applied either to all of them or to a prefix, so the two cannot drift apart. I would make the same call on day one.

The same property let the simulator pass for the real thing. Real and fake agents emit the same events ^[5], so no screen could tell a simulated run from a live one. The wrong-question run was the design working as built. The simulator was the default for most workflows, and nothing on screen said a run had used it ^[14].

The rule, the author wrote, is to "never ship a placeholder that looks real. Label it or remove it" ^[15]. Simulated data is now marked everywhere ^[16]. The simulator survives only as a "Dry run" button in the workflow designer, for checking the wiring for free, and the New Run screen cannot start a simulated run at all ^[16]. In my view the label also belongs in the events themselves, as a field on every event a run writes. A screen written next year would then show it without anyone having to remember.

A dry run would not have caught the first live failures. According to the post, they came in ways the simulator never could ^[8]. Every model call timed out instantly. The server has no IPv6, and when a hostname has several addresses, Node.js tries them one after another with 250 milliseconds per attempt by default ^[9]. The two-line fix preferred IPv4 and allowed three seconds per attempt ^[9]. That is twelve times the default ^[19].

Empty replies came from a model that spent its whole output budget on reasoning. It now gets one retry with double the room ^[10]. The researcher kept searching until it hit the step limit and ended with no notes. Tool results now report how many rounds are left, and the last round forces a write-up ^[11]. Page fetches give up after 20 seconds, and a host that times out once is skipped for the rest of the run ^[12].

The whole thing runs on a small server in the author's house ^[1]. Most agents use GLM-5.3 Flash through OpenRouter at about $0.15 per million input tokens, and the reviewer runs on Claude ^[6]. Search goes through the ddgs Python package for DuckDuckGo, which is free and needs no key ^[7]. A Node.js server runs the agents, stores every event and streams them to the browser ^[7].

Merging each browser's old local data into the server's database surfaced a subtler bug. Run IDs had been picked by each browser ^[18]. The text of the post breaks off before it describes the bug. Every event in the log is attached to a run ID, so I'd expect ID assignment to move to the server that now owns the log.