Build1 publisher3 min readPublished
31 of 70 injected defects survived Prismwar's passing gate suite
DHSeaDev's write-up on the browser card game Prismwar shows a replay check passing ten of ten matches because the recorder and the playback player both skipped the AI's defence step, so the round trip compared a bug against a copy of itself.
The Engineer · Build desk
What happened
- A mutation campaign on the browser card game Prismwar injected 70 deliberate defects into isolated lines, and 31 of them made the game fail silently while the gate suite reported everything was fine.
- The replay harness omitted the AI's defence step in the playback player and the scripted human player omitted it in the recorder, so ten recorded matches replayed identically through the same bug.
- Among the silent failures, the integrity check exempted unsigned saves, so a hand-written profile with no sig field imported as integrity:'ok' and granted 720 cards and a million wins.
- The rebuild routes the state machine through newGame(), the single place that refuses non-array decks, decks under 10 cards and missing players, with downstream code assuming the precondition holds.
- Non-observable paths are now pinned with a SHA-256 fingerprint over the outcomes of a 512-game corpus. That works because the AI engine has zero randomness once seeded.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint When the recorder and the player call the same function, the fidelity result is settled before the test runs: a record/replay comparison can only fail on a defect that sits on one side of it.
- exposure Omitting a field is the cheapest thing a forger tries first, and an integrity check with an exemption branch is bypassed by omitting the field the exemption tests for.
- cost Pinning a hash over a deterministic corpus moves the cost of every intentional balance change into review, where someone has to choose between bumping the baseline and reverting the diff.
- decision Whether 44 percent of your own injected defects would survive depends on whether your assertions check that a named defence ran or only that nothing crashed. The figure is a prompt to classify your assertions.
Delete the lower-bound check inside `validateDeck` and the gate suite still passes. A 4-card deck loads, then `checkWin` throws an uncaught error the moment someone attacks, and the error boundary turns the throw into a "Something went wrong" message. The assertion "no console errors" stayed green [10].
The blocker defect ran the other way. An attacker declared blockers, the code did not store which ones, and the match went on as if nothing had been blocked; replays of that match still reproduced it correctly, because the whole engine carried the same defect [11].
Fixing the replay test meant giving up the comparison. The team drove the AI's defence window directly and asserted that the match phase advances, which surfaced the omission immediately [8]. "A round trip cannot test its own driver. Neither can it test its engine," DHSeaDev wrote [9]. The rule the post draws from it: on a system with two paths, encode/decode, server/client, record/replay, do not let both paths share the same code on the test side [7]. The same move replaced a turn-count check. Instead of asking whether a game ends in 50 turns or fewer, the suite asserts that the stall detector measures progress, and deleting that measurement fails the test [20].
31 of 70 is 44 percent [21]. The other 39 made the game unplayable [3], and the two groups account for every mutation in the campaign [22], so the defects this suite caught were the ones that stopped the game running. The 70 were carefully crafted at chosen sites [1]. So 44 percent is not a mutation score you can line up against a tool's generated population. For the survival rate to say anything about your own suite, your assertions have to be the class theirs were. If your gate already asserts that a named defence ran, your number will be lower. According to the post, the team stopped treating "no crash" and "match outcomes are deterministic" as evidence [13].
The cheapest habit in the write-up is checking that the harness can fail at all. Before running the mutations, the team proved the harness could see a break, by installing an attacker key and verifying that it unlocked a legendary card it should not have [17]. When they later swapped the stall detector for a board-count check, they deleted the code the assertion depended on and watched the test fail [18]. Their loop is four steps [19]:
1. Write the defect. 2. Run the test to prove it catches the defect. 3. Remove the defence that catches it, and confirm the test fails. 4. Keep the defect in the suite as proof the test was running.
The post does not report how long the 70-mutation campaign took [23].
What to watch
- Whether the 70 mutations are published as a set, so other teams can run the same defects against their own harnesses.
- Whether the pinned 512-game hash survives the first real balance patch or gets bumped without anyone checking why it moved.
- A second campaign against the rebuilt suite would show how many of the 31 silent survivors the entry guards and property assertions now catch.