A test-migration harness gates every agent commit behind a real Playwright run

Ground truth is why this design can skip an evaluation framework. The old test states the behavior to preserve, the running application can confirm whether the migrated test works, the assertions can be checked for coverage, and the test runner emits an authoritative pass/fail ^[4]. All of that is checkable without a second model's opinion on whether the output looks good ^[5]. The author states the goal as a Playwright test that demonstrably preserves the behavior of the legacy one ^[20].

The scaffolding sits in the per-phase context. In the original harness each phase had its own Markdown job card, plan.md, implement.md, verify.md, review.md and close.md, and each card gave the agent one responsibility, the allowed tools, the expected output and the rules relevant to that phase ^[11]. The fresh contexts are deliberate: the verifier inspects the implementer's files and evidence instead of inheriting the implementer's reasoning ^[12]. Those five cards are five prompt assemblies, and each has to stay current as the codebase moves ^[16].

That loop assumes a terminal coding agent that can inspect a repository, edit files, run commands and use tools; the author names Claude Code and Codex, and says he would not use a chat prompt to migrate a large suite today ^[17]. The underlying rule is credited to an Anthropic engineer: do not make the human continually prompt the agent, define the task and build a system capable of prompting itself as the work unfolds ^[13]. Nobody sits beside the run sending "now verify that" or "now fix the selector" ^[14]. The author wrote, "A model may propose a migration. It never gets to declare that migration complete."^[6]

For the pattern to carry to another suite, the per-test cursor has to be real. The harness selects one independently verifiable test per unit of work ^[3]. A suite that logs in once and then depends on the order of the tests that follow lacks that cursor, so the verifiable unit slides back up to the file. File-level delegation is what the author tried first, and it made failures ambiguous, retries wasteful and progress fragile ^[10]. The application under test also has to be reachable, since the agent explores the live app when the legacy test is underspecified ^[1].

The evidence record has the same dependency. A hash pins the report that Playwright produced; the assertion mapping is the part that claims the report covered what the old test specified, and the old test is the behavioral specification here ^[7][15]. The post does not report how many tests the harness migrated, how often a first attempt failed verification, or what the runs cost ^[18]. The original harness ran a Ruby Watir suite to Playwright in a private codebase, and the public reference repository recreates the pattern on a bundled Protractor-to-Playwright example ^[8]. The author writes that "The framework names differ; the design problem does not."^[19]