The migration guide was wrong in 10 days. 48 JSON files and a playbook replaced it.

James Coombs wrote a migration guide for moving 962 files off Material UI onto a custom design system, and within 10 days four of the files it named had already been migrated and its counts were wrong ^[1][2]. He replaced the guide with structured data files and a handoff playbook, and reports that engineers on four other teams then migrated components without being walked through it ^[3][4].

The failure numbers are small, which is the point. The guide listed 941 files; the real count was 939 and dropping ^[5]. Notes referenced deleted files, and per-package breakdowns were stale in 3 of 12 tables, a quarter of them ^[6][7]. Coombs's worked example: the guide says 246 MUI files remain in the dashboard package, an engineer migrates three, the document still says 246 when the answer is 243, and the next person plans off the wrong number a week later ^[8]. He argues this is not a discipline problem, because asking engineers to hand-edit a prose document once per migrated file is a process that scales linearly with migration volume ^[9]. His narrower claim is the useful one: with 12 packages and active parallel work, the interval before a guide goes wrong is measured in days, not months ^[10].

The second defect was structural. The guide mixed procedures, data, and rationale in one document, so when the data changed the procedures referencing it drifted, and when the procedures were updated the rationale still described the old approach ^[11]. The early tooling had the same shape, and when the JSON schema changed the embedded prose instructions drifted while the tutorial kept describing old CLI flags ^[12].

The replacement is 48 data files, roughly 6,900 lines of JSON, driving all automation ^[13]: 12 component mapping files, 18 rules and checks, and 18 configuration and status files ^[14][15][16], averaging about 144 lines each ^[17]. Adding a mapping is a data change rather than a code change ^[18]. phases.json defines a 7-phase rollout with package order, T-shirt complexity sizing per package, and current status per file ^[19]. The load-bearing property is that every number appearing in human-readable output is computed from the JSON rather than typed into it, with `pnpm migration:status` counting files in each state and printing the totals ^[20]. Coombs is explicit that 6,900 lines of JSON is not simpler than 200 lines of prose, only more correct, since a wrong mapping fails a test while a wrong prose guide passes every check except a careful reader ^[21][22].

He also reports where this did not hold. The JSON was meant not to drift because the migration bot is its single writer, but the status file's last write was a human pull request with a six-month-old timestamp, which he calls the same failure one layer down ^[23]. The schema half works: tests validate confidence scoring, state machine transitions, plan validation, and rule validation ^[24]. Worth noting that the summary of his own post says 49 data files while the body says 48 and the three categories total 48 ^[25][26]. Small, and exactly the class of hand-maintained count the architecture exists to remove.

The outcome evidence is thin and self-reported: four teams migrating unassisted, and zero legacy imports in the migrated codebase ^[3][4], with no timeframe, rework rate, or reviewer count attached.

Watch whether the status file ever gets an actual single writer, since that is the one file that broke. Watch whether the four teams' migrations needed follow-up fixes, which is the real test of whether generated data replaced the guide or just replaced the reading of it.