Build1 publisher3 min readPublished
A visual regression percentage counts changed area, so a badge that reflowed everything beneath it scored far larger than the edit that caused it. Its author keeps the pipeline out of CI for related reasons.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
The percentage comes out of the comparator, not out of the design. diff_report.py compares PNGs with Pillow, draws boxes around the pixels that differ, and writes an HTML report plus summary.json [12]. Nothing on that path can separate an intended edit from a consequence of one. Padding on the badge changed its footprint, the rows below it moved, and the comparator counted all of it [2][3].
That is what makes a diff threshold unportable between projects. For a percentage measured on one codebase to mean the same thing on another, edits would have to stay inside their own region, the fixtures would have to be fixed, and the baseline would have to have been accepted under the same rendering conditions [4][8]. Reordering subscription rows violates the first of those by definition, so the number moves with the layout's coupling rather than with the count of mistakes.
The assertion is narrow by construction: given specified inputs and rendering conditions, how does the current static appearance differ from the appearance previously accepted [8]. According to the dev.to walkthrough it does not show that a purchase succeeds, that a button responds, or that a price description is correct, and an unchanged screenshot can sit on top of broken business logic [9]. A screenshot suite is the wrong instrument for asking whether the purchase went through, and it will answer anyway.
The constraint that shapes the whole design is that simctl offers no state control. capture.py launches the app with arguments such as -VRScreen paywall, -VRLang ja and -VRStyle dark [15], and those are application-defined: simctl forwards them and never reads -VRLang as a localization instruction [16]. So the harness owns the sequence end to end, in this order: parse the request, prepare fixed data, select the language, synchronize the theme, construct the controller, display the window [17]. Select the language after the labels exist and the screen renders with old text [18]. It renders that way every run, which is the part that matters, because run.sh's baseline step will accept it as the reference [13].
Coverage identity follows from the same problem. The case ran four languages, two appearance modes and four screens for 32 screenshot states [5]; divide by the four screens and each screen name stands for eight distinct renderings [7]. That is why the author treats language and appearance as part of a state's identity rather than metadata [6].
The toolchain bill is modest: no extra Swift testing library inside the app, and Xcode with simulator support, Python and Pillow on the capture host [19]. The per-project bill is the harness, the screen builders, the language and theme wiring, and the fixed test data [20]. Point-Free's swift-snapshot-testing already covers view controllers, device configurations and trait collections [23]; the choice here was an in-app entry point driven by an external script [24]. The author's own limit on the work, useful locally but not yet an unattended gate [21], is the honest read of the evidence, and the missing checks are legible from the mechanism: verify that the requested language and appearance were actually applied before the shutter fires.
Ranked by verification strength, evidence, and original report placement.
While updating ShotZen's paywall, the author reordered subscription options, emphasized a recommendation badge, and added a NO RENEWALS label to the lifetime option.
The screenshot report showed changes beyond the areas the author intended to edit: the annual subscription row, call-to-action button and footer had moved too.
The author's reading of visual regression reports: the percentage measures changed area, while the surrounding context explains whether the change is acceptable.
The case used four languages, two appearance modes and four screens: 32 screenshot states.
A screen name alone is not a sufficient coverage unit; language and appearance belong in its identity.
The assertion is narrow: under specified inputs and rendering conditions, how does the current static appearance differ from the appearance previously accepted?
Follow any of these and your For You feed starts watching them — no settings page required.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
One first-hand account, mechanically checkable in parts
Everything traces to one developer writing about his own app. The commands, file responsibilities and the SceneDelegate hook are the sort of thing a reader can confirm against Apple's simulator tooling in an afternoon, which is why we treat the architecture as solid. The findings that give the story its point are weaker: the footer moving, the badge swap shrinking the diff and the 3.49% figure all rest on screenshots only the author has seen. His willingness to state what pixel comparison cannot prove lifts the credibility of what he does assert.
One app, kept off the build server by choice
ShotZen is the only place this runs. It runs on a developer's machine, not a build server, because its author says the capture checks are not ready to be trusted unattended. There is no repository, package or second project named, so the 32-state sweep is the whole of the observed usage.
Pitched below what it demonstrates
The writeup argues its own case down. It volunteers that an unchanged screenshot can sit on top of broken purchase logic, declines the CI gate, and calls its interfaces naming conventions rather than a framework. A working 32-state pipeline and a correctly diagnosed reflow bug would support a warmer description than the author gives them.
The author's own paywall is the exhibit
One interested party appears, and it is the author, whose subscription app supplies both the tooling and the bug; dev.to publishes what contributors submit, without editorial verification. Pulling the other way, the post names Point-Free's library as a capable alternative and lists where its own capture is fragile, a candor that a typical pitch tends to skip on its way to closing.
Reproducible mechanics, unverified numbers
A reader with Xcode, Python and Pillow could rebuild most of what is described from the post alone, so the how-it-works layer holds up. The percentages, the causal story about the padded badge and the claim that plain text calmed the diff come from one person's screenshots with nothing in our coverage checking them, which caps how far this can be trusted.
build
SWE-Gate flunks 221 of 644 test-passing agent patches on rules mined from PR comments1 publisher
build
Screenshot retrieval gets a one-line install, and the comparison is text embeddings1 publisher
build
Delete one strong_release and Swift leaks 145 MB: ARC as a compile-time contract1 publisher
build
Pricing the three routes to shipping iOS without a Mac, honestly1 publisher
Publishers with included, body-backed reporting in this cluster.
1 article · September 7, 2026