Google's PageBreak agent confirmed two XSS flaws on hardened frameworks against more than 500 elsewhere

More than 500 against two is a ratio of better than 250 to one ^[1], and it compares raw counts of bugs. The hardened side is "hundreds" of apps ^[2]. Google has not published how many ordinary apps PageBreak covered, or what it costs to move an app onto the hardened stack ^[1][2]. Take 200 apps, the smallest number "hundreds" can mean, and two flaws come to one per 100 hardened apps ^[2].

Every one of those flaws is an exploit that ran, so neither side of the ratio is padded with guesses. Google puts PageBreak's false-positive rate near zero ^[9]. For comparison, Bitcoin Red Team's August sweep of 501 open-source projects logged 7,958 findings in 108 hours, and 24.7% of them had reproducible proofs at the time ^[12]. About 1,966 came with proofs. Roughly 6,000 did not ^[4].

The attacker in Google's test was also better equipped than an outsider. According to Cryptopolitan's report, PageBreak traces execution paths through a single code repository, maps page requests back to source code using live-traffic data, and gets logged-in access to internal sites from Google's existing scanners, and an outside researcher cannot copy that setup ^[10]. Against that level of access, the hardened frameworks gave up two XSS flaws ^[2].

The gap can be read other ways. The hardened apps may be newer or narrower than the ordinary ones, and a smaller attack surface would produce fewer flaws on any framework. The figure Google cites covers XSS alone, even though its validators also test for database query injection, path traversal and code execution ^[8]. Google also built both the frameworks and the agent that tested them, then cited the result as evidence that safe-by-design frameworks can withstand a relentless automated attacker ^[5].

I think the evidence supports the framework case for XSS. Whether frameworks beat patching depends on whether a migration costs less than the patches it saves. Google's own plans show it is paying for patching as well. It intends to link PageBreak more closely to CodeMender, a fix-writing agent, so a team can review a proposed patch next to each confirmed bug ^[11]. The obvious first job for that pairing is the 500-plus confirmed flaws already in the ordinary apps ^[1].

Google's own numbers would undo the framework case if the ordinary estate runs to 50,000 apps or more. At that size, 500 confirmed flaws come to one per 100 apps, the same rate as the hardened stack ^[3].