A QA vendor scored 17 public Playwright suites, and the grade is not the interesting number

The most consequential design decision in this scorer is a penalty cap: after three occurrences of the same rule in the same file, further hits stop counting toward the score, though the full findings list keeps every one ^[12]. That is the whole reason cal.com's suite can produce 1,215 findings from 53 files, the highest density in the corpus, and still land on 85 and clear the threshold ^[10]. Mattermost, with 284 files and 1,157 tests, produces 1,476 findings, roughly five per file against a corpus average of 4.5, and grades 90 ^[11][1]. cal.com alone accounts for about 22 percent of every finding in the corpus while holding 4 percent of its spec files ^[3]. What actually drags its grade is the locator ratio: 613 role-based calls against 875 raw ones, 41.2 percent native where the corpus sits at 69.9 percent ^[13][6].

So the grade measures how widely an anti-pattern is spread, not how often it fires. Two teams can quote the same tool and describe the same suite as a B-grade pass or as the worst offender on the list, and neither is lying.

One number in the writeup does not close. Thirty percent of 17,118 locator calls is roughly 5,150 raw selectors, while the no-raw-locators rule, at 60 percent of 5,467 findings, is roughly 3,280 ^[4]. That leaves on the order of 1,900 raw calls that produced no finding, and the post does not reconcile the two counts ^[7]. Since the run is one script over sparse clones of named subdirectories ^[5], it is checkable, which is more than can be said for most vendor charts.

The zero-assertion headline deserves the same arithmetic in the other direction. A hundred and two cases across 12 of 17 repos sounds endemic; measured against 5,943 tests it is 1.7 percent ^[9][2]. Not comforting, because every one of them is green and no pass/fail gate will ever surface them ^[9], but it is a rate rather than a rot.

The conflict of interest is disclosed and remains a conflict. A managed QA vendor built the instrument, wrote the rules, set the weights and the threshold of 80, and chose the 17 repos ^[3][2][4]. The AI-free framing is not what makes this worth reading ^[2]; the reproducibility is ^[5]. Anyone who thinks sqs-v1 flatters the vendor's own house style can fork it and reweight. Meanwhile the locator rules alone are 74 percent of all findings and the raw-selector rule is flagged in 16 of the 17 repos ^[7], which leaves at most one clean suite in the corpus ^[5], three years after Playwright's documentation began recommending role-based locators ^[14].