Anthropic's OSS Scanner ships unreviewed AI vulnerability reports to opted-in maintainers

Only about 21% of those internal candidates got a human triage. Roughly 23,000 were never examined by a person ^[20][21]. The triage work produced hundreds of disclosures at first, and some findings led to CVEs and patches ^[11]. Anthropic did not publish how many of the triaged candidates turned out to be false. These are the company's own results from its own testing across multiple projects ^[10]. For them to carry over to a given project, its code would have to resemble that test set. Its maintainer would also have to do by hand the sorting Anthropic did on 6,000 candidates.

The unreviewed format has already run at scale. Anthropic says it sent roughly 5,000 unverified reports straight to maintainers who asked for bulk submissions ^[12]. OSS Scanner makes unreviewed delivery the default for every enrolled project ^[1][3]. mezha.net, citing The Verge, reports the same warning that reports may contain errors or point to problems that do not exist ^[2][4][18]. The outlet says skipping review lets projects be scanned more often and faster ^[17]. Projects that need or prefer human-verified findings keep the Coordinated Vulnerability Disclosure route ^[9].

I think the reproducer is the best-engineered part of the report ^[6]. A maintainer can run it before reading the model's explanation. If it does not trigger the flaw, the maintainer has grounds to close the report without arguing with the prose. The bisect, where available, points to the likely commit that introduced the bug ^[6]. The suggested patch comes from the same model whose severity ratings Anthropic says can be wrong ^[3]. It deserves the scrutiny a pull request from a stranger gets, and the dev.to analysis says a patch should not bypass code review and testing ^[15].

The report format does nothing about volume. According to mezha.net, open-source projects do not always keep up with the flow of AI-generated bug reports, and Linus Torvalds and Google are among those dealing with it ^[13]. The outlet also credits AI tools with serious recent finds, including a vulnerability that hit almost all Linux distributions in May ^[14]. It says the scanning uses Anthropic's strongest models, Claude Mythos among them ^[5].

I'd expect the unreviewed stream to suit a project with a test suite and a maintainer who has hours to run reproducers. For a one-person project, the dev.to analysis notes that a validated disclosure can fit better where the capacity to assess a large volume of reports is limited ^[19].