An opt-in vulnerability-finding service for open-source software
Short summary
Anthropic is offering free, fully model-generated security scans to eligible open source projects through a new program called OSS Scanner, with reports delivered without human triage. Early testers say the findings were mostly valid, though the company admits some reports will be wrong.
- Anthropic's models found over 29,000 candidate vulnerabilities in six months, but only about 6,000 have been manually reviewed, so human review is the bottleneck.
- OSS Scanner reports go out unverified, but in a pilot, 85 of 97 critical and high-severity findings (88%) met the bar for Anthropic's disclosure process, and only one was a false positive.
- Eligible projects get reports with a reproducer, an explanation, and often a candidate patch; several maintainers said the reports were high quality, and some chained bugs into unauthenticated remote code execution.
Full summary
Anthropic says its models have gotten good enough at finding software bugs that the bottleneck is now human review. On CyberGym, an academic benchmark, LLMs went from catching under 20% of known vulnerabilities at the start of last year to over 85% this year. Over six months of scanning major open source projects, Anthropic turned up more than 29,000 candidate vulnerabilities but has only manually triaged about 6,000. Maintainers, tired of the low-quality AI slop they used to get, are now asking for everything Anthropic has, validated or not. Nearly 5,000 unverified reports have already gone out in bulk at maintainers' request.
The new piece is OSS Scanner, a free, opt-in service that runs Anthropic's strongest models (including Claude Mythos) against open source code with no human review in the loop. Each report includes a reproducer, an explanation, a bisection to find when the bug was introduced where possible, and a candidate patch. Anthropic's justification is speed: exploits can now be built in minutes, so projects that find and fix bugs faster have the edge. The company is still sending human-verified findings through its existing coordinated disclosure process, especially for projects without staff to triage. Early testers were mostly positive. wolfSSL reported 74 reports with only two invalid and five that became CVEs. Daniel Stenberg of curl said the scanner found one of the worst curl vulnerabilities reported in years. PostgreSQL's Noah Misch described fixes he could use nearly as-is. On a sample of 97 critical and high-severity findings across 48 projects, 88% met Anthropic's bar for disclosure, and only one was an outright false positive. Anthropic admits that the unfiltered pipeline will sometimes produce wrong reports and that some maintainers think severity ratings are inflated.
Eligibility follows the same criteria as Google's OSS-Fuzz, meaning projects need critical impact on infrastructure or user security, and decisions are made case by case. Core maintainers enroll by submitting a pull request to a GitHub repo. Anthropic also introduced a Cyber Verification Program that gives qualifying security professionals access to advanced cyber capabilities and fewer blocking classifiers, plus a Claude for OSS program offering free Claude Max 20x subscriptions to help maintainers fix vulnerabilities. The main thing to watch is whether a fully automated firehose of model-generated reports can keep pace with volunteer maintainers' capacity to act on them.