Tech Teardown 024 | oss-scanner: Anthropic's official vulnerability scanning repo, 960 stars in 5 days, yet not a single line of scanner code inside

On October 8th, Anthropic’s official account anthropics opened a new repository on GitHub called oss-scanner. Over five days, it earned more than 960 stars (from snapshots on October 10-14, with the repository established in 5 days). The Pull Request number has reached #598. Its popularity does not come from community discussions but from three factors: official releases, 76 heavyweight open-source projects voluntarily registering, and testimonials from maintainers of PostgreSQL, OpenSSL, wolfSSL, curl, and HotCRP in their release notes. What’s more interesting is the repository itself: after examining 184 files, no code for execution vulnerability scanning was found. This is a registry for scanning services, with the scanner staying on Anthropic’s servers. Yesterday’s anatomy analyzed the output layer of the agent ecosystem; today’s focus is on the security layer. These two articles cover two distinct aspects of AI companies engaging in open-source infrastructure development. This article will break down the content into six sections, following old traditions.

I. What It Is

Let’s start with the core conclusion: the oss-scanner repository does not contain scanner code but consists of a “registry + configuration protocol + self-checking tool” trio. The scanner itself runs on the service side and is not open-source. The open-source part addresses another issue: how an open-source project can submit itself to this scanner and conduct self-checks before delivery.

The service itself is Anthropic’s free vulnerability scanning for key open-source projects, available on a opt-in basis. Only projects that voluntarily register are scanned, and usage is governed by the terms of service on red.anthropic.com/oss-scanner. The official release notes are straightforward, with inspiration from Google’s OSS-Fuzz, replacing the fuzzer with the strongest language model (including Claude Mythos). Its predecessor was Anthropic’s internal Project Glasswing, which used Claude to find vulnerabilities, as well as a CVD process that had manually reviewed over 6,000 reports. The division between this service and Anthropic’s paid product Claude Security is also clear: OSS Scanner provides free scanning for open-source projects, while commercial customers use a separate channel.

Therefore, the form of this repository deserves a separate discussion: AI companies build open-source security infrastructure, not models or scanners, but an access protocol. This is completely different from traditional security tools that treat “code as the product” in an open-source manner. The registry being public means it’s traceable who is being scanned; the scanner being closed means how scanning is done remains a black box, with transparency split in two.

II. Repository Structure

The repository contains a total of 184 files (excluding .git), including 80 yaml, 51 md, and 4 py files, under an Apache-2.0 license.

The main body consists of 76 project directories under projects/. Each directory represents a registration file. The list is quite comprehensive: openssl, cpython, spidermonkey (Firefox’s JS engine), nss (Mozilla encryption library), homebrew, drupal, grafana, rabbitmq, jupyterlab, nestjs, valkey (a fork of Redis), uutils-coreutils, lvm2, cryptsetup; security tools include nuclei, grype, syft, zmap, radare2, ossec-hids, go-tuf, tinyssh, botan; several AI projects are also included, such as unsloth, autogpt, goose, and even Anthropic’s own claude-code-templates has registered. Note that curl itself is not on the list, but Daniel Stenberg, the maintainer of curl, is mentioned in the release article.

The core of each file is project.yaml, with two required fields: repo (Git repository address, can use #branch to lock the branch) and primary_contact (email for reporting). Optional fields include dockerfile, threat_model, auto_ccs, pgp, disabled, homepage. The example for openssl is representative: repo points to openssl/openssl and locks the master branch, with primary_contact using the openssl-security@ group email instead of an individual address, which is a demonstration of the rule emphasized in the README: “The email will be public, please use the security alias.”

Tool code is concentrated in tools/, with approximately 1,287 lines of Python and 142 lines of shell code: validate.py has 261 lines for configuration rule verification, while check_build.py and check_qemu.py are responsible for local build self-checking. templates/ contains configuration templates. The repository’s attitude towards contributions is also written at the end of the README: only registered PRs under projects/ are accepted, with no changes accepted for tools or templates. Five days had 598 PRs, almost all of which were waiting to be integrated into the project.

III. Mechanism Breakdown

The entire service process consists of four steps: build, network disconnect, scan, and email sending. Projects provide Dockerfile (either in their own repository or next to the registration directory, choose one; the former is recommended as it eliminates the need for PRs for modifications to the build), the scanner builds the project in a isolated VM. During the initial build phase, there is network access, but after entering the security audit phase, the network is completely disconnected. The scan results are sent to primary_contact by email. Each report includes three items: a self-contained reproducer, a vulnerability explanation (including the time point when the vulnerability was introduced using bisection), and a candidate patch.

The report is divided into two channels. One is the CVD channel, which undergoes manual review; the other is the fast track, where the entire model is generated without manual review and can be released as soon as it is scanned. According to published information, nearly 5,000 unverified fast track reports have been directly sent to those who requested the full report (this is Anthropic’s reported figure). The dual-channel design treats “verification capacity” as a scarce resource: those who need manual review must wait in line, while those who can afford it will receive the results first.

There is also nuance in the network interruption for auditing. The network is allowed during the build phase to pull dependencies, but it is cut off during the auditing phase to prevent the model from connecting externally during analysis: this avoids exposing project code or vulnerability details through environmental scanning and prevents the model from downloading data that could interfere with analysis. tools/check faithfully replicates this environment, which is its value as a self-audit tool—what you see locally matches what happens on the service side.

The local self-audit before registration is the most useful part of the repository. tools/validate.py converts the registration rules from the README into code: field whitelists, HTTPS validation, email format, PGP public key format, and maximum file size limits. It also blocks fields like workflow, budget, and cadence, which can only be set by service operators. tools/check <name> builds the project using the same method as the scanner, starts a shell in the network-interrupted finished image, and tests whether it can run within the container; if so, the scanner is likely to work as well. The --qemu variant uses QEMU on x86-64 Linux to reproduce the scanner’s VM layout. The documentation string in validate.py also includes an honest confession: it does not share code with the service-side scanner, and rule changes must be synchronized on both sides.

IV. Numbers and Verification

Let’s first look at Anthropic’s core numbers, all of which are reported by them. Over the past 6 months, using the latest model, they scanned the world’s most important software projects and found more than 29,000 candidate vulnerabilities. Of these, only about 6,000 had been manually reviewed, and the official admitted that the bottleneck lies with people rather than models. On the academic benchmark CyberGym, the vulnerability detection rate of LLMs has increased from less than 20% at the beginning of last year to over 85% this year.

The verification experiments are the hardest part of the numbers: Anthropic hired penetration testing experts familiar with CVD processes to review the 97 critical/high-level vulnerabilities reported by the scanner in 48 projects. Of these, 85 (88%) met CVD standards; among the remaining 12, 11 were real but resulted from duplicate reports, and only 1 was a true false positive (this is Anthropic’s reported figure).

The official also did not avoid addressing the weaknesses: some operators reported that severity ratings might be inflated, and the scanner might have misjudged the project’s threat model.

Testimonials from maintainers, as quoted from an Anthropic release, are as follows. Noah Misch of PostgreSQL said that a high proportion of the reported vulnerabilities are genuine defects, and many of the fixes included in these reports can be directly applied. Anton Arapov of OpenSSL Corporation compared it to saying that AI vulnerability reports from 18 months ago were terrible, and this batch is no less, if not better, than human-generated reports. Reports with actual exploits are like finishing a job for engineers. Todd Ouska of wolfSSL reported that out of 74 reports, only 2 were invalid, and 5 became CVEs. Daniel Stenberg of curl mentioned that one of the most serious curl vulnerabilities in recent years was discovered through such scans. Eddie Kohler of HotCRP is also on the list of endorsements. All these testimonials come from Anthropic’s releases and are unilateral quotes; they should be used with caution.

V. Judgment

Three points deserve attention.

First, there is a double deviation from disclosure policies. The README states: “Reports are generated by models and have not been manually reviewed, so there is no 90-day disclosure period, nor will they be made public.” Both pillars of traditional vulnerability disclosure practices have been broken here. There is a reason for this in engineering: without verification, applying a 90-day clock forces maintainers to count down on false positives; not making them public prevents unverified information from falling into the hands of attackers. But the cost is that the outside world can never audit what the system has reported and missed, and only Anthropic can calculate the 88% compliance rate.

Second, the cost of triage is shifted to maintainers. Free is true, but the README repeatedly states that reports have not been manually reviewed, and the service is aimed at projects with the capacity to handle verified high-risk reports. For giants like OpenSSL and PostgreSQL, which have dedicated security teams, this is a net gain; for projects with limited resources, the direct distribution of nearly 5,000 unverified reports may be a net burden. All 76 registered projects are heavy-duty projects, and this list itself serves as evidence of the service’s purpose. This is not an accusation, but a definition of boundaries: it filters out projects with security response capabilities, thereby excluding those that most need external security assistance.

Third, there is a drift in the implementation of configuration validation. validate.py does not share code with the service-side scanner; rules are manually synchronized on both sides. This is a clear technical debt signal in engineering, and the larger the scale, the more likely “local validation passes, but service-side import fails” will occur. With 76 projects and 598 PRs, this rule requires manual alignment of code twice each time it is changed.

Broadly speaking, this repository represents a new model: AI companies turn security capabilities into services, with open source being only the access protocol and registry. While the protocol is open, the capabilities are closed. Compared to the previous generation’s model with OSS-Fuzz, where “all infrastructure was open-source,” transparency gives way to iteration speed. Whether this exchange is worthwhile for maintainers depends on whether you believe the 88% compliance rate is an accurate self-reported figure, and how much capacity remains in your security queue.

VI. How It Can Be Used by Those Who Need It

Qualification requirements are not low: manual verification of being a core maintainer is required, along with OSS-Fuzz-style qualification checks to ensure the project handles unreliable inputs and has a sufficient number of downstream users. The process itself is simple: open a PR, add a projects/<name>/ directory, place project.yaml and Dockerfile (or the path to your own repository), and after merging, the scanner will automatically import it.

Two practical suggestions. First, although threat_model.md is optional, it is strongly recommended to write one. The correct usage in the README is to clearly state the severity level: whether SQL injection after authentication is considered high or critical, whether buffer overflow without a demonstrated exploit is capped at high, and under what conditions storage-type XSS should be upgraded. These exactly address the official recognized shortcoming of “overemphasized severity levels.” Second, run tools/validate.py and tools/check before registration. Note that check will execute the project’s Dockerfile over the network. The README should warn users to only check projects they trust or use a machine without significant loss. If you fear reports may be leaked, you can encrypt them with pgp. However, after encryption, only primary_contact should be sent, and it cannot be used with auto_ccs. If you want to pause without deleting the data, set disabled to true.

For ordinary developers, this repository is more like a textbook: 76 project.yaml examples are best practices for configuring security contacts, and the 261 lines of validate.py are clean examples of “writing document rules as validators.”

Information Sources

• Repository and documentation: README on github.com/anthropics/oss-scanner, tools/validate.py, directory list of projects, and openssl sample configuration (10-14 local snapshots, file count and line count counted by find+wc)
• Stars, PR numbers, number of projects: GitHub page and API, 10-14 snapshots (Repository created on 2026-10-08)
• Core numbers (29,000/6,000/near 5,000/88% of 97 are false positives/CyberGym detection rate): Anthropic.com published “Launching an opt-in vulnerability-finding service for open-source software” on 2026-10-08. All data is from Anthropic’s self-reported figures, with annotations placed throughout the text.
• Maintainers’ testimonials (PostgreSQL/OpenSSL/wolfSSL/curl/HotCRP): Testimonials from maintainers in Anthropic’s published article, processed as quoted.
• Disclosure policy, registration rules, usage of self-checking tools: Original README.