On September 17, Anthropic published something unusual for a frontier lab: a self-graded scorecard for how much of its own research and development work Claude actually does, with the methodology, the sample size, and the places its own measurement could be wrong all laid out in the same post. The headline number is that Claude now “leads” 26% of Anthropic’s AI R&D work, up from under 1% in February. The number that matters more is the one right next to it: as of August 2026, Claude is not operating fully autonomously on any measured slice of that work at all.
How Anthropic actually measured this#
The post — “Measurements for understanding the pace of AI development inside frontier labs” — introduces what Anthropic calls the R&D Automation Index, built on a six-level Automation Level (AL) scale running from AL0 (no AI involvement at all) to AL5 (fully autonomous, no human in the loop). The two levels that matter for the headline number:
- AL3, “Collaborates” — Claude does large chunks of work, but under close human direction.
- AL4, “Leads” — Claude completes most of a task end-to-end from a high-level prompt, while a human supervises rather than steers.
AL5 has not been reached for any measured category. Anthropic’s own worked example draws the AL3/AL4 line concretely: at AL3, an engineer stays actively tuned in while Claude investigates a broken data pipeline. At AL4, the engineer can hand off the failure alert and walk away — Claude handles whatever surprises come up on its own and documents its changes for review before anything ships.
To build the index, Anthropic sampled 20% of R&D staff weekly through July 2026, had Claude agents mine roughly 15,000 granular tasks out of Slack messages and internal records, and organized the results into a 542-node hierarchical taxonomy with 378 specific leaf-level work types — things as narrow as “eval platform defect diagnosis.” Each leaf node then gets scored against the AL scale.
The numbers, in context#
26% “leads,” up from under 1% in February — six months from essentially nothing to a quarter of measured work being handed off end-to-end. Over 90% of measured work is now at AL3 or above, meaning Claude is materially involved in the large majority of what Anthropic’s researchers and engineers do day to day, even where it isn’t yet trusted to run the whole task. And roughly 30,000 AI agents were doing research and engineering work concurrently on Anthropic’s primary internal agent platform as of the measurement window — a scale figure that’s easy to skim past but is, on its own, a bigger number than most companies’ entire engineering headcount.
What’s still explicitly not automated is the part Anthropic itself calls out as the actual bottleneck to recursive self-improvement: deciding which research direction is worth pursuing in the first place. Execution has scaled dramatically. Judgment — the “taste” of what to work on next — remains a human function, and Anthropic frames closing that gap, not raw task throughput, as the real threshold that separates “AI helps build AI faster” from “AI designs its own successor.”
The caveats Anthropic put in its own post#
This is the part worth taking as seriously as the headline, because Anthropic didn’t bury it. The index has real, named weaknesses:
- The judge grading its own homework. Automation-level ratings are produced by having a Claude model evaluate the work — meaning the same kind of model being measured is doing the measuring. Anthropic states this plainly: “we’re using our own models to evaluate our systems, which could mean that the ‘judge’ model could make the same kinds of errors as the model it is checking.” Human reviewers agreed with the model’s rating exactly 59% of the time, and within one level 97% of the time — a wide enough gap on exact agreement that the precise 26% figure should be read as directionally right, not laboratory-precise.
- A frozen basket. The 378-leaf taxonomy is locked to a July 2026 snapshot of what R&D work looks like. Genuinely new categories of work that emerge afterward won’t show up until the next taxonomy refresh, which understates the index’s ability to track a fast-moving frontier.
- Real disagreement on borderline calls. Anthropic acknowledges that reasonable people disagree on where a given task sits between AL3 and AL4 — the boundary isn’t crisp even for human raters.
- A snapshot, not a trend line. One measurement window doesn’t establish a trajectory on its own, even though “under 1% to 26% in six months” is the framing Anthropic itself leads with.
- No cross-lab comparability yet. Anthropic explicitly flags that comparing this index against what any other lab might report would require a shared methodology and third-party verification — something that doesn’t exist today, partly because labs have competitive reasons not to disclose this kind of internal detail at all.
Why this is the right way to publish a number like this#
This blog has been consistently critical of the pattern where an AI lab makes a productivity or capability claim built on a benchmark or a self-report that nobody outside the company can check — Cognition’s SWE-2 numbers, Meta’s Muse Spark configuration mismatch, xAI’s still-unshipped Grok 4.7 comparisons, GPT-6 Astra’s self-reported Terminal-Bench score before independent placement all got exactly that scrutiny in this blog’s coverage. Anthropic’s R&D Automation Index doesn’t escape the same category of problem — Claude judging Claude is a real, acknowledged conflict of interest, not a hypothetical one — but it’s the rare version of a self-reported number that ships with its own confidence interval, its own disagreement rate with human raters, and an explicit statement that the underlying methodology can’t yet be verified by anyone outside the building. That’s a materially higher bar than “trust our number,” even if it’s still Anthropic’s own number.
It also lands as a direct, if implicit, answer to OpenAI’s own September 6 self-report, which claimed 3.1 agent-workdays of research effort per human workday but measured activity volume, not autonomy level, and drew explicit criticism in this blog’s coverage for that distinction. Anthropic’s index is a different, arguably more honest instrument: it isn’t asking “how much AI activity is happening,” it’s asking “how much of this work would still get done correctly if the human supervisor looked away,” and it answers with a number — 26%, not 90%, not 100% — that under-claims relative to the loosest possible reading of its own data. Whether that discipline holds as the number climbs toward AL5 is the thing worth watching next, not this quarter’s snapshot.
Sources: Anthropic — “Measurements for understanding the pace of AI development inside frontier labs” (primary, Sept 17, 2026, full methodology and caveats); Bloomberg — “Anthropic Says Claude Drives 26% of Its Research and Development” (Sept 17, 2026, corroboration); Anthropic — “When AI builds itself” (companion piece on recursive self-improvement framing); this blog’s prior coverage of OpenAI’s research-acceleration self-report and Anthropic’s 80%-of-code disclosure from Code with Claude Tokyo.