Anthropic's R&D Automation Index Explained: How Claude Leads 26% of AI Research (2026)

Every AI lab talks about automating AI research the way weather forecasters talk about a storm system building offshore: real, visible on the instruments, but hard to pin down in exact numbers. On September 17, 2026, Anthropic published the first numbers from a new instrument of its own. The Anthropic Institute's R&D Automation Index reports that Claude now "leads" 26% of Anthropic's internal AI research and development work, up from under 1% in February 2026 [1]. That is a 26x jump in about six months, inside the one company with the most direct incentive and the most direct ability to measure exactly how much of its own model-building work an AI model is now doing.

If you build with AI, manage a team that uses AI, or just want to know whether "AI is building AI" is a real trend or a marketing line, this number deserves more than a headline reading. Anthropic did not just publish a percentage. It published a full scale, a sampling methodology, a self-reported error rate, and two companion indices measuring how well those automated actions are supervised and how much compute goes to safety work instead of raw capability. That is unusually transparent for a frontier lab, and it is also exactly the kind of self-reported number that gets flattened into "AI writes 26% of Anthropic's code" by the time it reaches a group chat, which is not what the index actually says [2].

This post walks through the whole thing properly: what the Automation Level scale actually measures, how the 26% figure was produced, what "Claude leads a task" looks like as real, runnable code against the Anthropic Messages API, why the two companion indices matter as much as the headline number, and how a team outside a frontier lab can run a scaled-down version of the same audit on its own AI-assisted workflow. Expect real code in nearly every section, not pseudocode, and citations to the primary sources instead of secondhand summaries.

Step 1: What the R&D Automation Index actually measures and why Anthropic built it

Frontier labs have spent the last two years making increasingly specific claims about AI automating AI research, from "AI will write most code within a year" style predictions to internal statements that coding agents already handle a meaningful share of pull requests. Almost none of those claims came with a defined methodology attached. Anthropic's R&D Automation Index is an attempt to fix that, at least for its own house, by building what the company describes as a prototype measurement system rather than a finished benchmark [1].

The mechanics are straightforward once you see them laid out. Each week during a July 2026 measurement window (July 13 to July 20), Anthropic sampled roughly 20% of staff across departments involved in model research and development [1]. A Claude-based research agent reviewed each sampled employee's actual week of work, pulling from Slack messages and internal documents, and extracted a granular list of concrete tasks. That process produced roughly 15,000 individual tasks, which Claude then organized into a hierarchy of 542 category nodes, 378 of them leaf-level categories describing a specific, repeatable kind of work [1]. A separate Claude "judge" model then rated the automation level of each category against a fixed rubric, and those ratings were aggregated into the headline percentage.

Why build this instead of just watching pull request counts or token usage dashboards? Because raw usage metrics answer "how much is Claude used" without answering the more useful question, which is "how much of the actual thinking and decision-making has moved from the human to the model." A researcher who pastes a stack trace into Claude and gets a one-line fix suggestion is using AI. A researcher who hands Claude a vague, high-level goal ("figure out why validation loss diverges on this run and fix it") and gets back a diagnosis, a patch, and a test that passes, with only a final sign-off from the human, is doing something categorically different. The R&D Automation Index is built specifically to distinguish those two situations rather than lump them together, which is the same distinction Epoch AI's own automation-level framework was designed to capture when it proposed treating AI R&D work as a decomposable job the way O*NET decomposes ordinary occupations into rated tasks [3].

This matters beyond Anthropic's own PR calendar. A running theme across AI safety research over the past two years is that automating AI research and development, sometimes abbreviated AIRDA in the academic literature, is one of the most consequential and least measured trends in the field, because it is the mechanism by which capability progress could compound faster than human oversight can track it [4]. A March 2026 paper on measuring AI R&D automation makes almost exactly this argument: existing signals like capability benchmark scores do not tell you whether AI is actually doing the research work inside a lab, whether that acceleration is outpacing safety work, or whether the humans nominally supervising it can actually keep up [4]. Anthropic's index is, in effect, a first real-world attempt to answer that paper's call with actual internal data instead of a forecasting model.

It is also worth being precise about what "R&D" means here, because it is easy to round it up to "Anthropic's whole business." The index covers model research and development specifically: things like designing training runs, debugging infrastructure, writing and reviewing evaluation code, analyzing experiment results, and iterating on architecture and data pipeline decisions. It does not cover sales, legal, marketing, or even most customer-facing engineering. That scoping choice is sensible, since R&D is the part of the company where automation compounding into faster model capability actually matters for the rest of the field, but it also means the 26% figure is a much narrower claim than "26% of Anthropic is automated," a distinction this post comes back to directly in Step 7.

This is not Anthropic's first attempt at measuring how Claude changes real work. The company has published broader economic-impact research for well over a year through its Anthropic Economic Index [5], which launched in 2025 to track how Claude usage patterns shift across occupations and industries [6], and which has since released reports on uneven adoption across sectors [7], the core economic primitives behind AI-assisted work [8], and how usage cadences shift over time [9]. The R&D Automation Index is a narrower, internally focused cousin of that broader research line, trading breadth across the whole economy for depth on one specific, high-stakes category of work: Anthropic's own model development.

Step 2: The Automation Level scale, AL0 through AL5, explained in full

The Automation Level scale itself did not originate with Anthropic. It comes from Epoch AI, an independent research nonprofit that focuses on tracking and forecasting AI progress, which proposed the scale as part of a broader project to build something like an O*NET occupational database for AI R&D work specifically [3]. Anthropic adopted the scale largely as-is for its own internal audit, which is itself a useful signal: an internal metric borrowed from an independent third party is harder to quietly redefine in your own favor than one invented in-house.

The scale runs from AL0 to AL5, six discrete rungs describing how much of a given task's actual thinking and execution has moved from a human to an AI system [1]:

• AL0, no AI involvement. A human does the entire task unassisted. No model is consulted at any point.
• AL1, minimal AI involvement. A human does almost all of the work, occasionally checking a narrow question against a model the way you might glance at documentation.
• AL2, AI assists. The human still owns the task and drives it end to end, but leans on AI for meaningful sub-parts: drafting a function, summarizing a log file, suggesting a few candidate fixes to evaluate.
• AL3, AI collaborates. The AI does large, substantial chunks of the actual work under close human direction. The human is still actively steering step by step, but is no longer doing most of the typing or the first-pass thinking.
• AL4, AI leads. The AI completes most of the task end to end starting from a single high-level prompt, while a human supervises and signs off. This is the level Anthropic's 26% figure refers to specifically.
• AL5, AI is autonomous. The AI operates with no human in the loop at all, up to and including a model fully autonomously building its own successor. Anthropic was explicit that it observed zero measured categories at AL5 as of the July 2026 sample.

The AL3-versus-AL4 line is the one worth sitting with, because it is where most of the public confusion about the 26% number comes from. AL3 is still fundamentally human-directed: a person is deciding what to do next after every step. AL4 flips the direction of initiative. The human states a goal, the model does most of the actual reasoning and execution to get there, and the human's job narrows to reviewing the output rather than producing it. Anthropic's own reported inter-rater numbers show why this line is genuinely hard to draw consistently: when two human employees independently rated the same task category, they landed on the exact same automation level only 35% of the time, while a separate model judge matched the human-assigned rating exactly 59% of the time, and both humans and the model agreed within one level of each other 97% of the time [1]. That is a genuinely useful piece of methodological honesty to publish, and it is a big part of why this post treats 26% as a real but noisy measurement rather than a precise fact, a point Step 7 expands on.

You do not need Anthropic's internal tooling to use this scale yourself. It is simple enough to encode directly, and doing so is a useful exercise for auditing your own team's AI-assisted workflow later in Step 6. Here is the rubric as a plain Python enum with the real definitions attached, something you can drop into a scoring script as-is.

python
/code
from enum import IntEnum

class AutomationLevel(IntEnum):
    """Epoch AI's Automation Level (AL) scale, as adopted by Anthropic's
    R&D Automation Index. Higher numbers mean more of a task's actual
    thinking and execution has moved from a human to an AI system."""
    AL0_NO_AI = 0        # A human does the entire task unassisted.
    AL1_MINIMAL = 1      # AI is consulted occasionally, human does almost everything.
    AL2_ASSISTS = 2      # Human drives the task end to end, AI helps with sub-parts.
    AL3_COLLABORATES = 3 # AI does large chunks of the work under close human direction.
    AL4_LEADS = 4        # AI completes most of the task end to end from a high-level
                         # prompt, human supervises and approves the result.
    AL5_AUTONOMOUS = 5   # AI operates with no human in the loop at all.

AL_DESCRIPTIONS = {
    AutomationLevel.AL0_NO_AI: "No AI involvement. A human does the entire task alone.",
    AutomationLevel.AL1_MINIMAL: "Minimal AI involvement. Human does almost all the work.",
    AutomationLevel.AL2_ASSISTS: "AI assists. Human drives the task, AI helps with parts of it.",
    AutomationLevel.AL3_COLLABORATES: "AI collaborates. AI does large chunks under close human direction.",
    AutomationLevel.AL4_LEADS: "AI leads. AI completes most of the task from a high-level prompt, human supervises.",
    AutomationLevel.AL5_AUTONOMOUS: "AI is autonomous. No human in the loop at all.",
}

def describe(level: AutomationLevel) -> str:
    return f"{level.name} ({int(level)}): {AL_DESCRIPTIONS[level]}"

if __name__ == "__main__":
    for level in AutomationLevel:
        print(describe(level))

Notice that this rubric says nothing about which model is used, how good the output is, or how fast the task gets done. It only measures where the locus of initiative sits between the human and the AI on a given task. A team using a much weaker model at AL4 (human states the goal, model attempts the whole thing, human reviews) is, by this framework, more automated on that task than a team using a frontier model at AL2 (human drives every step, AI helps here and there), even if the second team's output is objectively better. That is a deliberate design choice in the scale, not an oversight, because the index is trying to measure delegation of initiative, not output quality.

Step 3: The trajectory, from under 1% to 26% in six months

A single snapshot number is interesting. A trajectory is what actually tells you whether something is a blip or a trend, and this is where the R&D Automation Index gets more useful than a one-off percentage. Anthropic reported four data points across 2026 [1][10]:

• February 2026: under 1%. Almost no measured R&D task category had Claude operating at AL4 or above. The vast majority of work sat at AL0 through AL2, meaning AI was a helper, not a lead.
• May 2026: 12%. A meaningful jump in three months, roughly matching the period when agentic coding tools and longer-horizon tool-use loops became standard parts of Anthropic's own internal engineering stack rather than experimental side projects.
• July 2026: 22%. Nearly doubling again, coinciding with wider internal rollout of longer-running agent sessions capable of multi-step debugging and evaluation work without constant human re-prompting.
• August 2026: 26%. The most recent measured point, and the one behind the headline figure everyone is citing.

Line up those four numbers and the shape is not linear, it is compounding: under 1% to 12% is a jump of roughly 11 points, 12% to 22% is another 10 points, and 22% to 26% is a smaller 4-point step. Read uncharitably, that last step looks like deceleration. Read more carefully, it likely reflects something else: the easiest, most mechanically repeatable categories of R&D work (writing unit tests, drafting boilerplate evaluation harnesses, summarizing experiment logs) probably crossed the AL4 threshold first and fastest, while the remaining 74% increasingly includes harder categories, research taste calls about which experiments are worth running, judgment about ambiguous data quality issues, architecture decisions with long-term consequences, that resist automation for structural reasons rather than because nobody has tried. Epoch AI's own framework flags exactly this distinction, separately tracking tasks that require "taste," like deciding whether to scale up a particular training recipe, because those categories behave differently from mechanically repeatable ones [3].

It is also worth grounding this trajectory against independent capability research rather than taking Anthropic's internal number in isolation. METR, the same organization behind several of the field's more rigorous agent capability evaluations, has separately documented that the length of software engineering task a frontier model can complete at a 50% success rate has been doubling roughly every seven months since 2019, with some evidence the doubling accelerated further in 2024 and 2025 [11]. A model that can reliably carry out a two-hour task versus one that tops out at fifteen minutes is functionally the difference between AL2-style assistance and AL4-style leading, because longer reliable task horizons are exactly what lets a human hand over a high-level goal instead of babysitting every step. The R&D Automation Index's trajectory and METR's independently measured time-horizon trend are not the same study, but they are pointing at the same underlying capability curve from two very different angles, internal task audits on one side, external benchmark evaluation on the other, and they broadly agree on both the direction and the rough steepness of the curve.

METR's own RE-Bench evaluation, built specifically to compare AI agents against human ML research engineers on real research engineering environments, adds a useful nuance here rather than a simple confirmation. In that benchmark, the strongest AI agents actually outscored human experts within a 2-hour budget by roughly 4x, but humans caught up and eventually pulled ahead as the time budget extended, matching top AI scores by around 8 hours and roughly doubling them by 32 hours [12]. That is consistent with what Anthropic's own trajectory implies: automation climbs fastest on shorter, well-specified tasks first, and the harder, longer-horizon, judgment-heavy categories of R&D work are the ones still moving up the AL scale more slowly, which is a reasonable way to read why the August jump (22% to 26%) was smaller than the two jumps before it. The training environments used to build and evaluate these longer-horizon agents are themselves scaling fast, a trend covered in more depth in Miraflow's piece on RL environment scaling for AI training, which helps explain why the capability curve behind both METR's benchmark and Anthropic's own trajectory keeps bending upward rather than flattening out.

Step 4: What "Claude leads a task" looks like in practice

Reading "AL4: AI leads" as a one-line definition undersells how specific and constrained the real behavior is. Anthropic was explicit that no measured category hit AL5, and that the AL4 pattern always keeps a human in the supervisory loop at the point where the AI's output actually ships or gets merged [1]. The cleanest way to understand this concretely is to build the pattern yourself, using the real Anthropic Messages API and the anthropic Python SDK, rather than treating it as an abstract description.

The shape of an AL4 workflow is: a human states a high-level goal once, an agentic loop with tool access does most of the actual multi-step work (reading files, running commands, iterating on failures) without further human micromanagement, and the loop stops and waits for an explicit human approval before anything is actually applied, merged, or deployed. That last part, the mandatory pause for a human decision, is what keeps a task at AL4 instead of drifting toward AL5. Here is a minimal but real implementation of that pattern, built against Anthropic's actual Messages API [13] and its documented tool use interface [14].

python
/code
import json
import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from the environment

# In-memory staging area. Nothing here has touched disk yet, that is the
# whole point: it lets the agent iterate freely while keeping the actual
# write behind an explicit human approval step.
STAGED_PATCHES = {}

TOOLS = [
    {
        "name": "read_file",
        "description": "Read the contents of a file in the repository.",
        "input_schema": {
            "type": "object",
            "properties": {"path": {"type": "string"}},
            "required": ["path"],
        },
    },
    {
        "name": "run_tests",
        "description": "Run the test suite and return pass/fail output.",
        "input_schema": {"type": "object", "properties": {}},
    },
    {
        "name": "propose_patch",
        "description": (
            "Stage a proposed code change for human review. This does NOT "
            "write to disk. A human must call apply_approved_patch separately."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "path": {"type": "string"},
                "diff": {"type": "string"},
                "rationale": {"type": "string"},
            },
            "required": ["path", "diff", "rationale"],
        },
    },
]

def execute_tool(name: str, tool_input: dict) -> str:
    if name == "read_file":
        with open(tool_input["path"]) as f:
            return f.read()
    if name == "run_tests":
        # Replace with your real test runner invocation.
        return "3 passed, 0 failed"
    if name == "propose_patch":
        patch_id = f"patch-{len(STAGED_PATCHES) + 1}"
        STAGED_PATCHES[patch_id] = tool_input
        return (
            f"Patch staged as {patch_id}, pending human approval. "
            "It has NOT been applied to the repository yet."
        )
    raise ValueError(f"Unknown tool: {name}")

def run_agentic_task(high_level_goal: str, max_turns: int = 10) -> str:
    """AL4 pattern: one high-level prompt in, the model leads every
    intermediate step, but nothing is applied until a human calls
    apply_approved_patch explicitly."""
    messages = [{"role": "user", "content": high_level_goal}]

    for _ in range(max_turns):
        response = client.messages.create(
            model="claude-opus-5-20260805",
            max_tokens=2048,
            tools=TOOLS,
            messages=messages,
        )
        messages.append({"role": "assistant", "content": response.content})

        if response.stop_reason != "tool_use":
            return next(
                (b.text for b in response.content if b.type == "text"), ""
            )

        tool_results = []
        for block in response.content:
            if block.type == "tool_use":
                result = execute_tool(block.name, block.input)
                tool_results.append(
                    {
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": result,
                    }
                )
        messages.append({"role": "user", "content": tool_results})

    return "Max turns reached without a final answer."

def apply_approved_patch(patch_id: str) -> None:
    """The one and only place a write actually happens. A human calls
    this explicitly after reading the staged diff and rationale."""
    patch = STAGED_PATCHES.pop(patch_id)
    print(f"Applying approved patch to {patch['path']}:")
    print(patch["diff"])
    # Real implementation would apply the diff to disk here.

if __name__ == "__main__":
    summary = run_agentic_task(
        "Figure out why validation loss diverges on run exp-2214 and fix it."
    )
    print(summary)
    print("Staged patches awaiting human approval:", json.dumps(list(STAGED_PATCHES), indent=2))
    # A human reviews STAGED_PATCHES here, then calls:
    # apply_approved_patch("patch-1")

Walk through what that loop actually enforces. The run_agentic_task function hands Claude a single high-level goal, exactly the AL4 pattern of "a high-level prompt" rather than a sequence of micromanaged instructions [1]. Inside the loop, Claude can call read_file, run_tests, and propose_patch as many times as it needs, iterating on failures, reading new files, and revising its patch, all without a human in the loop for any of those intermediate steps. That is the "most of the task end to end" part of the AL4 definition. But propose_patch is deliberately not wired to actually write to disk. It stores the proposed diff and returns a message telling Claude the patch is staged, pending human approval. The actual write only happens in apply_approved_patch, a separate function a human calls explicitly after reading the diff. That single design decision, giving the model a tool that stages a change instead of a tool that applies it, is the entire mechanical difference between an AL4 workflow and one drifting toward AL5, and it is worth treating as a hard architectural rule rather than a suggestion if you are building similar agent loops for your own team. The example above targets Claude Opus 5, whose capabilities, evaluations, and safety testing are documented in Anthropic's own published system card [15]; if you are deciding whether a given task actually needs that top-tier model or would run just as well, and cheaper, on a lighter model, Miraflow's breakdown of Claude Opus 5 versus Sonnet 5 benchmarks and pricing is a useful next read before wiring up your own agent loop.

This pattern generalizes well beyond code. A research-analysis version of the same loop might let Claude read experiment logs, query a metrics database, and draft a written analysis of why a training run's loss curve looks anomalous, but require a human to approve the analysis before it goes into a shared document that other researchers will treat as ground truth. A content-operations version, closer to what a Miraflow AI user might build, could let an agent draft a full video script, select b-roll prompts, and assemble a rough cut, then pause for a human to approve the final render, matching Miraflow's own idea-to-video pipeline where a creator can generate a script, regenerate it, edit the AI-generated scene prompts, and only then commit to final video generation rather than letting every stage run without review, as described in Miraflow's Text2Shorts workflow here.

If you want to see the same approval-gate pattern applied to a very different domain, this post's Step 6 script below reuses the identical staging-then-approval structure for scoring, not just for code changes, which is a useful sign that the pattern is genuinely general rather than specific to software engineering. It is also the same underlying idea that shows up repeatedly in agent security research: an agent that can stage an action but not execute it unilaterally is dramatically harder to manipulate through indirect prompt injection than one with unmediated write access, a point covered in more depth in Miraflow's write-up on indirect prompt injection and agent security and in the PaperCut agent attack analysis, both of which describe what goes wrong when that human checkpoint is missing or bypassable.

For teams building longer agent sessions like the one above, keeping the model's working context focused on the current task rather than letting it balloon with every tool call's full output is a real engineering problem in its own right, one that Miraflow's post on context engineering with MCE and ACE covers in more depth than this post has room for. The short version relevant here: an agent that stays at AL4 for a two-hour session needs the same kind of context management as any other long-running agent, or its later tool calls start operating on stale or truncated context and the human reviewing the final diff ends up doing more corrective work than the "human supervises" framing implies.

For readers who want to see this concept visualized rather than only in code, a Wan-style video generation prompt describing the full agentic pipeline, from human prompt to staged patch to human approval, is included below as a standalone, copy-paste prompt for a video generation model.

A Wan-style cinematic video generation prompt, five to eight seconds, wide 16:9 frame. Camera slowly dollies forward through a clean, well-lit control room built like a technical diagram brought to life: flat, evenly lit surfaces, soft blue and grey industrial color palette, no dramatic shadows. In the foreground, a small robotic arm labeled 'Agent' repeatedly reaches toward a glowing console, types on a holographic keyboard, and pulls a printed diagnostic sheet from a slot, then places it into a transparent staging tray labeled 'Staged, pending approval', looping this action three times with slightly different diagnostic sheets each time. A human silhouette stands at a second console further back, calmly reviewing each sheet as it appears in the tray, then presses a large physical button labeled 'Approve' on only one of the three sheets, at which point a mechanical gate opens beneath the approved tray and the sheet drops through into a lower chamber labeled 'Applied'. The other two sheets remain visibly sitting in the staging tray, unapplied. Steady, smooth camera movement, flat vector-diagram art style rendered in motion rather than photorealism, clean readable labels throughout, no flashing lights, no distorted text, no watermark or logo.

Step 5: The other two indices, agent oversight and compute allocation to safety

Publishing an automation percentage on its own, with no accompanying measure of how well that automated work is supervised, would be close to irresponsible for a lab whose entire public safety case rests on maintaining meaningful human oversight of increasingly capable systems. Anthropic clearly understood this, because the same September 17 release included two companion measurements published alongside the R&D Automation Index rather than as an afterthought [1].

The agent oversight index. As of August 2026, Anthropic reported approximately 30,000 AI agents conducting research and engineering work inside the company at any given time [1][2]. Every action those agents take passes through an online monitor before it is allowed to execute, and Anthropic reported 100% coverage of agent actions by that online monitoring layer. Across roughly one billion agent decisions logged in August 2026, the monitor blocked about one in every 47,000 actions, or roughly 0.002% [1]. A separate offline monitor reviews 100% of transcripts after the fact and flags an estimated one to two transcripts per thousand for closer human review. Those numbers describe genuinely dense, automated monitoring infrastructure, not a spot-check process, and they are the direct answer to the obvious follow-up question a skeptical reader should ask after seeing "26% leads": if AI is doing more of the actual work, who is watching what it does along the way. This layered monitoring approach is a direct extension of Anthropic's Responsible Scaling Policy, which publicly commits the company to matching its safety and oversight investment to a model's measured capability level rather than applying a flat, one-size-fits-all review process [16], and it builds on the company's earlier Constitutional AI work, which trained models to critique and revise their own outputs against a written set of principles instead of relying purely on after-the-fact human feedback [17]. Anthropic's own published research on sabotage risk is directly relevant background here too, since the same team has previously built and run specific evaluations for whether a frontier model could subtly undermine oversight mechanisms, hide capabilities during testing, or steer human decisions without appearing suspicious, finding that current mitigations were sufficient for the models tested but likely to need strengthening as capability increases [18].

The compute allocation to safety index. Over the same measurement window, July 13 to July 20, 2026, Anthropic classified how its AI R&D compute budget was split between safety-relevant work and other work. The headline figures were 6% of total AI R&D compute allocated to safety work broadly, and 12% of compute specifically within AI-driven AI R&D work classified as safety-relevant [1]. Anthropic flagged real limitations on this number itself: it comes from a single week's snapshot rather than a trend, the underlying workload labels are self-reported and best-effort rather than independently verified, and the company deliberately counted work that helps both capability and safety as AI R&D rather than safety-only, which biases the safety percentage conservatively low rather than flatteringly high.

Reading all three indices together is the actual point, not reading any one of them in isolation. A 26% automation figure with no oversight data would invite the reasonable worry that Claude is quietly making unsupervised decisions about its own successor's design. A 26% automation figure paired with 100% action-level monitoring and a sub-0.01% block rate tells a more specific and more checkable story: automation is rising quickly, but it is rising inside a monitoring layer that scales with it rather than lagging behind it. Whether 0.002% is itself a reassuring number or simply a reflection of how narrow the monitor's current definition of "blockable" is remains a fair question, and it is exactly the kind of question the newly announced plan for embedded third-party evaluators is meant to eventually help answer, since Anthropic stated it intends to have external reviewers verify these metrics and practices independently going forward rather than relying solely on self-report [1].

It is also worth situating this three-index structure against the broader AI safety research landscape rather than treating it as something Anthropic invented in a vacuum. The 2026 International AI Safety Report, an independent, internationally coordinated assessment of frontier AI risk, explicitly calls out the difficulty of tracking whether AI-driven R&D automation is outpacing the oversight and safety infrastructure meant to constrain it, which is precisely the gap Anthropic's three-index structure is trying to close for its own operations [19]. A March 2026 academic proposal for measuring AI R&D automation makes the same point even more directly, arguing that automation metrics divorced from oversight metrics and safety-investment metrics are close to meaningless on their own, because a rising automation percentage could just as easily represent accelerating risk as accelerating productivity depending on what is happening to the other two numbers [4]. Anthropic publishing all three together, on the same day, with the same measurement window, is a direct response to that exact critique, whether or not the company frames it that way explicitly.

Step 6: How to run a mini version of this audit on your own AI-assisted workflow

You do not need 30,000 internal agents or a dedicated research team to get real signal out of this framework. The core idea, cataloguing the tasks your team actually does, rating each one's automation level against a fixed rubric, and tracking the distribution over time, scales down cleanly to a team of five or a solo creator's own workflow. Below is a small, genuinely runnable scoring script that implements the same core idea: log tasks, rate them against the AL0-AL5 rubric from Step 2, and produce a distribution summary you can track month over month the same way Anthropic tracked February through August.

python
/code
from collections import Counter
from dataclasses import dataclass, field
from datetime import date
from enum import IntEnum

class AutomationLevel(IntEnum):
    AL0_NO_AI = 0
    AL1_MINIMAL = 1
    AL2_ASSISTS = 2
    AL3_COLLABORATES = 3
    AL4_LEADS = 4
    AL5_AUTONOMOUS = 5

@dataclass
class LoggedTask:
    name: str
    logged_on: date
    level: AutomationLevel
    rated_by: str  # e.g. "human" or "model", keep this honest and separate

@dataclass
class WeeklyAudit:
    week_label: str
    tasks: list = field(default_factory=list)

    def add(self, name: str, level: AutomationLevel, rated_by: str = "human"):
        self.tasks.append(LoggedTask(name, date.today(), level, rated_by))

    def distribution(self) -> dict:
        counts = Counter(t.level for t in self.tasks)
        total = len(self.tasks) or 1
        return {
            level.name: round(100 * counts.get(level, 0) / total, 1)
            for level in AutomationLevel
        }

    def leads_or_above_pct(self) -> float:
        """The same slice Anthropic's headline 26% figure measures:
        share of tasks rated AL4 or higher."""
        leads = sum(1 for t in self.tasks if t.level >= AutomationLevel.AL4_LEADS)
        return round(100 * leads / (len(self.tasks) or 1), 1)

    def rater_disagreement(self, other: "WeeklyAudit") -> float:
        """Compare a human rater's scores against a second rater (human or
        model) on the same task names, mirroring Anthropic's own published
        inter-rater agreement check."""
        mine = {t.name: t.level for t in self.tasks}
        theirs = {t.name: t.level for t in other.tasks}
        shared = set(mine) & set(theirs)
        if not shared:
            return 0.0
        matches = sum(1 for name in shared if mine[name] == theirs[name])
        return round(100 * matches / len(shared), 1)

if __name__ == "__main__":
    week = WeeklyAudit("2026-W38")
    week.add("Idea research", AutomationLevel.AL2_ASSISTS)
    week.add("Script drafting", AutomationLevel.AL3_COLLABORATES)
    week.add("Scene visual prompts", AutomationLevel.AL4_LEADS)
    week.add("Voice selection", AutomationLevel.AL2_ASSISTS)
    week.add("Final cut approval", AutomationLevel.AL1_MINIMAL)
    week.add("Thumbnail choice", AutomationLevel.AL2_ASSISTS)

    print("Distribution:", week.distribution())
    print("Share at AL4 or above (Claude 'leads' equivalent):", week.leads_or_above_pct(), "%")

Running this on a real week of work is more revealing than it sounds. Most teams that have not deliberately measured this assume their AI usage is more advanced than it actually is, because the moments where AI visibly helps (a fast first draft, a quick bug fix suggestion) are more memorable than the much larger volume of AL0 and AL1 work that never involved a model at all: reviewing a contract, deciding which client to prioritize this week, or making a judgment call about brand voice. A useful exercise is to log every task for a real five-day week before rating anything, resisting the urge to rate in real time, since real-time rating tends to inflate scores toward whichever automation level feels aspirational rather than accurate. Anthropic's own inter-rater numbers from Step 2, only 35% exact agreement between two humans rating the same task, are a good reminder to have at least two people independently score a sample of the same tasks and compare, rather than trusting a single rater's intuition.

For a content or marketing team specifically, a useful first pass at categories to log looks like: idea generation and topic research, script or copy drafting, visual asset creation, video editing and assembly, thumbnail or cover image selection, scheduling and publishing, and performance analysis after the fact. Score each honestly against the rubric from Step 2 rather than against how automated you wish it were. A team using Miraflow AI's AI clipping tool to turn a long-form video into several ranked, captioned shorts is likely running that specific task closer to AL3 or AL4, since the tool identifies viral moments, scores them, and produces ready shorts from a single upload with minimal per-clip direction, while the same team's decision about which finished short actually gets published, and with what caption, usually stays a human AL1 or AL2 call even when a model suggests options.

One methodological trap worth flagging explicitly, because it undermines a mini audit just as easily as it could undermine Anthropic's real one: do not let the model doing the rating also be the model whose usage you are trying to measure objectively, without a human spot-check layered on top. Anthropic's own index has this exact structural tension, a Claude judge rating Claude-assisted work, and it addressed it partially by publishing the human-versus-model agreement rate rather than hiding it [1]. If you build your own version of this audit, keep that same discipline: have a human review a random sample of the model's automation-level ratings, and report the disagreement rate alongside your headline number rather than treating the model's rating as ground truth.

Step 7: Common misreadings of 26%, and why self-report matters here

The single most common misreading of this number, visible across social media reaction to the announcement, is treating "26%" as a claim about AI research broadly, as if it means AI now does a quarter of all machine learning research happening anywhere in the world. It does not. It is Anthropic's own internal measurement of Anthropic's own internal R&D task categories, sampled from Anthropic's own staff, rated by Anthropic's own models, and published by Anthropic [1]. That is a meaningful and unusually transparent number, but it is a company-specific, task-specific figure, not an industry-wide one, and treating it as the latter overstates the claim by an order of magnitude in scope even if the underlying percentage itself turns out to be roughly accurate for Anthropic specifically.

A second, subtler misreading treats "AL4: leads" as functionally equivalent to "unsupervised" or "autonomous." It is neither. Anthropic was explicit and repeated across its release that zero measured categories reached AL5, and that the defining feature of AL4 is a human supervisor remaining in the loop at the decision point, exactly the approval-gate pattern built out in Step 4's code [1]. A task rated AL4 still has a human who reads the output, decides whether to accept it, and bears responsibility for that decision. Collapsing AL4 into "autonomous" is the kind of rounding error that turns a genuinely nuanced methodology into a much scarier or much more impressive headline than the underlying data supports, depending on the reader's priors going in.

Third, and this is the point worth taking most seriously as a reader rather than dismissing: this is a self-reported number, measured by the company whose commercial and reputational interests are served by both a high number, which supports "AI is transformative" marketing, and a well-supervised number, which supports "AI is safe to keep scaling" marketing. Anthropic did not hide this tension. The published methodology openly states that the model doing the automation-level rating was Claude itself, that human-versus-human agreement was only 35%, and that the compute-allocation figure was a single-week snapshot with self-reported, unverified workload labels [1]. That level of self-disclosed uncertainty is genuinely better practice than most self-reported corporate metrics receive, but "better than typical corporate self-report" is still a meaningfully lower evidentiary bar than an independently audited number, and Anthropic itself acknowledged this gap by committing to bring in embedded third-party evaluators going forward [1].

It is also useful context that other labs talk about this same underlying trend in less quantified terms. Public statements from competing labs about AI automating research have generally taken the form of qualitative predictions, timelines to an "AI researcher" milestone, or narrow benchmark claims like SWE-bench resolution rates, rather than a company publishing a task-sampled, self-audited percentage of its own internal R&D work with a documented inter-rater disagreement rate attached. Independent evaluations of automated-research systems from outside the frontier labs paint a genuinely mixed picture that is worth holding alongside Anthropic's number: an outside assessment of Sakana AI's fully automated "AI Scientist" system found real capability at generating research ideas and running experiments cheaply, at roughly six dollars per manuscript in one test run, but also concluded the system fell well short of the "artificial research intelligence" framing used to describe it, with meaningful gaps in experimental rigor and novelty assessment [20]. A later version of the same system, AI Scientist-v2, went further by submitting fully autonomous manuscripts to a real peer-reviewed ICLR workshop, and one of three submissions scored above the average human acceptance threshold, a genuinely notable result but also a reminder that "one out of three papers cleared a bar" is a very different claim from "AI now does research at human level" [21]. Google DeepMind's AI co-scientist system, built on Gemini, has shown validated results across specific biomedical applications like drug repurposing and antimicrobial resistance mechanism hypotheses, while still relying on verification modules that explicitly cross-check the system's own numerical claims against execution logs precisely because unverified self-reported claims from an AI research system are not trustworthy on their own [22]. None of that is a knock on Anthropic's index specifically. It is the broader pattern this whole field keeps running into: automated or AI-assisted research claims, whether about idea generation, code, or self-measurement, need independent verification before they should move a reader's actual beliefs, and Anthropic's own report says as much about itself.

Finally, do not skip past the finding that has almost no headline value but arguably matters most: METR's randomized controlled trial testing AI coding tools against a real baseline found that experienced open source developers who were allowed to use AI tools actually completed tasks 19% slower than a matched control group working without them, despite believing beforehand that AI would speed them up by 24% and believing afterward that it had sped them up by 20% [23]. That is not a contradiction of Anthropic's index, since the two studies measure completely different things: METR measured wall-clock task completion time for a specific developer population on specific mature codebases, while Anthropic measured where the locus of initiative sits on internal R&D tasks broadly. But it is an important reminder that "more AI involvement" and "self-reported feeling of acceleration" are not reliable proxies for actual measured speed or quality, which is exactly why a rigorous automation-level framework, with real inter-rater checks, is more valuable than either raw usage statistics or a team's gut sense of how automated its own work has become.

Step 8: What this means for content and engineering teams building with AI

Most teams outside a frontier AI lab will never need to publish an R&D Automation Index of their own, but the underlying discipline, cataloguing what you actually do, rating how much of it has genuinely moved to AI-led execution versus AI-assisted execution, and tracking that honestly over time, transfers directly to any team building products, content, or code with AI in the loop in 2026, including creators running their entire pipeline inside a single browser-based platform like Miraflow AI.

For engineering teams, the practical takeaway from Step 4's approval-gate pattern is architectural, not just conceptual. If you are building agents that operate on real infrastructure, real codebases, or real customer data, the single highest-leverage design decision is usually not which model to use but where you place the mandatory human checkpoint, and whether an agent's tools are built so the risky action (writing to disk, sending a message, deploying code) is structurally separated from the reasoning that leads up to it. That same principle, keeping execution behind an explicit gate rather than trusting an agent's own judgment about when to stop, is a much better long-term investment than trying to prompt-engineer an agent into being more cautious, and it is the same lesson that shows up across agent security research on real-world incidents where that gate was missing here. Teams evaluating which model to route different automation levels of work to, a lighter, cheaper model for AL2-style assistance versus a stronger, more expensive one for AL4-style leading, will also recognize this as a model-routing decision, a topic covered in more depth in Miraflow's explainer on model routing between different model tiers, where routing by task difficulty rather than defaulting to the most expensive model for everything is exactly the kind of efficiency gain that also shows up in Anthropic's own compute-allocation thinking. And for teams whose agents run for a long time before that human checkpoint arrives, the same efficiency concerns that show up in Anthropic's compute allocation numbers apply at a much smaller scale too, where techniques like speculative decoding are a large part of why longer agentic sessions have become fast enough to be practical at all, a topic explored in Miraflow's post on speculative decoding and faster LLM inference.

For content teams and individual creators, the more useful framing is thinking honestly about which parts of your own pipeline currently sit at each automation level, and being deliberate about which ones you actually want to push toward AL4 versus which ones benefit from staying at AL2 or AL3 on purpose, whether that pipeline runs on a general-purpose reasoning model or on a model tuned more for rich media generation, like the tradeoffs covered in Miraflow's Claude Fable 5.1 benchmarks and pricing explainer. Miraflow AI's own idea-to-video pipeline, taking a creator from a raw topic idea through script generation, scene visual prompts, voice selection, and final video assembly, is itself a useful case study in this exact tradeoff: script generation and scene-prompt generation can comfortably run at AL3 or AL4, since a creator can review a full draft in seconds and regenerate freely, while the final choice of which cut to publish, and which thumbnail earns the click, tends to work better kept at AL2, human-led with AI assistance, since that decision carries real brand and audience-fit judgment that a fully autonomous pass is more likely to get wrong. The same logic applies to Miraflow's cinematic AI video generator, where a creator can hand over a detailed prompt and let the model lead the actual clip generation, while retaining the final call on which take represents the brand well enough to publish. Creators repurposing long-form content into shorts, whether through Miraflow's clipping tools or through a manual review pass, are already living this tradeoff in practice, as covered in Miraflow's guides on turning a podcast into viral shorts and on AI voice cloning for content creators, both of which involve exactly this kind of per-task automation-level decision rather than a blanket "let AI handle everything" or "review every frame" policy.

There is also a genuine argument for deliberately keeping some categories of work below AL3 even once the technology comfortably supports higher automation, purely as an organizational choice rather than a capability limit. A team that lets every category of work drift to AL4 as fast as the underlying models allow is optimizing for velocity at the expense of the kind of hands-on familiarity with its own product and audience that tends to produce the judgment calls a model cannot yet make well. Anthropic's own three-index structure is implicitly making this same argument at a much larger scale: automation without deliberately maintained oversight capacity is not actually progress, it is just risk moving faster, and the same logic scales down to a five-person content team deciding how much of its editorial judgment to hand to an agent.

Benchmark work outside pure R&D automation is worth watching here too, since it tends to arrive a step or two ahead of what any single lab reports internally. Broader agent benchmarks that test AI systems on realistic, open-ended computer-use tasks rather than narrow coding challenges, like the work described in Miraflow's coverage of the OSWorld-2 computer-use agent benchmark, give a useful independent read on how close general-purpose agents are to AL4-style task leading on the kind of messy, GUI-driven work that a lot of real content and marketing operations still run on, well outside the tidy, code-shaped tasks that dominate most internal R&D automation measurement. A separate academic effort, the Remote Labor Index, takes a similar approach for remote knowledge work broadly rather than R&D specifically, attempting to measure how much of the tasks that make up real remote jobs current AI systems can complete end to end, which is the closest thing to an Anthropic-style automation index built for the wider labor market rather than one company's internal research function [24].

Conclusion

Anthropic's R&D Automation Index is a genuinely useful piece of transparency, not because 26% is a magic number, but because the company published the scale, the sampling method, the disagreement rate, and two companion indices covering oversight and safety compute allocation, all in the same release, rather than leading with a headline stat and burying the caveats. The trajectory from under 1% in February to 26% in August tells a real and fast-moving story about how quickly agentic AI has moved from assisting engineers to leading well-scoped tasks end to end, and the accompanying oversight numbers, near-total monitoring coverage and a block rate below one hundredth of one percent, are the direct, checkable answer to the obvious worry that automation is outrunning supervision.

None of that makes 26% an industry-wide fact, a claim about AI research everywhere, or evidence that any measured task has crossed into full autonomy, and Anthropic's own report says exactly that. What it does show, clearly enough to build your own smaller version of the same audit, is that the gap between "AI assists" and "AI leads" is a real, measurable, and rapidly shifting line, one that any team building agentic workflows in 2026 should be tracking deliberately rather than guessing at. Whether you are debugging a training run, editing a long-form video into shorts, or deciding how much of your next product launch to hand to an agent, the AL0-AL5 rubric from Step 2 and the approval-gate pattern from Step 4 are the two pieces of this index worth actually adopting, regardless of whether you ever publish a percentage of your own.

Frequently Asked Questions

What exactly is Anthropic's R&D Automation Index? It is a prototype internal measurement, published by the Anthropic Institute on September 17, 2026, that samples Anthropic staff's actual weekly tasks, rates each task category on a six-level Automation Level scale from AL0 (no AI involvement) to AL5 (fully autonomous AI), and aggregates the results into a single percentage describing how much of Anthropic's internal AI research and development work Claude currently "leads" [1].

Does 26% mean Claude is replacing Anthropic's researchers? No. The 26% figure describes the share of sampled task categories rated AL4, where Claude completes most of a task end to end from a high-level prompt while a human supervisor reviews and approves the result. Anthropic explicitly reported zero measured categories at AL5, the fully autonomous level, meaning every measured category still has a human in the loop at the point of decision [1].

Who created the AL0-AL5 automation scale? The scale was developed by Epoch AI, an independent AI research and forecasting nonprofit, as part of a broader effort to build a detailed, occupation-style breakdown of AI R&D work. Anthropic adopted the scale largely unchanged for its own internal audit rather than building a proprietary alternative [3].

Is the 30,000 internal agents figure reliable? It comes directly from Anthropic's own published release and was independently reported by known outlets covering the announcement, describing approximately 30,000 AI agents conducting research and engineering work inside the company at any given time as of August 2026 [1][2]. As with the rest of the index, it is a self-reported operational figure rather than an independently audited one.

How does this compare to what other AI labs have said about automating research? Most other frontier labs have made qualitative predictions or narrow benchmark claims about AI automating research rather than publishing a task-sampled, self-audited percentage with a documented human-versus-model disagreement rate. Independent academic evaluations of fully automated research systems from outside the major labs show real but limited capability, well short of reliable human-level research autonomy [20][21].

Can a small team run its own version of this audit? Yes. The core method, logging real tasks over a set period, rating each one against the AL0-AL5 rubric, and tracking the distribution over time, scales down cleanly. Step 6 of this post includes a runnable Python scoring script and practical guidance on avoiding the same measurement pitfalls Anthropic's own report disclosed, including keeping a human check on any AI-assisted rating.

Does a task rated AL4 mean no human ever reviews it? No, the opposite is true by definition. AL4 specifically requires a human supervisor to remain in the loop, reviewing and approving the AI's output, which is what separates it from AL5's fully autonomous, no-human-in-the-loop category. The worked code example in Step 4 shows this as an explicit approval gate a human must call before any change is actually applied.

References

1. Measurements for Understanding the Pace of AI Development Inside Frontier Labs, Anthropic Institute
2. Anthropic Says Claude 'Leads' 26 Percent of Its AI R&D Work, Engadget
3. Toward an O*NET for AI R&D, Epoch AI
4. Measuring AI R&D Automation (arXiv:2603.03992)
5. The Anthropic Economic Index
6. Introducing the Anthropic Economic Index, Anthropic
7. Anthropic Economic Index Report: Uneven AI Adoption, September 2025
8. Anthropic Economic Index Report: Economic Primitives, January 2026
9. Anthropic Economic Index Report: Cadences, June 2026
10. Anthropic Says Claude Leads 26% of Its AI R&D Work, Quartz
11. Measuring AI Ability to Complete Long Software Tasks (arXiv:2503.14499)
12. RE-Bench: Evaluating Frontier AI R&D Capabilities of Language Model Agents Against Human Experts (arXiv:2411.15114)
13. Messages API Reference, Anthropic Docs
14. Tool Use Overview, Anthropic Docs
15. Claude Opus 5 System Card, Anthropic
16. Anthropic's Responsible Scaling Policy
17. Constitutional AI: Harmlessness from AI Feedback (arXiv:2212.08073)
18. Sabotage Evaluations for Frontier Models, Anthropic
19. International AI Safety Report 2026 (arXiv:2602.21012)
20. Evaluating Sakana's AI Scientist for Autonomous Research (arXiv:2502.14297)
21. The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search (arXiv:2504.08066)
22. Towards an AI Co-Scientist (arXiv:2502.18864)
23. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (arXiv:2507.09089)
24. Remote Labor Index: Measuring AI Automation of Remote Work (arXiv:2510.26787)