top of page

Anthropic Says Prompt Injection Is Nearly Solved, but the Zero Needs Context

Aug 11
11 min read

Anthropic says its newest defenses reduced successful prompt injection attacks to almost zero, challenging a risk many researchers considered fundamental to AI agents.

Boris Cherny, who leads Claude Code, attributed the change primarily to stronger model training. He said layered safeguards pushed the measured success rate even lower. Those layers include prompt injection probes and Claude Code’s auto mode, which classifies tool actions before allowing them to proceed.

The claim matters because prompt injection has limited how safely agents can browse websites, read repositories, process email, and operate connected tools. OpenAI has described the problem as comparable to online scams and unlikely to disappear completely. Anthropic is now offering a more optimistic answer, supported by unusually low results in its latest evaluations.

However, a zero observed attack rate does not establish universal immunity. It describes performance against a defined collection of attacks, environments, tools, and scoring rules. New attacks, broader permissions, or different deployment settings can produce different results.

That distinction will become more important as Anthropic makes auto mode the standard Claude Code experience. The company is moving from conservative permission prompts toward automated decisions at the same moment it is claiming a major security improvement.

Anthropic’s Prompt Injection Claim Comes From a Layered Test

The important change is not that Anthropic added another filter. Its underlying model appears considerably harder to manipulate before filters intervene.

Prompt injection occurs when untrusted content contains instructions intended for an AI rather than its user. An agent might encounter those instructions inside a webpage, repository, tool response, image, or downloaded document.

The attack works because an agent must interpret both trusted commands and untrusted data within its working context. A malicious document can imitate an instruction, asking the agent to ignore its task, reveal information, or call a dangerous tool.

Cherny’s layered defense claim points to three separate protections. The first is model alignment, meaning training that teaches Claude to distinguish the user’s goal from commands embedded in external content.

The second layer is a prompt injection probe. This server-side safeguard examines material returned by tools before the material enters the agent’s context. It can flag suspicious instructions hidden in files, shell output, websites, or external services.

The third layer is auto mode’s intent classifier. Auto mode does not grant Claude unrestricted permission. It evaluates proposed actions and blocks activity classified as destructive, irreversible, or outside the intended environment.

Anthropic’s published results indicate that each layer addresses a different failure point. Training reduces the chance that the model follows a hostile instruction. The probe attempts to identify hostile content before the model acts on it. The action classifier limits what an attacker can accomplish after influencing the model.

That separation matters. A detector can miss cleverly disguised content, while an aligned model can still misunderstand ambiguous instructions. A tool classifier provides another chance to stop the resulting action.

Anthropic previously described this strategy in its research on browser injection defenses. During training, Claude encounters simulated webpages containing malicious instructions. The model receives a reward when it identifies and refuses those instructions while continuing the legitimate task.

That training objective is harder than teaching a model to refuse obviously malicious text. A useful browser agent must still read instructions on legitimate sites, follow user-supplied procedures, and interact with unfamiliar interfaces. Excessive caution can make it secure but ineffective.

The new results suggest Anthropic has improved that security and usefulness balance. They do not show that every possible injection has been eliminated.

The Near-Zero Result Has More Than One Number

“Approximately zero” combines several evaluation settings, and those settings should not be treated as interchangeable evidence of complete security.

According to the Claude Opus 5 system card, the model was evaluated against indirect prompt injections in browser and computer-use environments. Indirect attacks place malicious instructions in content the agent retrieves, rather than in the user’s direct request.

A summary of the system-card results reports that adding a prompt injection probe reduced attack success to 0.18% in two tested configurations. Another browser evaluation reportedly covered 129 environments that were not used during training.

With auto mode added to the stack, no attack succeeded across the evaluated browser scenarios. That is the basis for describing the observed rate as approximately zero. It is a notable test result, especially because the attacks targeted previously unseen environments.

Yet zero successes and zero underlying risk are different statements. When a benchmark records no successful attack, it establishes an upper bound within that sample. It cannot establish that the true rate is exactly zero across every future attack.

The denominator also matters. A system tested against hundreds or thousands of attempts provides stronger evidence than one tested against a small collection. Repeated attacks against the same environment do not necessarily represent the diversity of attacks that appear after deployment.

The evaluation’s threat model matters just as much. A browser attack designed to alter a shopping task differs from a repository attack that hides commands in source comments. Both qualify as indirect prompt injection, but they expose different tools, permissions, and opportunities for harm.

Independent research reinforces this caution. The LivePI benchmark tests agents in production-like workflows containing webpages, files, repositories, email, and group-chat content. Its authors reported total attack success rates between 10.7% and 29.6% across five frontier systems in their tested configuration.

Those realistic agent tests do not directly contradict Anthropic’s numbers. They use different models, harnesses, attacks, permissions, and scoring criteria. Instead, they demonstrate how much measured security can change with the benchmark.

NetInjectBench offers another warning about narrow conclusions. Its network-operations scenarios separate untrusted artifacts from trusted policy metadata and measure unsafe tool use. Naive execution produced an 82.5% unsafe-action rate across 240 attack instances.

The network agent benchmark targets a specialized environment, not Claude Code’s exact production stack. Still, it illustrates why agents with consequential tools need system-level controls beyond a model’s refusal behavior.

Anthropic’s strongest evidence is therefore comparative. Opus 5 appears substantially more resistant than earlier Claude models under Anthropic’s evaluations. The company also reports that its probe and action classifier reduce the remaining failures further.

That supports a claim of meaningful progress. It does not support removing isolation, access controls, audit logs, or human review from high-impact workflows.

Claude Prompt Injection Defense Starts With Training

The central technical shift is Anthropic’s decision to treat prompt injection resistance as a trained model capability, not only a perimeter-security problem.

Early defenses often placed filters between an agent and the content it consumed. The filter searched for obvious phrases, encoded instructions, suspicious formatting, or known attack patterns.

That approach resembles an email spam filter. It works well against familiar attacks but can fail when attackers change wording, distribute instructions across several pages, or conceal intent inside legitimate data.

OpenAI has raised this limitation directly. Its research on agent security design says advanced attacks increasingly resemble social engineering. They create plausible stories that manipulate an agent’s reasoning rather than presenting an obvious command to ignore prior instructions.

A detector must decide whether text is data, a legitimate procedure, or an attack. Those categories can overlap. A repository can correctly instruct a developer to run a setup command, while a compromised dependency can place a nearly identical instruction in a malicious file.

Model-level training gives the agent another source of context. Claude can evaluate whether an instruction supports the user’s stated objective, whether its source has authority, and whether following it would require an unexplained change of plan.

Anthropic calls this instruction hierarchy and alignment work part of the model’s defense. The model learns that content retrieved from the environment should not automatically inherit the authority of the user or system.

This framing addresses a common explanation for prompt injection: role confusion. Language models can understand the words in an instruction while failing to identify who is entitled to issue it.

Recent research has tested that theory by measuring how models represent the speaker behind different instructions. One study found that spoofed reasoning placed in user prompts or tool outputs could produce high attack rates across several systems.

Training can reduce that confusion by exposing the model to adversarial examples. Anthropic creates simulated content that mixes useful information with malicious directives, then rewards the model for completing the intended task safely.

The advantage is generalization. A model can potentially recognize a new attack by its relationship to the user’s goal, even when no detector has seen the exact wording.

The limitation is that training data cannot represent every future environment. An attack can exploit an unusual tool, a long chain of individually harmless requests, or contextual details that make malicious behavior appear legitimate.

Attackers can also optimize against the deployed model. Once they learn which patterns trigger refusal, they can search for inputs that preserve harmful intent while avoiding those patterns.

This is why Anthropic still uses probes and classifiers despite the stronger model. The company’s architecture assumes every layer can fail.

For developers, that is the most transferable lesson from the Anthropic prompt injection results. Training provides a stronger foundation, but safe deployment still depends on the surrounding system.

A coding agent should receive only the credentials, network access, and filesystem permissions required for its task. Teams should separate untrusted inputs from executable instructions and log consequential tool calls.

That principle also applies when an agent searches a company’s technical knowledge base. Retrieved documentation can improve an agent’s work, but the retrieval path also expands the content an attacker might influence.

Auto Mode Turns the Security Claim Into a Product Bet

Anthropic is using its improved defenses to remove permission friction, making the security result part of Claude Code’s core product design.

Claude Code traditionally asks before executing many commands, editing files, or performing actions with broader consequences. Those prompts keep a human in the loop, but they also interrupt long-running work.

Some developers respond by granting broad permissions or using configurations that bypass repeated approvals. That choice increases productivity while weakening the boundary between a manipulated agent and the surrounding system.

Auto mode is Anthropic’s proposed middle path. It routes actions through a classifier that decides whether each action can proceed without a manual response.

Anthropic’s auto mode architecture describes controls at multiple points. A server-side probe examines tool outputs for injection attempts. A separate classifier reviews proposed actions and blocks those considered unsafe.

The classifier focuses on intent rather than a fixed command list. Deleting one generated temporary file might be reasonable, while deleting a large project directory should trigger a block. Sending data to an unfamiliar endpoint presents a different risk from fetching public documentation.

When the system blocks an action, Claude can attempt a safer approach. Repeated attempts or uncertain cases can return control to the user.

This structure reduces interruptions without equating convenience with unrestricted access. It also creates a new dependency: developers must trust the classifier’s interpretation of both the task and the command.

False negatives are the obvious danger. A malicious instruction might cause an action that appears ordinary in isolation, such as reading a configuration file before sending a routine request.

False positives create a quieter problem. If auto mode blocks legitimate actions too often, developers may disable safeguards to finish their work. Security controls that regularly obstruct normal tasks can encourage riskier behavior.

Anthropic’s decision to enable auto mode by default will provide a much larger test than any pre-release benchmark. Developers use unusual build systems, internal command-line tools, remote services, and repositories with complex automation.

That diversity will show whether the classifier remains accurate outside Anthropic’s curated environments. It will also reveal whether the prompt injection probe handles content from plugins, model context protocol servers, package managers, issue trackers, and generated build logs.

The product bet extends beyond Claude Code. Browser agents, research assistants, and enterprise workflow tools all face a similar tradeoff between autonomy and approval.

An agent that asks permission for every step cannot complete substantial work unattended. An agent with broad, persistent authority can turn one successful injection into data theft or destructive action.

Auto mode attempts to move human review from every routine operation to the smaller set of actions that carry material risk. If it works at scale, other agent vendors will face pressure to offer comparable automation.

If it fails visibly, frequent permission prompts may return as a necessary security boundary.

Anthropic and OpenAI Now Frame the Risk Differently

Anthropic’s results challenge a pessimistic view of prompt injection, but OpenAI’s position better captures the uncertainty beyond any fixed benchmark.

OpenAI has compared prompt injection with scams and social engineering. Its public guidance says the problem is unlikely to be fully solved because adversaries continually adapt to deployed defenses.

That position does not mean model training is ineffective. OpenAI also uses specialized training, automated monitors, user confirmations, network restrictions, and supervision for sensitive contexts.

The disagreement concerns what “solved” should mean. Anthropic is emphasizing measured attack resistance after several defenses are combined. OpenAI is emphasizing the open-ended contest between attackers and deployed agents.

Both positions can be true within their intended scope. Anthropic can reach zero observed successes on a difficult evaluation, while prompt injection remains an unresolved security category.

The distinction resembles traditional software security. A product can pass penetration testing without becoming invulnerable. The test provides evidence about known threats and the tested attack surface, not every future exploit.

Prompt injection is especially difficult because legitimate and malicious content use the same interface: natural language. There is no simple equivalent to separating executable code from plain text when an agent must interpret text to decide what to do.

An attacker also needs only one path through the defense stack. The model, probe, classifier, permission system, connected service, and human operator each contribute to the outcome.

Anthropic’s stack offers several opportunities to stop an attack. It also produces several components whose interactions must be tested together.

For example, a model might correctly flag suspicious content but later summarize it in a way that preserves the malicious instruction. A classifier might block direct data transfer but allow several benign-looking steps that achieve the same result.

An attack can also target the user rather than the model. A compromised page might persuade the agent to request approval using a misleading explanation. The user then becomes part of the attack path.

Benchmarks rarely capture every organizational condition. Enterprise deployments include shared credentials, custom tools, old repositories, internal proxies, inconsistent permission policies, and users with different levels of security awareness.

That is why the headline should remain “nearly solved” rather than “eliminated.” Anthropic has disclosed evidence of a major reduction under tested conditions. Real-world assurance requires independent reproduction and ongoing deployment data.

A credible Claude prompt injection defense should also report usefulness alongside attack resistance. A system that refuses unfamiliar tasks can achieve an excellent security score while failing as an agent.

Anthropic’s public research acknowledges this balance, but detailed comparisons across workloads remain important. Buyers need to know how often safeguards block safe work, not only how often they stop attacks.

What to Watch as Anthropic Makes Auto Mode the Default

The next evidence will come from deployment, independent testing, and Anthropic’s response to attacks designed after its defenses became public.

The first signal is auto mode’s default rollout. Watch whether Anthropic limits it by model, account type, environment, or organization policy. A cautious rollout would indicate that the company still sees meaningful differences across deployment settings.

Also watch the rate of user-facing blocks. Frequent denials would suggest that the classifier maintains safety partly by restricting normal activity. Very few denials might indicate good calibration, or insufficient caution.

The second signal is independent adaptive testing against the complete Claude Code stack. Existing public benchmarks establish valuable baselines, but many do not reproduce Anthropic’s exact combination of Opus 5, probes, auto mode, tools, and permissions.

The strongest independent test would use attacks created after researchers studied the released system. It would include poisoned repositories, deceptive tool output, image-based instructions, multi-step exfiltration, and attacks distributed across a long session.

Researchers should report both attack success and task completion. They should document what access the agent received, which safeguards were active, and whether the attacks were known during training.

A zero result under those conditions would strengthen Anthropic’s claim. Successful adaptive attacks would not erase the progress, but they would define the boundaries of the current defense.

The third signal is how quickly the company handles real incidents. Prompt injection protection is not a one-time model property. It is an operational process involving detection, investigation, updated training, and rapid deployment of new safeguards.

Anthropic should disclose enough incident information for customers to understand the affected surface. Security teams need to know whether a failure came from the model, a probe, an action classifier, a permission rule, or a connected tool.

Developers should not wait for that evidence before adopting basic precautions. Run autonomous coding agents in isolated environments when possible. Restrict secrets, network destinations, and production credentials. Review changes before deployment and keep destructive operations recoverable.

Treat repository text, tool output, and downloaded files as untrusted even when they look familiar. A poisoned dependency or copied support response can carry instructions that no developer intentionally approved.

Most importantly, do not interpret Anthropic’s approximate zero as permission to give an agent unlimited authority. Interpret it as evidence that layered defenses can reduce a serious risk without forcing constant manual approvals.

That is a meaningful shift. The open question is whether it survives contact with millions of repositories, new attack strategies, and developers who expect auto mode to keep working without supervision.

Anthropic has set a measurable standard for that debate. Now independent researchers and real-world users need to test it. If the results hold, prompt injection will look less like an unavoidable barrier and more like a security problem that can be managed. If they do not, the failures will show where the next layer must go.

Get your work moving in remio

See what needs your attention and take the next step in Workspace.

remio currently supports Windows 10+ (x64) and Macs with Apple silicon.

Your AI Partner at Work
Get more done with remio

Plan. Create. Deliver.
All in one place.

bottom of page