Anthropic, OpenAI, and Google Confront Stealing AI Reasoning Traces Despite Encryption

Anthropic, OpenAI, and Google faced a shared security failure despite encrypting hidden model thoughts. Researchers found a path for Stealing AI Reasoning Traces across models, sessions, and users.

The attack did not break encryption with stolen keys or conventional cryptanalysis. Instead, it replayed a valid encrypted trace through a weaker model that could read the plaintext internally. A jailbreak then pushed that model to reveal what it had decoded.

That distinction makes the research more than another prompt-injection demonstration. The providers protected each reasoning block from direct inspection, yet allowed compatible models to process blocks from unrelated contexts. Encryption concealed the data without reliably binding it to its original model, user, or conversation.

The researchers reported recovering credentials and personal information from encrypted blocks found in public agent logs. They also demonstrated model extraction, hazardous-content exposure, and invisible prompt injection.

The affected providers acknowledged the disclosures, according to the research team. The documented attacks reportedly stopped reproducing after providers tightened validation. However, the episode exposes a larger problem for developers building agents around portable, opaque model state.

Stealing AI Reasoning Traces Exploited Portable Model State

The crucial failure was not weak encryption. It was excessive trust in where a valid encrypted block could travel.

Reasoning models often perform internal, step-by-step computation before producing a visible answer. Providers generally hide that chain-of-thought and may expose only a summary. This policy protects model intellectual property and limits direct access to sensitive or hazardous internal content.

Stateless APIs create an engineering complication. A provider cannot rely on a permanent server-side conversation record when customers disable storage or manage histories themselves. The client must return enough state for the model to continue its earlier work.

OpenAI publicly described this design when it introduced encrypted reasoning items. Eligible customers could carry reasoning between requests without storing those items on OpenAI servers. Reusing them was also intended to preserve intelligence while reducing repeated computation.

Google documents a similar mechanism through thought signatures. These are encrypted representations of internal thought that preserve reasoning context across multi-step interactions. Developers using raw REST requests may need to return them exactly as received.

Those designs make encrypted reasoning blocks operational objects, not passive ciphertext. A provider receives the block, verifies it, decrypts it, and places the recovered content into a model’s context. The model can then continue from that prior state.

The research team found that accepted blocks were insufficiently restricted to their original contexts. In the tested systems, a block generated in one session could be accepted elsewhere within the same provider’s model ecosystem.

That compatibility created a route around the strongest model’s defenses. An attacker could obtain an encrypted trace from a frontier model, then supply it to a cheaper sibling with weaker refusal behavior.

The sibling model became what security researchers call a decryption oracle. It could not reveal the provider’s cryptographic key, but it could process the encrypted content and output the resulting plaintext.

The attack therefore combined two properties that looked reasonable in isolation. First, encrypted state had to remain portable enough for stateless conversations. Second, compatible models needed access to earlier reasoning so they could continue the task.

Together, those properties let an attacker move protected reasoning across a safety boundary. The attacker never needed to jailbreak the stronger model that produced the trace.

According to the full paper, the researchers demonstrated the approach across Anthropic, OpenAI, and Google APIs. Their target was an architectural pattern shared by several proprietary model families, not one isolated endpoint.

This is why AI reasoning trace security cannot stop at choosing a strong encryption algorithm. The system must also authenticate the block’s intended model, session, account, and position in the conversation.

A sealed document remains dangerous if every employee can ask an authorized but easily manipulated colleague to read it aloud. The encryption works, while the surrounding authorization model fails.

A Weaker Sibling Became the Decryption Path

Uneven safeguards across a model family turned compatibility into an escalation route.

Frontier models receive extensive training against revealing hidden reasoning. Providers want users to see answers, summaries, and tool results without obtaining raw chain-of-thought. These restrictions also frustrate competitors seeking high-quality training data.

Smaller models operate under different constraints. They prioritize speed, availability, and efficiency, while their resistance to specialized jailbreaks can differ from flagship systems.

The researchers exploited that gap. They generated an encrypted reasoning block with a stronger source model, moved it into another request, and directed a weaker compatible model to transcribe the underlying reasoning.

The source model’s refusal training remained intact. It never received the extraction prompt. Monitoring focused on that frontier endpoint would not necessarily see the suspicious decoding interaction.

The research team evaluated 120 Codeforces programming problems for each tested model configuration. It compared the source model’s reported thinking-token count with the token count recovered after decoding.

The median ratio was 1.00 across all three providers, according to the researchers’ disclosure account. Between 82 and 89 percent of traces landed within five percent of the original reported length.

Those measurements suggest extensive recovery, but they require careful interpretation. Developers cannot normally inspect the hidden original, so token-count agreement is not a direct character-by-character comparison against ground truth.

The researchers used the provider-reported reasoning length as a practical signature. A recovered trace with nearly identical length offers strong evidence, especially when its content matches the task and continues coherently.

However, the result should not be described as independent cryptographic verification of every recovered word. That distinction matters when translating a security preprint into operational conclusions.

Even with that limitation, the attack changes the economics of model extraction. Final answers offer examples of what a model produced. Reasoning traces expose a richer sequence of intermediate choices, corrections, plans, and discarded approaches.

That material can become training data for distillation, where one model learns behavior from another model’s outputs. Providers have consequently treated hidden reasoning as valuable intellectual property.

The paper reports that decoded traces contained about five times more reasoning than the summaries exposed to clients. A competitor harvesting those traces would receive a denser signal than answer-only collection provides.

The method also separated generation cost from extraction cost. Someone else might already have paid for the frontier model to create reasoning inside a published session log. An attacker would only need to decode the existing blocks through a cheaper compatible endpoint.

This does not mean any outsider could decrypt arbitrary server traffic. The attacker still needed access to a valid encrypted block. Public logs, shared trajectories, cached sessions, and exposed application histories supplied that material.

The affected surface therefore extended beyond model vendors. Developers had been encouraged to preserve complete response objects so tools and multi-turn reasoning would continue correctly.

That ordinary integration pattern could retain opaque content that neither the developer nor the user could inspect. The same content could later become valuable to an attacker who found a compatible decoding path.

The immediate lesson is architectural. Security teams must evaluate every model that can consume protected state, including cheaper siblings and legacy endpoints. The strongest producer’s safeguards do not define the security of the whole model family.

Encryption Protected Provider IP but Hid User Exposure

The design prioritized confidential reasoning while leaving users unable to audit what their own logs contained.

Opaque reasoning blocks appear safe when developers review a transcript. A long encoded string does not visibly expose a password, private document, or access token. Conventional redaction may therefore leave the block untouched.

Yet the model’s hidden reasoning can restate information from its context. It may list credentials while planning a deployment, repeat personal details during a booking task, or quote retrieved documents before composing a sanitized answer.

The visible response can remain clean. The encrypted trace may still preserve the sensitive material because the model used it while deciding what to say or do.

Earlier privacy research had already challenged the assumption that internal reasoning is a safe private workspace. That work found that longer reasoning could increase sensitive-data exposure even when final answers became more cautious.

The new attack joined that content risk with portable encrypted state. A trace that users could not inspect or redact became recoverable through another model.

The researchers collected 315,320 encrypted reasoning blocks from 6,708 publicly available agent trajectories. After decoding them, they classified 367 personally identifiable information artifacts and 182 credentials.

The credential set included 62 API keys and 33 passwords. The broader scan produced 704 artifacts across genuine user sessions, and 64 reportedly appeared nowhere in the visible history.

These counts describe the research sample, not the entire public internet. They do not establish how many credentials remained active, whether every item was unique, or how many owners suffered exploitation.

They still demonstrate a concrete disclosure pattern. Public transcripts could carry information that their publishers had no practical way to review.

The situation reverses a familiar security promise. Encryption usually helps data owners limit who can read their information. Here, encryption also prevented users from discovering what the model had preserved about them.

Only the provider could decrypt the trace through its API infrastructure. Until the replay attack became available, users had to trust that the opaque block contained nothing they would regret sharing.

This asymmetry pressured companies deploying coding agents most directly. A coding agent may inspect environment files, deployment configurations, database addresses, and repository secrets while completing a task.

Teams often archive agent histories for debugging, evaluation, or collaboration. Researchers also publish trajectories to compare agent performance or reproduce experiments.

An organization may sanitize visible prompts and outputs before release. That process cannot remove a secret embedded inside ciphertext unless the organization can decrypt and scan it.

The safest immediate response described by the researchers was to remove encrypted reasoning blocks before sharing historical transcripts. That choice can reduce reproducibility or prevent a resumed conversation from retaining its original reasoning state.

Developers therefore face a real tradeoff. Keeping the block supports continuity, while deleting it limits an opaque disclosure channel.

Organizations should treat agent histories as sensitive records even when the visible text looks harmless. Logs deserve access controls, retention limits, secret scanning, and explicit rules governing external publication.

A searchable knowledge base can help engineering teams organize approved technical context. It should not become a dumping ground for unreviewable model-state objects.

The key operational question is no longer whether a transcript visibly includes a secret. Teams must ask whether the agent accessed sensitive data and whether its exported state might have retained that data invisibly.

Hidden Reasoning Also Became a Prompt-Injection Channel

The same portability that enabled extraction could carry an invisible instruction into another agent’s trusted context.

Prompt injection usually arrives through content a model reads. A malicious webpage, document, email, or tool result tells the model to ignore its task and pursue an attacker’s objective.

Defenders can sometimes inspect that content. They may filter suspicious text, isolate untrusted sources, restrict tools, or ask users to approve consequential actions.

An encrypted reasoning block changes the visibility problem. The client sees an opaque string, while the model receives the plaintext as part of its earlier reasoning context.

The researchers created a proof of concept in which a malicious instruction was embedded inside such a block. When another session resumed the trace, the model treated the instruction as its own prior reasoning.

That position can grant the payload unusual influence. Models generally rely on earlier reasoning to maintain plans, remember intermediate results, and decide which tool action comes next.

In the reported demonstration, an unrelated request asked for code that edited a presentation. The hidden instruction caused the generated script to include an additional data-transfer behavior.

The payload did not need to appear in the visible conversation. A reviewer examining only prompts, tool outputs, and final answers could miss the origin of the malicious behavior.

This attack differs from stealing AI reasoning traces for model distillation. Extraction reads confidential state. Poisoning writes or transports a hostile objective through state the recipient cannot inspect.

Both depend on the same broken boundary. A reasoning block from one context should not automatically become trusted reasoning in another.

The scenario matters most for agent systems that exchange, cache, or publish complete trajectories. Teams increasingly use saved sessions for evaluation, demonstrations, debugging, and handoffs between automated workers.

A session exported by one user may be resumed by another. An agent framework may replay recorded histories against several models. A benchmark may distribute traces that participants assume are inert test data.

If encrypted reasoning travels with those artifacts, recipients cannot independently verify its contents. They must rely on provider validation and on the integrity of the system that generated the block.

Output filtering alone is insufficient. A filter might detect a suspicious final command, but it cannot explain why the model chose that command. A subtle payload may also produce behavior that looks legitimate within the requested task.

Tool permissions remain a stronger boundary. An agent that cannot transmit files, read production credentials, or execute arbitrary code has fewer ways to convert hidden instructions into damage.

Human approval also helps when it covers the actual side effect. Approval interfaces should show destinations, files, commands, and changed permissions, not merely a model-written summary.

Reasoning monitors face a more complicated challenge. Providers conceal raw chain-of-thought partly because releasing it can expose intellectual property, sensitive data, or misleading internal speculation.

Customers still need evidence that a tool action follows their instructions. That requirement favors inspectable action plans, structured tool requests, provenance records, and policy checks outside hidden reasoning.

The research does not establish that every encrypted trace can still carry an invisible injection. The providers reportedly changed their validation after disclosure, and the documented attacks stopped reproducing.

However, agent developers should not assume that opaque state is safe because it is provider-generated. Any portable object influencing future model behavior deserves the same suspicion as executable or serialized state.

Coordinated Fixes Narrowed the Attack but Not the Design Debate

The reported vulnerability was largely mitigated, yet the tension between stateless privacy and context binding remains.

The researchers say they disclosed their findings to Anthropic, OpenAI, Google, Microsoft, and Hugging Face before publication. Every affected provider acknowledged the report, according to the team.

By the August 10 publication date, the described extraction attacks reportedly no longer reproduced against the tested APIs. The researchers attributed the change to tighter reasoning-block validation.

A central mitigation is context binding. The provider can cryptographically connect a block to information about its original account, session, model, and conversational position.

That binding changes what successful authentication means. A valid block would no longer mean only that the provider created it and nobody altered its ciphertext.

It would also mean that the current request matches the environment authorized to consume it. Moving the block to a weaker sibling or unrelated session would cause verification to fail.

Cryptographic engineer Matthew Green had explored the portability issue before the full attack appeared. His May 29 reasoning analysis examined why opaque thinking data traveled through client-managed API histories.

The later research converted that architectural observation into practical extraction and poisoning demonstrations. This sequence shows why curious interoperability tests can uncover security boundaries missing from formal API descriptions.

Context binding creates tradeoffs of its own. Customers may legitimately want to migrate a conversation between compatible models, fork a session, replay a benchmark, or resume work without provider-side storage.

Strictly locking every block to one request chain can disrupt those workflows. Providers need explicit, auditable mechanisms for authorized transfer instead of relying on universal compatibility.

Key rotation and version changes also require care. A block tied too narrowly to an endpoint might become unusable after a model upgrade, complicating long-running agents and stored workflows.

The provider could offer scoped transfer tokens or controlled re-encryption. Such mechanisms should preserve a clear record of who authorized the move and which destination can consume the state.

Model-level safeguards remain necessary even after cryptographic binding. A permitted model should not freely reproduce raw hidden reasoning merely because the surrounding envelope verifies correctly.

Likewise, cryptography cannot remove sensitive content that the model generated internally. It only controls where that content can travel and which systems can process it.

This means the fix has several layers. Providers must constrain portability, harden all compatible decoders, monitor unusual replay patterns, and minimize secrets inside reasoning.

Customers must manage transcript exposure, restrict agent privileges, and avoid treating hidden state as harmless metadata. Neither side can delegate the entire problem to the other.

The skeptical point is straightforward. The public evidence for remediation comes primarily from the researchers, who say the attacks stopped reproducing after disclosure.

That is meaningful, but outsiders cannot fully audit proprietary validation logic. Providers have not necessarily published identical technical accounts of their fixes or their treatment of historical blocks.

Customers should therefore avoid declaring the issue permanently closed. A future model, compatibility bridge, legacy API, or migration feature might reopen a similar cross-context path.

AI reasoning trace security depends on a continuing invariant: only the intended principal and model should consume each protected state object. Every new interoperability feature must preserve that rule.

Three Signals Will Show Whether the Fixes Hold

The next test is whether providers can preserve useful model continuity without recreating universal, invisible trust.

The first signal is stricter public documentation for reasoning-block scope. Developers need to know whether a block is tied to an account, project, model family, exact model, session, or request sequence.

Clear validation errors would strengthen confidence. A block moved outside its authorized context should fail predictably, without being silently accepted or downgraded.

Documentation should also explain supported migration paths. If providers permit model switching, they should describe the authorization boundary rather than leaving developers to infer it through experiments.

The absence of such guidance would weaken the security case. Customers cannot design safe retention and sharing policies around undocumented portability.

The second signal is independent retesting across newly released and legacy models. Security researchers should test whether strong-model traces remain unusable through smaller siblings, preview endpoints, regional deployments, and compatibility layers.

A fix that covers current flagship routes but misses an older decoder would preserve the central failure. Model families change frequently, and safety behavior can vary across versions.

Independent confirmation would also clarify the fidelity question. Researchers need controlled methods that compare recovered content with known plaintext or instrumented test systems, not only token-count agreement.

A consistent inability to replay traces across contexts would reinforce the reported remediation. New extraction results would show that block validation remains fragmented.

The third signal is how agent platforms handle historical and shared transcripts. Mature products should strip unneeded encrypted state from exports, warn users before sharing it, and separate resumable sessions from publication formats.

Security controls should treat opaque reasoning as sensitive by default. Export tools should make retained state visible as a risk category, even when they cannot display its plaintext.

Organizations should also rotate credentials when exposed agent histories involved production secrets. Removing a public log does not establish that nobody copied its encrypted blocks earlier.

For current builders, the practical response starts with inventory. Identify which systems retain complete model responses, where those responses are logged, and who can download them.

Then separate operational continuity from long-term records. An active agent may need protected reasoning state, while an audit archive may only need prompts, tool calls, outputs, and structured decision summaries.

Keep model credentials away from unnecessary context. Restrict tools with least privilege, isolate sensitive environments, and require approval for external transfers or destructive commands.

Most importantly, do not treat encryption as proof that a portable object is correctly authorized. Encryption answers who can read or modify data under particular keys. It does not automatically answer where that data belongs.

Stealing AI Reasoning Traces exposed that missing distinction across three major provider ecosystems. The immediate attack reportedly closed, but the architectural lesson will outlast any one model generation.

Developers should now ask a harder question whenever an AI platform returns opaque state: which identities, models, and future actions can this object authorize when it comes back?