AI Reasoning Can Be Extracted, and the US-China Distillation Fight Just Got Harder
- Ethan Carter

- Aug 15
- 14 min read
Google News surfaced a new warning for frontier AI companies: concealing a model’s internal reasoning does not necessarily prevent outsiders from recovering useful traces. Researchers found that carefully structured prompts exposed reasoning-like information from open-weight models, without access to their weights or hidden activations.
That finding adds technical evidence to an increasingly political dispute. OpenAI and Anthropic accuse Chinese laboratories of using American models to generate training data at scale. China rejects the broader allegations as an attempt to suppress its technology sector.
The important shift is not proof that any commercial model has surrendered its actual internal thoughts. The research did not test proprietary production systems for downstream distillation. Instead, it shows why hiding a chain of thought at the interface may provide weaker protection than model developers assumed.
The result creates an uncomfortable distinction. A company can keep its model weights secret while still exposing enough behavior for another developer to reproduce selected capabilities. That turns every public chatbot and application programming interface into a potential source of training signals.
Why the Google News Headline Understates the Dispute
This is no longer a narrow argument about whether one company copied another company’s answers.
The immediate research comes from a paper titled Hidden Thoughts Are Not Secret. Its authors study Reasoning Exposure Prompting, or REP, a method designed to elicit visible reasoning traces through ordinary prompts.
REP begins with examples generated by a separate shadow model. Those examples are wrapped in formats resembling shell commands, Python sessions, notebook cells, or tool calls. The target model then receives a new problem inside the same pattern.
The method does not directly inspect hidden neural activity. It encourages the target to externalize a trace that resembles the reasoning induced by the prompt. That difference matters because visible text is not automatically a faithful transcript of internal computation.
However, fidelity is not the only security concern. A trace can still provide useful supervision even when it is an imperfect explanation. A smaller student model can learn recurring strategies, decompositions, checks, and response structures from that material.
The researchers tested open-weight models in controlled settings. They did not attack commercial APIs or claim to reconstruct proprietary weights. They also acknowledged that their experiments covered a limited collection of models and reasoning benchmarks.
Those boundaries prevent a sweeping conclusion. The paper does not establish that Claude, Gemini, or OpenAI’s production systems can be copied through the same prompts. It does show that interface-level suppression alone cannot be treated as a complete defense.
The Google News framing asks whether reasoning can be “extracted,” but that verb compresses several different processes. An attacker might recover a visible rationale, imitate behavioral patterns, reproduce benchmark performance, or train a smaller model on generated solutions.
None of those outcomes requires recovering the teacher model’s exact internal state. Model extraction usually targets useful behavior, not a perfect digital clone. The attacker wants enough capability to reduce independent research and training work.
That makes the dispute more practical than philosophical. Researchers do not need to settle whether language models truly reason like humans. They only need to determine whether exposed signals help another model solve more problems.
A July 2026 ACL study gives that question added weight. Its authors reported meaningful reasoning improvements after distillation with fewer than 920 examples. Their controlled case study involved much smaller conditions than an industrial campaign.
The study did not establish universal extraction efficiency. Results from one base model and selected benchmarks cannot predict performance across frontier systems. Still, it challenges the assumption that capability transfer always requires enormous datasets.
Distillation itself remains a normal machine-learning technique. A developer uses outputs from a stronger teacher to train a smaller student. Google, OpenAI, Anthropic, and other laboratories use variations of this method within their own product families.
The conflict begins when a third party gathers outputs without permission, evades access controls, or builds a competing model from them. At that point, providers describe the activity as adversarial distillation or model extraction.
The vocabulary carries political weight. “Distillation” sounds like ordinary engineering, while “theft” implies ownership and unlawful acquisition. The law has not fully resolved where routine learning ends and prohibited capability extraction begins.
The Mechanism Does Not Need to Recover a Model’s Actual Thoughts
A useful reasoning trace can transfer problem-solving behavior without revealing the teacher’s literal internal computation.
Language models generate text token by token, using hidden numerical states that are not readable sentences. A displayed chain of thought is another generated output. It may reflect parts of the computation, but it can also rationalize an answer after the fact.
That limitation has been documented across earlier interpretability research. Models sometimes produce convincing explanations that do not accurately identify why they selected an answer. Therefore, extracting a rationale is not equivalent to reading a deterministic program.
Yet distillation does not require perfect introspection. Suppose a teacher repeatedly breaks a geometry problem into the right subproblems. A student trained on those outputs can learn that structure, even if the teacher’s visible explanation omits hidden processing.
The same principle applies to coding and tool use. A teacher can demonstrate how to inspect a repository, choose a command, test an assumption, and recover from failure. Those sequences become examples of successful behavior.
Reasoning models made these outputs especially valuable. They often spend more computation before answering and produce better solutions on mathematics, science, coding, and planning tasks. The resulting traces can contain richer supervision than final answers alone.
REP targets this difference between hidden computation and visible supervision. Its code-like wrappers appear to influence how a model treats demonstrations. The model may interpret them as text to reproduce or inspect, rather than ordinary conversational explanations.
The researchers measured trace similarity and downstream usefulness. Their results suggest that prompting can expose capability-bearing material under some experimental conditions. They do not show that every exposed token corresponds to a concealed internal token.
This distinction should shape any responsible account of the research. Calling REP a mind-reading attack would overstate the evidence. Calling it harmless prompt formatting would ignore the downstream training value that motivated the study.
The strongest interpretation sits between those extremes. A deployed model can disclose useful reasoning signals even when its interface hides raw chain-of-thought text. Those signals can then support behavioral imitation or student-model training.
Commercial providers already avoid presenting raw reasoning for several reasons. Hidden traces may contain unsafe material, internal instructions, private data, or confusing intermediate guesses. Companies often return condensed summaries instead.
Reasoning summaries improve usability, but they introduce a security tradeoff. A detailed summary helps legitimate users understand an answer. The same summary can become structured synthetic data for someone training a competing system.
Reducing detail is not a complete solution either. Final answers still contain knowledge and behavior. Repeated queries can reveal preferences, decision boundaries, coding styles, tool choices, and corrections across many tasks.
An extraction campaign can also combine several signals. OpenAI told a congressional committee that observed pipelines had moved beyond basic chain-of-thought collection. Its February 2026 letter described synthetic-data generation, filtering, and reinforcement-style preference optimization.
That broader pipeline matters more than any single prompt. The attacker can generate problems, collect solutions, score outputs, remove weak examples, and train a student. It can then use the student’s failures to select another query batch.
This creates an iterative loop. The target model functions as teacher, data generator, evaluator, or reward model at different stages. Each role can transfer value without revealing weights.
Detection becomes harder when requests resemble legitimate use. A developer might normally ask a model to grade answers, solve coding problems, or explain mathematics. The security signal appears across account relationships, timing, volume, and repeated task structures.
That is why the new research changes the defensive question. Providers cannot focus only on whether raw chain-of-thought text is displayed. They must examine how combinations of ordinary outputs support systematic capability transfer.
Anthropic’s Allegations Put Scale Behind the Research
The laboratory findings become politically important because providers say real campaigns already operate at industrial scale.
Anthropic said in February that it had identified campaigns linked to DeepSeek, Moonshot AI, and MiniMax. According to its distillation investigation, the campaigns generated more than 16 million Claude exchanges through roughly 24,000 fraudulent accounts.
Those figures come from Anthropic and have not been independently audited in full. The named companies have not publicly validated Anthropic’s attribution. They should therefore be treated as allegations supported by the provider’s internal telemetry.
Anthropic said it reached its conclusions through IP correlations, request metadata, infrastructure indicators, and information from industry partners. The company described coordinated accounts and proxy services designed to bypass regional restrictions.
The laboratory attributed more than 150,000 exchanges to DeepSeek-linked activity. It said those requests targeted reasoning, grading, and politically sensitive response behavior. Some prompts allegedly asked Claude to reconstruct step-by-step reasoning behind completed answers.
Anthropic associated more than 3.4 million exchanges with Moonshot AI. It said the activity targeted agentic reasoning, coding, analysis, computer use, and vision. A later phase allegedly attempted to reconstruct reasoning traces.
MiniMax accounted for more than 13 million alleged exchanges, according to Anthropic. The company said those requests emphasized agentic coding and tool orchestration. It also claimed the campaign shifted traffic shortly after a new Claude release.
The scale transforms the security model. A single curious user is unlikely to reproduce a frontier system. Tens of thousands of coordinated accounts can create a large, diverse, and continuously refreshed synthetic dataset.
Account volume also gives an operator resilience. A provider can block one account without interrupting the collection system. Proxy services can distribute traffic across regions, payment methods, and usage patterns.
OpenAI has made similar accusations. It told lawmakers that most adversarial distillation activity observed on its platform appeared to originate from China, with some activity from Russia. It specifically described DeepSeek-linked efforts to bypass access restrictions.
These claims remain company assessments, not final judicial findings. Providers control the relevant logs, detection systems, and attribution methods. Outside researchers cannot easily reproduce those conclusions without access to the same telemetry.
China has rejected the wider narrative. In April, its Foreign Ministry called American allegations groundless and accused Washington of smearing Chinese AI achievements. The Chinese embassy also emphasized cooperation and intellectual-property protection.
That disagreement creates the article’s central opponent: capability transfer as routine engineering versus capability extraction as unauthorized appropriation. The same technical workflow can support both descriptions, depending on access, consent, scale, and purpose.
DeepSeek’s own work illustrates why the distinction is difficult. Its R1 research openly described distilling reasoning patterns into smaller models. The resulting family helped establish that advanced reasoning behavior could move into more accessible systems.
The peer-reviewed DeepSeek-R1 paper documented reinforcement learning and distillation within the company’s model-development process. That published research does not establish that DeepSeek used unauthorized American outputs.
It does demonstrate the strategic value of reasoning distillation. A capable teacher can help create smaller students that retain selected abilities. Those students can run with lower inference requirements and reach more developers.
American laboratories use related methods internally. Their objection is not that knowledge distillation exists. It concerns competitors obtaining proprietary capabilities through restricted services and allegedly deceptive access methods.
This makes terms of service central but not sufficient. A contract can prohibit competing-model training, automated extraction, or reverse engineering. Enforcement becomes difficult when users operate through intermediaries or outside the provider’s jurisdiction.
The technical research sharpens that problem. If ordinary prompts can recover useful supervision, providers cannot reliably protect their advantage by withholding model weights. They must police patterns of access to a service designed for repeated querying.
The US-China Fight Is Turning Model Access Into Security Policy
Washington now treats unauthorized distillation as a national-security issue, not merely a contract dispute between AI companies.
The shift became explicit in April 2026. A White House memorandum accused foreign entities, principally based in China, of conducting deliberate campaigns against American frontier models.
The administration promised cooperation with AI companies on detection, defense, attribution, and penalties. Its position connects model extraction with the wider effort to preserve an American lead in advanced computing.
Congress has pursued a similar direction. Proposed legislation would create processes for identifying foreign actors accused of extracting key technical features from closed American models. Potential responses include sanctions and trade restrictions.
That approach treats model capabilities like strategic technology. Traditional export controls limit access to advanced chips and manufacturing equipment. Distillation raises the possibility that restricted actors can obtain useful software capabilities through remote services instead.
The economic logic is straightforward. Training a frontier model requires chips, energy, data, engineering labor, and repeated experiments. Querying an existing model can shift part of that cost back onto the provider.
A successful student still needs its own base model, training infrastructure, evaluation, and deployment stack. Distillation is not a button that converts answers into an identical competitor. However, it can reduce expensive trial and error.
American policy advocates therefore describe adversarial distillation as a way to bypass compute controls. Chinese organizations can use foreign infrastructure to generate training data, while reserving domestic computing capacity for other work.
That argument contains an unresolved measurement problem. Similar benchmark scores do not prove that one model copied another. Laboratories often train on overlapping public data, use shared research methods, and optimize against the same evaluations.
Attribution needs more than performance resemblance. Strong evidence would include account metadata, payment links, shared infrastructure, unusual query sequences, internal documents, or reproducible signatures in the resulting model.
Providers say they possess some of that evidence, but much remains private. Public policy is moving faster than independent verification. Sanctions based on opaque platform telemetry would create obvious due-process concerns.
The Google News story matters because technical studies can be pulled into that evidentiary gap. Research showing that extraction is feasible does not establish that a named company performed it. Capability, intent, attribution, and legal liability remain separate questions.
There is also no simple ownership rule for reasoning patterns. Copyright usually protects expression rather than abstract methods. Trade-secret protections depend on secrecy and improper acquisition. Contract law depends on the agreement and the relevant jurisdiction.
Model outputs complicate each category. Providers offer them to customers specifically for reuse in documents, software, analysis, and products. Restrictions usually focus on automated collection or training a competing model, not everyday downstream work.
Enforcement can also affect legitimate research. Security teams need to test whether systems leak reasoning or expose sensitive behavior. Developers may use synthetic data to build specialized tools without attempting to clone a frontier model.
Broad detection systems risk confusing those activities with extraction. A researcher generating thousands of evaluation examples can resemble a distiller. A company grading customer content might resemble reinforcement-learning data collection.
Providers will need layered evidence before imposing serious penalties. Rate limits and account reviews can use lower thresholds. Public attribution, sanctions, or criminal claims demand stronger standards.
The policy dispute also reaches open-weight development. Open models allow researchers to inspect weights, run controlled experiments, and reproduce findings. They also make capability transfer easier because outputs and internal traces can be collected locally.
Restricting open releases would not solve API extraction. It would shift more research behind closed providers and reduce independent scrutiny. Washington has therefore tried to support open-weight development while targeting unauthorized extraction from proprietary systems.
That balance will remain unstable. Stronger open models reduce dependence on American APIs, which can weaken provider control. Yet open releases also expand domestic research, startup competition, and transparent security testing.
The real policy choice is not open versus closed. It is how much access a frontier provider can offer while controlling systematic reuse. No current combination of contracts, monitoring, and interface design fully resolves that tension.
What the Research Still Does Not Prove
The strongest claims in the distillation row run ahead of what independent experiments have established.
First, the REP paper did not extract reasoning from commercial frontier models for student training. Its authors avoided proprietary systems because providers restrict automated extraction and competing-model development.
That choice was ethically appropriate, but it limits external validity. Production systems contain classifiers, hidden prompts, traffic monitoring, account controls, and model-specific reasoning interfaces. An open-weight laboratory setup cannot reproduce all those defenses.
Second, exposed reasoning is not necessarily authentic reasoning. A model can generate different explanations for the same answer. Prompt wrappers may alter the reasoning path rather than recover an original concealed trace.
The paper itself reports signals consistent with this possibility. Stronger wrappers can change the model’s behavior. Higher similarity under a prompted condition does not prove recovery of a unique internal transcript.
Third, downstream gains do not demonstrate broad capability theft. A student can improve on selected benchmarks while remaining weaker across unfamiliar domains. Distillation often transfers narrow patterns more easily than general competence.
Benchmark contamination creates another concern. If teacher and student already encountered related problems during training, later improvements can be difficult to interpret. Researchers need carefully separated tasks and robust generalization tests.
Fourth, no public experiment establishes how much proprietary research cost can be avoided. Querying a teacher still incurs expenses. Filtering millions of outputs and training a student also requires substantial infrastructure.
Greg Brockman, OpenAI’s president, framed the issue as a technical contest in July. He said providers can detect patterns such as response scoring, synthetic-data generation, and attempts to recover reasoning.
That confidence should not be read as proof that detection works reliably. Attackers can reduce request volume, distribute tasks, mix extraction traffic with normal use, or rely on intermediaries. Defenders face false positives and incomplete attribution.
Google has reported frequent model-extraction attempts from private-sector entities around the world. That detail broadens the story beyond a simple bilateral conflict. Capability extraction is a general competitive threat, even when political attention centers on China.
The focus on Chinese laboratories also risks obscuring common industry practices. American companies license data, generate synthetic examples, distill internal models, and learn from public research. The disputed boundary is permission and access, not the basic technique.
There is a safety argument on both sides. Providers warn that distilled systems can inherit capabilities without corresponding safeguards. A student trained mainly for performance may omit refusals, monitoring mechanisms, and deployment controls.
Open-model advocates answer that transparency supports independent testing and wider access. They also question whether proprietary laboratories should define legitimate reuse unilaterally. Those disagreements will not disappear through better classifiers.
The most responsible conclusion is narrower than either political narrative. Reasoning-like outputs can contain transferable value. Hiding raw chain-of-thought text does not eliminate that value, and systematic collection can threaten a provider’s technical advantage.
Everything beyond that requires case-specific evidence. A research method does not prove a national campaign. Provider telemetry does not automatically establish legal theft. Strong benchmark results do not reveal exactly where a model learned its skills.
That verification gap should remain visible throughout coverage. The Indian Express question is worth asking because the answer is conditional. Some reasoning behavior can be elicited and transferred, but an entire model cannot be reduced to stolen text traces.
Three Signals Will Show Whether Distillation Defenses Work
The next phase will be measured through provider controls, independent replication, and government standards rather than stronger rhetoric.
The first signal is whether independent researchers reproduce REP across more architectures. Tests should include varied reasoning models, multilingual tasks, coding agents, and systems that return only short summaries.
Researchers also need stronger definitions of success. Trace similarity, answer accuracy, and student generalization measure different outcomes. A defense that blocks exact trace recovery might still permit useful behavioral imitation.
Replication would strengthen the paper’s security warning if the effect persists across models and unfamiliar tasks. Weak or inconsistent results would suggest that the current method depends heavily on selected architectures and benchmarks.
The second signal is how providers change access controls. Anthropic already describes account clustering, infrastructure analysis, traffic classification, and partner intelligence. OpenAI says it combines machine learning with human review.
More aggressive defenses will affect ordinary developers. Providers might lower rate limits, require stronger identity checks, retain traffic longer, or restrict detailed reasoning summaries. Each change imposes privacy, cost, or usability tradeoffs.
The most informative evidence will be operational. Providers should report detection precision, appeal outcomes, false-positive rates, and recurring attacker adaptations where security permits. Raw allegation counts reveal little without those measures.
The third signal is whether governments create an evidence standard for model-extraction accusations. The White House’s distillation memorandum establishes political intent, but enforcement requires attribution procedures.
A credible framework should separate terms-of-service violations from espionage and national-security threats. It should specify what evidence supports sanctions, how affected companies can respond, and what role independent review plays.
The framework must also protect legitimate interoperability and research. A blanket prohibition on learning from model outputs would reach far beyond industrial extraction. It could constrain evaluation, accessibility tools, education, and specialized application development.
For developers, the immediate lesson is practical. Do not assume a hidden chain of thought makes a deployed system extraction-resistant. Evaluate the training value of every output surface, including answers, summaries, grades, and tool traces.
Enterprise buyers should ask a different set of questions. They need to know how providers retain prompts, detect abusive automation, investigate account suspensions, and separate customer data from security monitoring.
Knowledge workers also have a stake. Tighter defenses can reduce visible explanations and make advanced models harder to access. They can also limit integrations that rely on large-scale processing through shared accounts.
The broader Google News audience should watch whether the debate stays tied to evidence. The technical risk is real, but the political language can outrun what research proves. Both facts can be true at once.
The decisive question is not whether an AI model keeps a secret inner monologue. It is whether public interaction exposes enough structured behavior to lower the cost of building a competitor.
Current research says that risk cannot be dismissed. Provider allegations suggest the incentive already exists at industrial scale. What remains unresolved is who crossed the line, how much capability transferred, and which defenses work without closing useful access.
Readers should treat the next replication, access-policy change, or enforcement action as evidence in that narrower case. Follow those signals instead of assuming that every strong Chinese model was copied, or that hidden reasoning is safely hidden.


