China AI Safety Governance Framework 3.0 Turns Its Focus to Agents and Control

China released the China AI Safety Governance Framework 3.0 on September 14, marking its third revision in three years. The document keeps China’s risk-based model but shifts attention toward AI agents, physical-world systems, and the danger of losing human control. That change matters because AI is moving from generating answers to performing actions.

The National Technical Committee 260 on Cybersecurity, known as TC260, released the framework during the opening of China’s 2026 National Cybersecurity Week in Jinan. The Cyberspace Administration of China, or CAC, guided its development.

The publication is not a new statute, and it does not automatically impose fresh penalties on AI companies. Its importance comes from somewhere else. TC260 frameworks can shape technical standards, testing practices, sector guidance, and the expectations that regulators apply to developers and service providers.

Version 3.0 also arrives one year after the framework added broader social and ethical risks. China is now pressing the same governance model into systems that can use tools, access files, communicate with other services, and act with less direct supervision.

That creates the central tension. China wants AI adoption to expand across businesses and public services, while insisting that increasingly autonomous systems remain predictable, traceable, and subject to human intervention.

What the China AI Safety Governance Framework 3.0 Changes

The framework treats autonomous action, not better conversation, as the defining safety problem of the next AI deployment cycle.

According to the official framework release, TC260 retained three elements from earlier editions: risk classification, technical countermeasures, and comprehensive governance. It updated the risks inside that structure and adjusted the recommended responses.

That continuity is important. Version 3.0 is not a rejection of China’s previous approach. It is an attempt to apply the same approach to systems with broader access, longer operating periods, and more influence outside a chat window.

The document describes AI applications spreading from specialized settings into mainstream use. It names work assistants, personal assistants, and agent-enabled phones as examples. It also distinguishes systems that answer questions from systems that perform tasks.

An AI agent is a system that can perceive information, retain context, make decisions, call tools, and execute steps toward a goal. Those abilities can reduce routine work, but they also create longer chains of possible failure.

A chatbot can produce an incorrect answer. An agent can accept that answer, open a service, change a record, send information, or trigger another process before a person notices the mistake.

Version 3.0 therefore gives agent risks more explicit treatment. Its annex on AI agent risk management covers the system lifecycle, from development and deployment through operation and withdrawal.

The framework asks organizations to examine what an agent reads, which instructions it trusts, what tools it can call, and whether its actions remain within the user’s request. These questions address the process behind an output, not only the output itself.

That distinction changes how safety must be evaluated. A final report may look accurate even if the agent exposed confidential data while producing it. A completed transaction may appear valid even if malicious instructions redirected one step of the workflow.

Prompt injection illustrates the problem. A prompt injection attack places hidden or deceptive instructions in content that an AI system processes. An agent browsing a webpage could interpret those instructions as commands, even when the user never approved them.

The framework also discusses risks from compromised plug-ins, manipulated tool responses, excessive permissions, and unreliable memory. Each weakness becomes more serious when the system can act without confirmation.

Embodied intelligence receives greater attention as well. This term covers AI connected to machines that sense or affect the physical world, including robots, vehicles, and industrial equipment.

Once software can control a physical device, a cybersecurity failure can become a safety incident. A manipulated instruction might affect movement, production, access control, or equipment operation.

The document further identifies recursive improvement and self-accelerating development as issues requiring vigilance. It does not state that current systems have escaped human control. Instead, it argues that faster autonomous optimization deserves sustained testing and oversight.

The result is an AI safety framework explained through a new operational reality. Model behavior still matters, but permissions, tools, connections, memory, and physical access now belong inside the safety boundary.

Why China Is Updating Its AI Governance Playbook Again

Three annual releases show that China sees AI governance as a moving technical process, not a policy document that can remain fixed for years.

TC260 published the first edition in September 2024. That version organized risks involving models, algorithms, data, systems, cybersecurity, physical safety, cognition, and ethics.

The first framework established principles that still shape the series. It prioritized innovation while calling for inclusive, prudent, risk-oriented, and cooperative governance.

Version 2.0 followed on September 15, 2025. It preserved the original structure but added a category for derivative risks, meaning broader consequences that emerge through widespread AI use.

Those risks included effects on employment, social structures, resource demand, education, human relationships, and the environment. The second edition also added a clearer principle of trustworthy use and prevention of loss of control.

Official commentary on the second framework said it contained 14 comprehensive governance measures and four safety guidelines. It also placed scientific and technological ethics more firmly inside the AI lifecycle.

Version 3.0 builds on that foundation rather than creating another classification system. Its timing reflects how much the deployment target has changed.

Many generative AI products once operated as isolated interfaces. Users entered prompts, models returned text, and people decided what to do next. That separation is disappearing.

Agents can now search internal documents, call software interfaces, write code, operate browsers, prepare messages, and coordinate multistep tasks. Even modest autonomy expands the number of decisions a system makes between human checkpoints.

China’s policy work in 2026 already signaled that shift. In May, the CAC, the National Development and Reform Commission, and the Ministry of Industry and Information Technology issued implementation guidance for AI agents.

The agent guidelines define agents as systems capable of autonomous perception, memory, decision-making, interaction, and execution. They also identify 19 application scenarios across research, industry, consumer services, public welfare, and social governance.

That document promotes adoption while stressing safety, controllability, and standardized development. Framework 3.0 supplies a broader risk-management layer around the same direction.

This is why the update is more than annual policy maintenance. China is trying to establish safety expectations before agents become ordinary infrastructure across sensitive sectors.

The framework identifies government, finance, education, broadcasting, healthcare, and emergency management as areas needing sector-specific guidance. A single horizontal rule cannot capture the different consequences of failure in those environments.

An unreliable shopping assistant might waste a customer’s time. An unreliable medical or emergency-management system can influence decisions involving health, public resources, or immediate safety.

China AI governance therefore combines general principles with plans for narrower standards. The framework describes the risks, while later standards can translate selected parts into requirements that auditors and testing organizations can assess.

This layered structure also allows policy to change without rewriting every binding rule. Regulators can update technical guidance as system behavior changes, then incorporate selected controls into standards or sector regulations.

For companies, the pace creates a practical burden. A safety review designed for a text model may not cover an agent that uses external tools. A review designed for one deployment may become incomplete after permissions, integrations, or operating scale change.

Version 3.0 addresses that problem through agile governance. Risk assessments should consider application context, system intelligence, and deployment scale, then change as those conditions change.

The annual sequence makes the policy message difficult to miss. Deploying a more capable system means reopening the safety case, even when the underlying model name remains unchanged.

The Core Tradeoff Is Innovation Versus Verifiable Control

China wants organizations to deploy useful agents, but it expects them to show that autonomy remains bounded throughout the entire task.

The framework describes innovation as a priority and supports controlled experimentation, including regulatory sandboxes. A regulatory sandbox allows selected products to operate under limited conditions while authorities and developers observe risks.

At the same time, the document calls for systems that remain under human control. Those two goals do not naturally align once an agent can choose tools and intermediate actions.

A tightly constrained agent is easier to monitor but less flexible. A broadly authorized agent can complete more tasks, yet it creates a larger attack surface and more paths toward unintended behavior.

Consider a workplace research assistant. It might search internal notes, browse public websites, compare records, and prepare a recommendation. Each capability looks reasonable when evaluated separately.

The risk appears in their combination. A public page could contain a hidden instruction telling the agent to retrieve confidential material. Broad file access could let it comply, while an outbound connection could expose the result.

Preventing that sequence requires more than filtering the agent’s final text. Developers must separate trusted instructions from untrusted content, minimize permissions, validate tool calls, and record actions for later review.

The framework calls for stronger monitoring, early warning, emergency response, and testing. It also supports vulnerability databases and information-sharing mechanisms involving developers, service providers, and technical institutions.

These measures point toward an evidence-based model of control. A company should be able to show what the system accessed, which decision triggered an action, and when a human could stop it.

That expectation creates pressure for developers, enterprise buyers, and operators. Developers must build safeguards into agent architectures. Buyers must evaluate the entire deployment rather than relying on a model provider’s safety claims.

Operators also need clear responsibility boundaries. When a model provider, application developer, plug-in vendor, cloud service, and customer all influence one workflow, accountability can become fragmented.

Framework 3.0 assigns responsibility across model and algorithm developers, service providers, and users. However, a broad statement of responsibility does not automatically resolve disputes after a complex failure.

Supply chains make the problem harder. An agent may rely on an open-source model, third-party retrieval software, external APIs, and enterprise data systems. A vulnerability in any component can affect the final behavior.

The framework acknowledges different risks in open and closed models. Closed systems can limit outside auditing and customization because one provider controls their safety mechanisms.

Open models allow broader inspection and adaptation, but safeguards can be removed or bypassed. Updates may also fail to reach every modified or privately deployed copy.

Neither approach eliminates governance work. Closed providers must offer meaningful evidence about controls, while open deployments need local testing, configuration management, and a plan for security updates.

The same tradeoff appears in model accuracy. Hallucination, meaning a plausible output unsupported by reliable evidence, becomes more dangerous when an agent treats it as a basis for action.

Teams cannot solve that problem by adding a warning below the output. They need mechanisms that require authoritative evidence, block unsupported actions, or route high-impact decisions to people.

These controls reduce speed and add operational cost. They can also limit the autonomy that makes agents attractive. That is the central bargain behind the China AI Safety Governance Framework 3.0.

The document does not require every system to receive identical treatment. Its risk-grading approach considers the importance of the scenario, the system’s intelligence, and the scale of use.

That approach is more practical than applying maximum controls everywhere. It also raises hard questions about thresholds.

Organizations must decide when an ordinary assistant becomes a high-impact agent. They must determine which permissions create material risk and which actions require confirmation.

The framework offers direction, but many operational answers will depend on later standards, sector rules, and evaluation methods. Until those arrive, companies must interpret broad principles within their own technical environments.

A Framework Is Not the Same as an Enforceable Rulebook

The publication signals regulatory priorities, but it does not by itself establish a complete compliance test for every AI agent.

TC260 is China’s national cybersecurity standards body. Its technical documents can influence regulation, procurement, assessments, and formal standards, but their legal effect depends on the document and the rules connected to it.

Framework 3.0 is best understood as a governance map. It identifies risk categories and recommends technical and institutional responses. It does not replace applicable laws or mandatory standards.

China already has binding rules for some generative AI services. The Interim Measures for Generative AI Services apply mainly to services offered to the public within mainland China.

Other requirements can involve algorithm filings, security assessments, personal information, data security, content governance, and sector oversight. The relevant obligations depend on the provider, service, users, and deployment context.

A finalized national standard provides a useful contrast. China’s GB/T 45654-2025 covers basic safety requirements for generative AI services, including training-data safety, model safety, and protective measures.

An English standard translation notes that it supports filing, registration, testing, and evaluation. It applies to providers and can guide regulators and third-party evaluators.

That standard is more specific than the broad framework. It discusses procedures for training data, generated content, model security, and safety assessment.

Even then, outside analysts have questioned how consistently demanding measures will be enforced during rapid model competition. The same uncertainty is greater for recommendations that remain at the framework level.

Version 3.0 calls for regular testing of potential technological loss of control. Yet the release does not establish a universal test suite, public score, or single threshold that decides whether an agent is safe.

Terms such as controllability, trustworthy use, and catastrophic risk also require measurable definitions. Without them, two organizations can claim compliance while applying very different safeguards.

Independent access presents another unresolved issue. Closed models can prevent external researchers from examining internal controls. Open deployments can be difficult to inventory after modification and redistribution.

Incident transparency will matter as much as predeployment testing. A company can pass a scheduled evaluation and still experience failures after integrating new tools or changing system instructions.

Public reporting remains uneven across the AI industry. Without comparable incident data, regulators and buyers cannot easily judge which controls work outside laboratory conditions.

The framework recommends sharing information about vulnerabilities, defects, risks, and incidents. Building a useful reporting mechanism will require clear submission rules and protection against incomplete disclosure.

Cross-border coordination is another ambitious part of the document. Version 3.0 supports the United Nations as a central channel for global AI governance and calls for crisis-management mechanisms.

That goal faces political and technical obstacles. Governments differ on national security, content rules, privacy, intellectual property, open models, and acceptable state access to data.

Even shared concern about dangerous capabilities does not guarantee agreement on evidence. Countries may use different evaluations, threat models, and disclosure requirements.

The framework’s international language should therefore be read as a policy position, not proof of global alignment. Its influence will depend on whether China publishes usable testing methods and supports reciprocal technical cooperation.

There is also a risk that broad safety language becomes a barrier for smaller developers. Large companies can fund legal teams, audits, red-teaming, monitoring, and incident response.

Smaller labs may struggle with the same expectations, particularly when standards change quickly. Sandboxes and shared testing resources could reduce that burden, but their accessibility will matter.

The skeptical reading is straightforward. Framework 3.0 accurately identifies several emerging risks, yet identification is easier than implementation.

Its real value will be measured by the standards, evaluations, reporting systems, and enforcement practices that follow. Without those mechanisms, controllability remains a goal instead of a verified property.

China AI Governance Now Extends Beyond the Model

The policy boundary increasingly covers the whole operating environment, including data, memory, tools, infrastructure, users, and physical devices.

Earlier AI governance debates often centered on training datasets and model outputs. Those subjects remain important, but agents make the surrounding system equally significant.

A model’s behavior can change when developers alter system instructions, retrieval sources, memory, plug-ins, or available tools. The base model may remain identical while the deployment acquires new risks.

This means model evaluation alone cannot certify an agentic product. Testing must include the assembled system and the conditions under which people use it.

The framework recommends safety requirements across research, development, deployment, operation, and decommissioning. Decommissioning matters because old credentials, stored memory, and connected services can remain exposed after a product closes.

Data governance remains a central concern. The document calls for protecting personal information throughout training, annotation, use, and output, including deidentification where appropriate.

It also highlights important and core data in areas such as government and finance. Agents operating in these sectors can cross information boundaries faster than human users, particularly when permissions are inherited from connected accounts.

Memory introduces another layer. Persistent agent memory can improve continuity, but it can also retain incorrect, sensitive, or maliciously planted information.

A safe deployment needs rules for what the agent stores, how long it keeps information, and who can inspect or delete it. Memory should not become an invisible record that shapes later actions without user awareness.

Tool governance is just as important. Developers need allowlists, permission limits, action previews, transaction boundaries, and strong authentication for high-impact operations.

An agent should not receive permanent administrative access merely because one workflow occasionally needs elevated permission. Temporary and task-specific authorization reduces the damage from mistakes or attacks.

Human oversight also needs a precise design. A confirmation box does little if users cannot understand the proposed action or if frequent alerts encourage automatic approval.

Effective intervention requires timely, relevant information. Users need to see what the agent plans to do, which data it will use, and what consequences the action carries.

Logging supports both oversight and investigation. Records should capture tool calls, data access, permission changes, model decisions, and human approvals without creating another uncontrolled store of sensitive information.

The framework’s attention to embodied AI raises the stakes further. Physical systems need safe fallback states, environmental limits, and ways to stop operation when communication or perception fails.

A warehouse robot, laboratory system, or connected vehicle cannot rely solely on a text-based refusal mechanism. Safety must exist in software, hardware, operating procedures, and physical controls.

Cybersecurity teams also face a dual role. AI can automate code review, vulnerability discovery, attack detection, and remediation, as the framework recognizes.

Attackers can use the same capabilities to increase the speed and scale of malicious activity. Agentic systems can chain reconnaissance, exploitation, and data handling with less direct input.

This creates pressure to move from reactive defense toward continuous monitoring. Organizations need to evaluate the behavior of their own agents and detect AI-assisted attacks from outside.

The broader scope aligns China AI governance with a general lesson emerging across markets. Safety depends on how a model is connected and used, not only on what the model can do in isolation.

For enterprise buyers, that changes procurement questions. Benchmark scores and model cards are insufficient when a product can reach internal records or initiate transactions.

Buyers should ask which tools the agent can access, how permissions are granted, whether external content is treated as untrusted, and how actions can be reversed.

They should also ask whether the provider discloses incidents and supports independent testing. A polished interface cannot substitute for operational evidence.

Knowledge workers face a similar challenge on a smaller scale. An assistant that organizes research or prepares a report still needs reliable source boundaries and clear handling of private material.

Good information practices become part of AI safety. Teams need traceable sources, deliberate access controls, and review steps that match the consequence of each task.

That does not require rejecting automation. It requires treating agents as participants in workflows, with defined authority and observable behavior.

Three Signals Will Show Whether Framework 3.0 Has Teeth

The next phase depends on measurable standards, real incident reporting, and sector-specific controls rather than another statement of principles.

The first signal is the publication of concrete agent testing standards. These should define evaluation procedures for prompt injection, tool misuse, permission escalation, memory corruption, and loss of task control.

Clear tests would strengthen the framework’s credibility. They would give developers a shared target and let buyers compare safety claims using similar evidence.

Vague assessment language would weaken that result. If every provider chooses its own tests and thresholds, the market will receive many claims but little comparability.

The second signal is a functioning incident and vulnerability reporting system. Framework 3.0 calls for databases and information-sharing mechanisms involving developers, providers, and technical institutions.

A useful system would publish categories, reporting timelines, remediation expectations, and enough anonymized detail for others to improve defenses. It would also distinguish minor errors from serious security events.

Consistent reporting would support the framework’s agile-governance model because authorities could update controls using observed failures. Sparse or inaccessible reporting would leave policy dependent on hypothetical risks.

The third signal is sector-specific implementation. Government, finance, education, healthcare, media, and emergency management need different thresholds for autonomy and human review.

Sector guidance should define which agent actions need approval, which data sources remain restricted, and what evidence organizations must retain. Procurement requirements may become an early route for these controls.

Detailed sector rules would show that the framework is moving from national strategy into operating practice. Generic guidance would leave critical decisions to individual organizations with uneven expertise.

International coordination deserves attention, but it is a longer test. Crisis protocols require agreement about contacts, evidence, confidentiality, and response procedures before a major incident occurs.

The most immediate evidence will come from domestic implementation. China has now published three versions of its AI safety framework in three consecutive years.

That pace demonstrates attention, but repetition alone does not establish effectiveness. The important question is whether the latest risks become testable requirements before agent deployments spread further.

Developers should watch TC260 work plans, draft standards, and public consultation documents. Enterprise buyers should track whether regulators begin referencing agent controls during assessments or procurement.

Knowledge workers should pay attention to product-level permission changes. More capable assistants will increasingly request access to files, accounts, communications, and business systems.

The China AI Safety Governance Framework 3.0 makes a clear judgment: AI safety can no longer stop at the model’s answer. It must cover every step between a user’s request and a system’s action.

The next one to three months should reveal whether that judgment gains operational detail. Look for agent test methods, incident-sharing rules, and sector controls that turn broad principles into evidence.

Before granting an AI assistant broader authority, ask three questions: What can it access, what can it change, and how can you stop it? Those answers will show whether controllable AI exists in practice, not merely on paper.