Anthropic and AE Studio Introduce GRAM to Control Dual-Use Knowledge in AI Models

Anthropic and AE Studio introduced a new technique called GRAM that adds removable modules inside large language models. The modules hold specific dual-use knowledge during training. Teams can delete the modules later to strip out those capabilities.

GRAM stands for Gradient Routing Auxiliary Module. Engineers insert a small set of extra neurons at every transformer layer. During training the model learns to route topics such as virology, cybersecurity, nuclear physics, and specialized programming languages into those extra neurons instead of spreading them across the whole network.

After training the team either removes the modules or keeps them. Removal eliminates the targeted knowledge while preserving overall model capability. Retention keeps the knowledge available only for trusted deployment settings.

How the modules function during training

The research team trained models ranging from 50 million to 5 billion parameters on both synthetic and real data. In every layer they added one auxiliary module sized to capture the target knowledge without interfering with other representations. Gradient updates are directed so that dual-use content preferentially updates weights inside the added module. General knowledge continues to update the base model weights.

The result is a clean separation visible in activation patterns. When the auxiliary module is later zeroed out or deleted the model shows near-zero performance on the restricted topics while accuracy on standard benchmarks stays intact.

Data filtering achieves similar levels of suppression on the tested tasks. GRAM matches that benchmark while leaving the base model untouched. The added modules can be stored separately and reinserted on authorized hardware when needed.

Comparison with earlier unlearning techniques

Earlier unlearning methods require additional fine-tuning after the model is complete. Those steps often leave detectable traces that allow partial recovery of the removed content through targeted prompting or continued training. GRAM avoids that recovery path because the knowledge never existed in the base weights.

Experiments showed that models with removed modules resisted standard relearning attacks more effectively than models treated with post-training unlearning. The separation created by gradient routing at each layer appears to reduce the residual pathways that usually allow knowledge to re-emerge.

The method also avoids the need to retrain from scratch when new dual-use categories are identified. A new module can be added and trained on the previously isolated category without changing the rest of the model.

Performance results across model sizes

The team evaluated GRAM on models from 50 million parameters up to 5 billion parameters. On every scale the accuracy drop on general benchmarks stayed within measurement noise after module removal. Simultaneously accuracy on held-out dual-use tasks fell to the level observed in fully filtered training runs.

Tests used mixtures of synthetic data designed to stress isolation and real web-derived data that contained both restricted and unrestricted content. The same pattern held in both regimes. Larger models showed slightly cleaner separation, consistent with increased capacity for routing.

The experiments did not introduce new safety risks during the training phase. The auxiliary modules remained under the same compute and data constraints as any other training run.

Practical deployment options

Organizations that need to deploy models in restricted environments can strip the modules before release. Models intended for internal research or government use can retain the modules under access controls. The same base weights therefore support multiple deployment profiles.

Storage of the auxiliary modules adds negligible overhead. Each module occupies a fraction of the total model size because it focuses on a narrow slice of knowledge. Teams can version the modules separately from the base model.

The approach also simplifies auditing. Reviewers can inspect only the content of the auxiliary modules rather than searching the full weight set for hidden knowledge.

Remaining questions for the field

The paper leaves open how many distinct modules a single model can support before routing interference appears. It also does not test whether modules trained on one model transfer cleanly to another model of different size or architecture.

Longer-term relearning attacks that combine module removal with subsequent continued pretraining on new data were not fully explored. The current results show resistance under standard attacks but do not rule out future adaptive methods.

Independent replication on larger frontier-scale models will be needed to confirm whether the separation remains clean when parameter counts reach hundreds of billions.

What to watch next

The next three months will show whether other labs adopt similar routing techniques or publish follow-up work that extends the module concept to multimodal models. Any new attack that succeeds in recovering knowledge from removed modules will be closely studied.

Regulatory discussions about dual-use model releases may reference GRAM as a concrete technical option for controllable deployment. Whether standards bodies include module removal as an accepted control measure remains to be seen.

Anthropic and AE Studio have published the core method and evaluation code. Wider testing by the research community will clarify both the strengths and limits of this isolation approach.

AI teams that manage internal knowledge bases already face similar questions about separating sensitive and general information. Structured routing methods like GRAM may eventually influence how those teams organize their own retrieval systems for specialized domains.