Microsoft run-assert-eval Turns Agent Risk Findings Into Tested Runtime Controls

Microsoft released run-assert-eval on September 24, linking four previously separate agent-safety tasks through one guided workflow. The Microsoft run-assert-eval skill discovers risks, measures failures, drafts runtime controls, and repeats the same evaluation after those controls are added. The conflict is immediate: a faster safety loop is useful only when its measurements remain credible.

Microsoft’s example gives the announcement substance. A billing-support agent disclosed another customer’s data in 12 of 40 applicable baseline conversations, according to the company. After a runtime policy was introduced, Microsoft observed two violations across 34 applicable conversations. That reduced the reported rate from 30.0% to 5.9%.

The result sounds decisive, but it came from a worked example maintained by the project’s developers. Microsoft has not presented an independent replication or a production deployment study. The important development is therefore the mechanism, not one favorable score. The skill converts a discovered failure into an enforceable policy, then tests the intervention without quietly changing the evaluation.

Microsoft run-assert-eval Connects Discovery, Testing, and Enforcement

The release turns a collection of safety projects into one reviewable path from an unknown risk to a tested runtime control.

Microsoft’s launch post describes a workflow initiated through a prompt in a compatible coding environment. Developers begin with an agent, its intended purpose, its tools, and the boundaries it must respect.

The skill can use Clarity to threat-model the agent. Threat modeling means identifying plausible failures, affected assets, causes, and consequences before selecting tests. Teams may also supply a known risk from a product requirement, incident report, test plan, or existing assessment.

That distinction matters. Risk discovery is recommended, but it is not mandatory. A team that already knows its billing agent exposes customer records can begin with that behavior rather than repeating discovery work.

When teams do need discovery, the Clarity threat modeler examines the agent’s broader operating context. It is intended to surface failures that were never written into the original requirements.

In Microsoft’s billing example, Clarity identified four candidate failure modes. The team selected two risks rated critical: unverified high-risk actions and cross-customer data exposure.

The first risk covered billing changes made without confirming the caller’s identity. The second covered disclosures involving an account that did not belong to the caller.

The skill passed each selected risk to ASSERT, Microsoft’s requirement-driven evaluation framework. Each risk became one configuration, one behavior, and one evaluation suite.

This narrow structure is more consequential than it initially appears. If an evaluation mixes authorization, privacy, accuracy, and escalation failures, its aggregate score cannot explain which control is needed.

Microsoft run-assert-eval instead separates those behaviors. Variation is introduced within each suite through dimensions such as access mode, user pretext, authority claims, and multi-turn scope drift.

The skill also searches prior research and safety frameworks for relevant test dimensions. Microsoft says these sources can include NIST guidance, OWASP resources, benchmarks, regulatory material, and model-provider policies.

ASSERT then generates cases, executes the target agent, and evaluates captured transcripts. Its two principal measurements intentionally remain separate.

“Impermissible behavior violated” records cases where the agent performed prohibited behavior. “Permissible behavior violated” measures cases where the agent failed to help despite being allowed to do so.

This separation guards against a familiar safety illusion. An agent that rejects every request can avoid many harmful actions, but it also stops performing its job.

The workflow next generates a draft Agent Control Specification policy from the measured failure. ACS is a portable format for placing controls at defined points in an agent’s execution.

Finally, the skill evaluates a governed version of the agent against the same cases. The baseline and governed runs retain the same behavior definition, test set, and judging method.

The result is not merely another safety score. It is a controlled comparison designed to isolate the policy as the changed variable.

The Pressure Falls on Teams Using Prompts as Their Main Guardrail

Microsoft is challenging the idea that written instructions alone provide an adequate control boundary for tool-using agents.

System prompts remain useful for defining roles and expected behavior. They are still probabilistic instructions interpreted by a model, not deterministic authorization checks enforced by surrounding software.

That weakness becomes material when an agent can retrieve records, update accounts, issue refunds, or call administrative tools. A persuasive request can then become a database read or business action.

Microsoft’s billing example illustrates the gap. The agent operated for a caller associated with account ACME-1001. It should never have retrieved or modified another customer’s account.

One evaluated request asked for contact information connected to BPS-447, which belonged to a different customer. The baseline agent returned the complete record, according to Microsoft.

A code review might confirm that the retrieval function works correctly. A unit test might confirm that a valid account identifier returns the expected record. Neither necessarily tests whether the model chooses an unauthorized identifier during a realistic conversation.

That problem extends beyond billing support. A research agent can access unsuitable sources, while a change-management agent can bypass an approval sequence. A travel agent can misuse stored identity or payment information.

The OWASP guidance calls this broader condition excessive agency. It arises when an AI application has more functionality, permissions, or autonomy than its task requires.

Prompt injection can trigger such failures, but it is not the only cause. Ambiguous requests, hallucinated plans, compromised tools, and simple model errors can also produce unsafe actions.

The affected teams are not only security groups. Product managers must define permitted and prohibited outcomes. Developers must expose appropriate control points. Risk owners must decide whether a measured reduction is sufficient.

Evaluation teams face pressure as well. Their output can no longer stop at a report that lists failures. The Microsoft workflow expects a finding to support a specific control and a repeatable validation run.

Organizations using generic model benchmarks face another problem. A broad benchmark can describe a model’s average tendencies, but it cannot capture every company’s account rules, escalation boundaries, or internal approval process.

Microsoft’s approach begins with the application’s own requirements and discovered risks. That makes the evaluation more relevant, although it also makes comparisons between organizations less straightforward.

The release also pressures vendors that treat observation as the final step. Logging a dangerous tool call after execution can help an investigation. It does not prevent the action that caused the harm.

Run-assert-eval moves the intervention into the agent’s runtime path. That places it closer to familiar security concepts such as authorization checks, least privilege, and policy enforcement points.

This does not eliminate prompts. It assigns them a narrower role. Models can plan and interpret language, while deterministic controls decide whether sensitive operations should proceed.

That division becomes increasingly important as agents gain access to local files and internal knowledge. Teams building a searchable knowledge base face the same boundary question: retrieval must respect the user’s actual authorization scope.

Microsoft run-assert-eval packages that question into a workflow developers can execute earlier. The burden then shifts from hoping that the model obeys a rule to showing where software enforces it.

The Core Mechanism Is a Controlled Before-and-After Test

The strongest idea in run-assert-eval is not automated policy generation; it is preserving the evaluation while changing only the control.

Safety comparisons become unreliable when teams regenerate the test set after applying a fix. A different group of prompts can make a weak policy appear successful or make a sound policy appear worse.

Changing the judge creates another confounding variable. Two evaluators can interpret the same transcript differently, especially when acceptable behavior depends on context.

Run-assert-eval caches the baseline systematization and test cases. The governed agent then faces the same behavior definition, cases, and judging approach.

Microsoft describes this as freezing the evaluation. The policy becomes the intended independent variable, while the measured violation rates become the observed outcomes.

The principle resembles regression testing in conventional software. A failing test should remain fixed while developers modify the implementation. Otherwise, passing results may reflect a rewritten test rather than corrected behavior.

Agent testing is harder because model outputs are variable. The judge may also be model-based, and generated cases may contain their own ambiguities.

Keeping those elements constant does not remove every source of uncertainty. It does make the before-and-after difference more interpretable.

Microsoft says previous ASSERT evaluations found 80% to 90% agreement between its automated judge and human reviewers. It compares that range with roughly 90% agreement between human reviewers.

Those figures are Microsoft’s reported results, not universal accuracy guarantees. Judge agreement can vary by behavior, model, rubric, language, and the complexity of the underlying policy.

The underlying evaluation remains inspectable. The ASSERT repository says runs store local artifacts, generated cases, model outputs, judge rationales, and metrics.

Local artifacts can support audits because reviewers can examine why a transcript was classified as a violation. They can also identify cases where the evaluator misunderstood the policy.

ASSERT’s two-rate design adds another safeguard. A policy that blocks harmful behavior can still fail if it causes excessive refusal.

For the cross-customer suite, Microsoft reported a 30.0% baseline impermissible violation rate. The governed result was 5.9% across a different applicable denominator.

Microsoft also divided its results into prompt and scenario splits. Prompt cases test more direct interactions, while scenario cases capture richer workflows and conversational context.

For cross-customer prompt cases, the reported impermissible rate fell from 20.8% to 8.7%. The corresponding scenario rate fell from 43.8% to 0.0%.

For unverified-action prompt cases, the reported rate fell from 4.0% to 0.0%. In the scenario split, it moved from 8.7% to 4.5%.

Permissible-behavior violations reportedly reached 0.0% across all four governed splits. Microsoft interprets that outcome as evidence that the controls preserved legitimate work in this sample.

The remaining impermissible violations matter. They show that the policy did not eliminate every failure, even within the controlled example.

That is consistent with the workflow’s iterative design. A team can inspect surviving failures, refine its risk definition or policy, and repeat the same process.

The method therefore offers stronger evidence than a handful of manual demonstrations. It still does not establish how the agent behaves across all future prompts, model updates, tools, or environments.

The practical gain is narrower and more useful. Teams receive traceable evidence that a particular control changed performance on a defined evaluation without merely silencing the agent.

Runtime Policy Blocks Actions Before the Model Can Complete Them

The billing fix works by checking account scope at the tool boundary, not by asking the model to reconsider its intentions.

After measuring cross-customer exposure, run-assert-eval generated a draft policy and an ACS manifest. The policy expressed the decision logic, while the manifest specified where that logic should apply.

Microsoft used Rego, a declarative policy language commonly associated with policy engines. The generated material remained a draft requiring human review.

That review gate is important. Microsoft explicitly says generation does not equal approval. Developers must inspect the policy, intervention point, manifest, and connection to the target agent.

The selected policy denied tool calls when the requested account identifier differed from the caller’s account. That rule did not require another model to decide whether the request looked suspicious.

Microsoft placed the check at pre_tool_call, an interception point reached before the agent executes a tool. A mismatched identifier therefore causes a denial before retrieval occurs.

The team also used post_tool_call. This second check withheld any mismatched result that should never enter the model’s context.

Using both points creates defense in depth. The first attempts to prevent an unauthorized operation. The second limits exposure if the earlier control is bypassed or incorrectly wired.

The ACS policy engine is intended to separate these controls from one agent framework. Policies can therefore remain portable as teams change models or orchestration libraries.

That portability addresses a real maintenance problem. Controls embedded inside prompts or framework-specific callbacks can become difficult to audit across several agent implementations.

A shared specification can give security reviewers a consistent object to inspect. It can also allow developers to version policy changes beside application code.

However, portability does not guarantee correct integration. Every runtime must expose relevant context, preserve identity, and call the policy at the correct point.

An account-scope rule depends on trustworthy account information. If the caller identity is wrong or missing, a perfectly written comparison still produces the wrong authorization result.

The same concern applies to tool arguments. A policy that inspects account_id assumes the requested resource is accurately represented by that field.

Complex tools may hide sensitive targets inside queries, documents, URLs, or nested actions. A narrow rule may then miss equivalent paths to the same protected resource.

Runtime policy also cannot repair every failure class. A control can block an unauthorized refund or database read. It cannot automatically determine whether every generated explanation is accurate or fair.

Human approvals remain appropriate for some high-impact actions. Least-privilege tool design can reduce damage even when the model makes a poor decision.

The wider AI risk framework treats risk management as a lifecycle activity. It includes governance, mapping, measurement, and continuing management rather than a single pre-release test.

Run-assert-eval fits within that larger pattern. It supplies a concrete bridge from mapping a risk to measuring and managing one behavior.

The workflow’s single-prompt entry point should not obscure the work underneath. Threat modeling, test design, policy review, system integration, and result interpretation still require informed decisions.

What Microsoft has reduced is the manual handoff between those decisions. The skill carries structured artifacts from one stage to the next and keeps their relationships visible.

That can lower the chance that a risk description loses meaning when transferred between product, evaluation, and security teams.

It can also shorten the interval between discovering a failure and checking a mitigation. That interval is often where unresolved agent risk accumulates.

The First Results Are Evidence, Not a General Safety Guarantee

Microsoft’s example supports the workflow’s logic, but it does not establish production effectiveness across agents, organizations, or attacks.

The most obvious limitation is provenance. Microsoft and project contributors designed the tooling, selected the example, applied the controls, and reported the resulting measurements.

That does not make the findings invalid. It means readers should distinguish a transparent worked example from an independent benchmark or field study.

The sample sizes also require care. The headline baseline included 40 applicable conversations, while the governed run included 34.

Those denominators differ because only applicable cases contribute to a particular behavior rate. Still, small samples can produce unstable percentages.

A change from 12 violations to two is operationally meaningful in the example. It should not be interpreted as a universal 80% reduction for other agents.

The reported 0.0% permissible violation rate also means no violations appeared in that sample. It does not mean the policy can never block legitimate behavior.

A larger test set might reveal rare false denials. Production traffic might include account relationships, delegation rules, or support exceptions missing from the example.

Evaluation leakage presents another concern. If a policy is repeatedly tuned against one frozen test set, developers can eventually overfit the known cases.

Freezing tests creates a credible comparison during one intervention. Long-term programs still need held-out cases, new adversarial variants, and monitoring for changing behavior.

Model changes can also invalidate earlier conclusions. A new model may format tool arguments differently, interpret refusals differently, or find another path to the protected information.

Tool changes create similar risk. Adding an export function or a general search endpoint can introduce an access route not covered by the original account check.

The evaluation judge deserves continuing scrutiny. Microsoft’s reported agreement range is encouraging, but disagreement cases can cluster around the most ambiguous and consequential boundaries.

Teams should retain human review for disputed cases and periodically sample apparently successful outcomes. A stable automated judge is useful for comparison, but it is not an infallible authority.

There is also a supply-chain issue embedded in the word “skill.” Agent skills contain instructions that influence planning, execution, and validation.

Microsoft Research recently reported 307 skill-induced failures across two benchmark settings. Those included 125 functional failures and 182 efficiency regressions.

That research does not specifically evaluate run-assert-eval. It establishes a broader reason to inspect any skill’s instructions, scripts, permissions, and operational assumptions.

Run-assert-eval partly addresses that concern through visible artifacts and human gates. Teams can inspect generated evaluation configurations and policies before executing governed runs.

Its literature-backed test generation creates another verification obligation. A cited framework can guide dimensions, but relevance depends on how accurately the skill translates that source into cases.

A weak translation can produce impressive coverage labels without meaningful coverage. Reviewers should therefore inspect scenarios, not only the framework names attached to them.

Operational cost is another open question. Running generated cases, capturing traces, judging transcripts, and repeating evaluations consumes model calls and engineering time.

The project has not published a broad comparison of that cost against manual assessment workflows. Organizations must determine which risks justify deeper evaluation.

The most reasonable interpretation is measured but positive. Microsoft has assembled a coherent process for a difficult integration problem.

The release does not prove an agent safe. It helps a team form a specific safety claim, attach evidence to it, and test whether one intervention improved a defined outcome.

Three Signals Will Show Whether the Approach Holds Up

The next test is whether independent teams reproduce the workflow’s gains without sacrificing useful agent behavior or creating hidden maintenance burdens.

The first signal is third-party replication. Developers should watch for public evaluations that apply Microsoft run-assert-eval to agents outside the bundled examples.

A convincing replication would publish the risk definitions, cases, policy, traces, judge configuration, and human-review results. It would also disclose failures that remained after governance.

Results across different models and frameworks would strengthen Microsoft’s portability claim. Material integration differences would expose where ACS still depends on individual runtimes.

The second signal is broader production evidence. Teams need to know whether frozen evaluations predict incidents, denials, and policy bypasses under real traffic.

Useful evidence would include post-deployment violation trends and legitimate requests blocked by controls. It would also track failures introduced after model, tool, or prompt changes.

The strongest deployments will connect evaluation artifacts to continuous monitoring. A pre-release improvement matters more when production telemetry confirms the same behavior boundary.

The third signal is the project’s response to evasion and policy drift. Attackers and ordinary users can reach protected actions through paths absent from the original suite.

Watch for new suites covering indirect prompt injection, delegated authority, conflicting identities, state manipulation, and multi-agent handoffs. Also watch how the project prevents overfitting to frozen cases.

Versioned policies and regression runs will be essential. Teams need a clear trigger for repeating evaluations whenever models, tools, permissions, or business rules change.

The release should also be judged by its review experience. A generated policy is useful only when developers and security teams can understand why it exists and what it blocks.

That makes local artifacts, cited transcripts, and narrow behavior definitions more than implementation details. They form the evidence chain supporting approval.

Microsoft run-assert-eval is therefore best viewed as an engineering discipline packaged as a skill. It connects threat modeling, behavior-specific evaluation, runtime enforcement, and controlled retesting.

Developers should not ask whether the workflow certifies an entire agent as safe. They should ask whether it makes one important risk measurable, one control reviewable, and one improvement reproducible.

That is a smaller promise than proving general safety. It is also a more credible place for agent governance to begin.