AWS HCLS Agent Skills Target Healthcare AI’s Reasoning Gap

AWS released 38 open-source AWS HCLS agent skills after finding a troubling gap: foundation models can cite clinical rules while applying them incorrectly. The collection covers 11 healthcare and life sciences domains. AWS says skilled agents won 69.5% to 85.9% of comparisons across a 410-prompt evaluation.

That result challenges a common assumption about specialized AI. More retrieved information does not necessarily produce better judgment. An agent can locate the right guideline, repeat its terminology, and still mishandle thresholds, ordering rules, or evidence categories.

AWS is proposing a different intervention. Its skills encode decision procedures as readable files that an agent loads when a task matches defined triggers. The main contest is therefore not AWS against another cloud provider. It is explicit methodology against general model knowledge.

The approach also carries an important qualification. AWS generated the evaluation prompts and used another model to judge the answers. The results are encouraging, but they are not independent clinical validation. That distinction matters before any skilled agent reaches a consequential workflow.

AWS HCLS Agent Skills Turn Expert Procedures Into Open Files

The release packages domain methodology separately from the foundation model, making specialized reasoning easier to inspect and update.

AWS published the collection on September 16, 2026. Its launch snapshot contained 38 skills spanning 11 domains, including genomics, drug discovery, healthcare operations, clinical data, and medical imaging.

Each skill is a structured SKILL.md file rather than a new model. The files contain decision frameworks, ordered procedures, parameter references, validation criteria, and known failure conditions. YAML front matter tells compatible agents when a skill applies and what it requires.

The collection follows the Agent Skills standard, which defines a portable format for packaging instructions, scripts, and supporting resources. AWS released its collection under the permissive MIT-0 license.

AWS divides the files into reasoning skills and pipeline skills. Reasoning skills describe how to evaluate a problem. Pipeline skills provide commands, parameters, and implementation patterns for completing technical work.

That distinction addresses two different failure modes. An agent might choose the wrong clinical criterion even when its code runs correctly. It might also understand the criterion but generate an invalid command, obsolete parameter, or incorrectly ordered pipeline.

The genomic variant interpretation skill illustrates the first category. It encodes the evidence framework behind the established ACMG guidelines, including classification criteria and population-frequency considerations.

A related variant-calling skill illustrates the second category. It provides implementation guidance for tools such as GATK4, including annotation groups and configuration patterns. The first skill shapes judgment, while the second supports execution.

This structure differs from retrieval-augmented generation, or RAG. RAG retrieves relevant passages from an indexed source before generating an answer. A skill instead supplies the procedure for using information, including ordering rules and error checks.

The two methods can complement each other. Retrieval can provide a current policy document or study. A skill can tell the agent which evidence to prioritize, which exclusions apply, and how to test the conclusion.

That difference is especially relevant in healthcare. A guideline is rarely just a collection of facts. It often contains dependencies, thresholds, exceptions, hierarchies, and documentation requirements that must be applied in sequence.

The files can also be revised without retraining a foundation model. A team can update a policy threshold or implementation warning in readable text. Reviewers can then inspect the exact change.

Portability is another part of the pitch. AWS says the format works with more than 20 agent environments, including Kiro, Amazon Quick, Strands Agents, Claude Code, and OpenAI Codex.

Installation is intentionally simple. Developers can clone the open repository and inspect every file before using it:

A universal skills installer offers a second route:

Kiro users can run the included installer and select a preconfigured HCLS agent. Strands developers can load the skills directory through AgentSkills, then attach it to an agent.

These options lower the integration barrier, but installation is only the first step. Organizations still need to verify each procedure against current policies, local protocols, and intended use.

The collection therefore changes where specialized behavior lives. It moves part of the domain layer from model weights and hidden prompts into versionable files. That makes the reasoning aid visible, but visibility does not guarantee correctness.

Why General Models Can Know the Rule and Still Get the Decision Wrong

Healthcare AI often fails at procedural application, not simple fact recall.

AWS opens with a revealing example. An agent receives a request to classify a TP53 missense variant under ACMG and AMP criteria. The model can identify the relevant framework, yet mishandle its evidence categories.

It might skip population-frequency thresholds or invent computational predictor scores. The response can sound authoritative because its vocabulary is correct. The underlying classification can still be wrong.

This is a harder problem than hallucinating a nonexistent guideline. A fabricated citation creates an obvious verification target. A correctly named framework applied incorrectly can survive a superficial review.

Similar risks appear across healthcare and life sciences. Claims adjudication requires hierarchy resolution and current coefficients. Imaging pipelines depend on the correct order of preprocessing operations. Drug research requires explicit evidence ranking and translational constraints.

Foundation models learn statistical associations across enormous text collections. That training can produce broad factual fluency. It does not ensure reliable execution of every specialized decision process.

System prompts have related limits. A prompt can tell an agent to follow a standard, check its work, and avoid unsupported conclusions. Those instructions rarely contain the complete operating procedure for every domain task.

Loading every possible procedure also creates problems. AWS estimates that putting all 38 skills into one context consumes about 80,000 tokens. Irrelevant instructions can compete for attention and increase inference overhead.

The collection uses progressive disclosure to address that issue. An agent first sees compact metadata describing available skills. It loads detailed instructions only after matching a request to a relevant capability.

AWS also provides a multi-agent configuration for Kiro. A lightweight coordinator routes requests among eight specialists. Each specialist loads roughly 15,000 tokens of domain material instead of the entire collection.

This design creates a new dependency: routing accuracy. A strong clinical procedure provides little value when the coordinator selects the wrong specialist. Cross-domain questions can also require several skills in a deliberate sequence.

Still, the routing model clarifies AWS’s central argument. The company is not claiming that one longer prompt solves healthcare reasoning. It is proposing modular procedures selected for the task at hand.

That approach pressures teams relying on increasingly capable general models without a separate methodology layer. Bigger context windows and stronger benchmarks do not automatically encode local policy or enforce procedural consistency.

It also pressures retrieval-only architectures. A system that retrieves five relevant paragraphs may still lack instructions for reconciling them. The agent needs rules for conflicts, thresholds, missing evidence, and escalation.

Fine-tuning offers another path, but it hides more behavior inside model weights. Updating a single policy may require new training data, validation, and another deployment cycle. A text-based skill can be reviewed through familiar change-control practices.

Skills should not replace retrieval or fine-tuning in every case. Their advantage appears strongest when the missing element is an explicit procedure. They are less useful when success depends mainly on unavailable facts or inaccessible patient data.

A mature architecture can combine all three approaches. Retrieval supplies current evidence, skills supply operating logic, and fine-tuning shapes recurring behavior. The release makes the procedural layer easier to separate and audit.

That separation also supports knowledge blending, where teams connect retrieved context with organized working knowledge. In regulated settings, the crucial question remains how the agent turns that context into a decision.

Three Workflows Show What Structured Reasoning Changes

The worked examples focus on mistakes that remain plausible enough to escape casual review.

AWS demonstrates the skills through drug repurposing, Medicare risk adjustment, and brain-imaging preprocessing. Each example compares a general response with one shaped by explicit domain procedures.

The drug-discovery scenario asks an agent to evaluate approved drugs that affect TGFBR1 signaling for idiopathic pulmonary fibrosis. A generic agent can produce a list of related inhibitors and summarize available literature.

That answer does not necessarily rank evidence consistently. It can mix direct target engagement with weaker pathway associations. It might also discuss biological plausibility without testing clinical translatability.

The skilled version activates drug-repurposing and translational-research procedures. It prioritizes interaction types and selected databases, then ranks candidates through a defined evidence hierarchy.

The workflow also maps the proposed mechanism to fibrosis processes. It examines fibroblast transitions, epithelial changes, and extracellular matrix deposition. Finally, it considers safety history, therapeutic windows, and model relevance.

This does not make the resulting candidate clinically valid. It makes the reasoning path more explicit. A researcher can challenge the ranking criteria or replace a threshold without reconstructing an opaque response.

The second example involves a Medicare Advantage plan with 12,000 members. The task is to calculate Risk Adjustment Factor scores from diagnosis and demographic data under the CMS-HCC Model V28.

A superficially plausible pipeline can omit disease hierarchies, use an outdated model, or sum coefficients before resolving superseding conditions. Those mistakes can inflate scores while leaving the SQL readable.

The skilled agent applies deduplication within the measurement year and resolves hierarchy relationships before aggregation. It also segments members by demographic and eligibility categories.

This scenario shows why procedural correctness matters commercially and legally. CMS maintains official risk-adjustment resources, while payment models and mappings can change by year.

A static skill can itself become outdated. Organizations must therefore bind the skill to a verified policy version and test its assumptions. Portability should not become an excuse for copying old logic into a new payment year.

The third example covers T1-weighted MRI preprocessing for voxel-based morphometry. A research team needs to process scans from 45 healthy adults using FSL and ANTs.

An unskilled agent can name the expected steps but put them in the wrong order. AWS highlights bias-field correction and skull stripping as a consequential dependency.

Correcting intensity inhomogeneity before skull stripping helps protect the brain mask from uneven signal near tissue boundaries. Reversing the steps can distort extracted tissue and bias later analysis.

The skilled response specifies the order, supplies a script, and includes intermediate quality controls. It also lists failure conditions involving orientation, residual shading, neck tissue, and registration.

Here, the value does not come from obscure facts. Many models know the names of the tools and operations. The skill connects those facts through an ordered process with checkpoints.

Across all three cases, the agent becomes less likely to jump directly from a request to a polished output. It must expose assumptions, rank evidence, follow dependencies, and describe failure modes.

That pattern is significant for enterprise adoption. Healthcare buyers often need repeatability and traceability more than an impressive first answer. A procedure that reviewers can inspect provides a clearer validation target.

However, these examples come from the collection’s creators. AWS selected the tasks, procedures, and contrasts. Independent teams must test whether the same gains survive different datasets, organizations, and operational constraints.

The 410-Prompt Test Favors Skills, but the Harness Changes the Result

AWS reports a meaningful advantage, yet the 16.4-point gap between harnesses warns against treating one win rate as universal.

The evaluation used 410 prompts. Of those, 380 targeted individual skills and 30 required cross-skill reasoning. AWS compared each agent with a baseline version that lacked access to the skills.

One configuration used Kiro CLI with automatic model selection. The agent could use a thinking tool and read files. The skilled and baseline conditions ran within that general setup.

The second configuration used the Strands Agents SDK with Claude Sonnet 4.6 explicitly selected. Both conditions received a thinking tool, while the skilled condition also loaded the collection.

AWS used Claude Opus 4.7 as an automated judge. It scored scientific accuracy, coherence, relevance, critical thinking, and actionability on a 100-point scale.

The company emphasized win rate because model judges often compress scores into a narrow range. A win records whether the skilled response scored above its baseline counterpart.

The overall Kiro win rate was 69.5%, with a Cohen’s d effect size of 0.39. Cohen’s d measures the difference between group means relative to their pooled variation.

The Strands configuration produced an 85.9% overall win rate and an effect size of 0.97. That is a much larger reported effect under the second harness.

Critical thinking showed the clearest recurring improvement. Its win rate reached 78.0% in Kiro and 85.1% in Strands. Corresponding effect sizes were 0.65 and 1.03.

Scientific accuracy won 69.3% of Kiro comparisons and 86.2% under Strands. Actionability won 68.0% and 77.3%, respectively.

Those numbers support the mechanism behind AWS HCLS agent skills. The largest benefit appears in applying frameworks, challenging assumptions, and producing executable next steps.

The results also show that agent infrastructure matters. A 69.5% win rate and an 85.9% win rate describe different practical expectations. Skill selection, context handling, model choice, and available tools can all affect performance.

AWS’s technical report acknowledges several limitations. All scoring was automated, and no human experts validated a subset of the answers.

The test prompts were also generated with a model. That creates possible alignment between how questions were framed, how skills were written, and how the judge rewarded responses.

The report says each individual skill had only 10 prompts. That sample is too small for confident skill-by-skill rankings. AWS notes wide confidence intervals and unstable rankings between configurations.

The Kiro configuration also used automatic model selection rather than a pinned version. Both conditions shared that mechanism, but exact model behavior was not fully controlled.

The evaluation is strongest as evidence for the collection-level approach. It is weaker as proof that every skill improves every task, or that an agent can make unsupervised clinical decisions.

Baseline quality further complicates the result. AWS found correlations of minus 0.59 and minus 0.61 between baseline strength and skill benefit. In general, weaker baseline answers gained more.

In Kiro, weak-baseline prompts recorded an 87% skill win rate. Medium prompts reached 79%, while strong prompts fell to 55%. The strong tier’s mean score declined by 0.3 points.

Strands showed a similar pattern. Weak prompts reached a 96% win rate, medium prompts reached 89%, and strong prompts reached 54%.

This creates a practical tradeoff. Skills offer their largest gains when models lack reliable procedures. They can add noise or constraints when a model already handles a task well.

AWS also reports lower variance in some domains. Clinical-data score variation in Kiro fell from 6.8 to 3.3, a 51% reduction. More consistent responses can matter even without a dramatic average improvement.

Consistency is not the same as safety. A systematically wrong procedure can produce consistently wrong answers. Teams need reference datasets, human review, and controls around consequential use.

The FDA’s AI lifecycle guidance offers a useful wider context. Medical AI requires ongoing attention to risk, documentation, monitoring, and change management.

AWS does not claim that these files satisfy regulatory requirements or replace professional judgment. The evaluation measures response quality under controlled prompts, not patient outcomes or regulatory acceptance.

The Real Contest Is Explicit Methodology Versus Plausible Improvisation

AWS is betting that transparent procedures can reduce silent reasoning errors without locking expertise inside one model.

That bet has several strengths. A skill is readable, diffable, and versionable. Reviewers can inspect its thresholds, identify missing exceptions, and trace a changed response to a changed file.

The format also separates domain work from model procurement. A healthcare team can preserve a reviewed procedure while testing different agent environments or foundation models.

This portability can reduce dependence on one vendor’s prompt system. It also makes shortcomings easier to compare. The same skill can be evaluated across two harnesses with controlled prompts.

Open licensing encourages external review and adaptation. A hospital, payer, or laboratory can add organization-specific policies. It can also remove sections that do not apply locally.

Yet customization creates governance obligations. Once an organization changes a skill, AWS’s evaluation no longer validates that version. The team owns its evidence, testing, approvals, and update process.

Readable text can also create false confidence. A procedure may look reasonable to software reviewers while containing a subtle clinical error. Domain experts must review both the content and its activation conditions.

Trigger design deserves equal attention. Broad triggers can load a skill where it does not apply. Narrow triggers can miss the cases that need it most. Incorrect routing can hide behind an otherwise strong skill.

Conflicts are another unresolved area. Two valid skills may reflect different jurisdictions, populations, or policy years. An agent needs a deterministic way to select one or request clarification.

Local protocols can diverge from public standards. A health plan may use organization-specific review rules. A laboratory may apply gene-specific specifications that refine a general framework.

Data access also limits what procedures can accomplish. A skill cannot recover missing clinical history, resolve poor imaging quality, or verify an undocumented diagnosis. It can only instruct the agent to detect or escalate the gap.

Security teams must inspect scripts and dependencies before installation. Open-source skills can include executable resources, commands, or links. Organizations should treat them as code, not harmless prompt text.

Evaluation design must also reflect actual deployment. Teams should not copy the 410 prompts and assume comparable results. Their tests should include local data shapes, ambiguous requests, adversarial inputs, and outdated-policy traps.

Human review should focus on the highest-risk transitions. Those include evidence classification, exclusion criteria, hierarchy resolution, and final recommendations. Generic style checks provide little protection against procedural errors.

The primary advantage is therefore not that skills make an agent an expert. They give experts a concrete artifact to review. That is a more defensible starting point than relying on undocumented model intuition.

The release also reframes model competition. If domain procedures live outside the model, buyers can compare models on how reliably they follow those procedures. Raw recall becomes only one part of the decision.

That shift favors agent platforms with strong routing, context isolation, observability, and evaluation tooling. It pressures systems that offer impressive answers without exposing which instructions shaped them.

What Healthcare AI Teams Should Watch Next

The next evidence must show that skill gains survive independent experts, changing policies, and real deployment conditions.

The first signal is independent replication. Clinical, payer, and research teams should run blinded comparisons using prompts they created themselves. Human experts should judge at least a representative sample.

Replication would strengthen AWS’s central claim if gains remain across different models and organizations. Smaller or inconsistent effects would suggest that the published results depend heavily on the original harness.

The second signal is governance around updates. Healthcare procedures change, and local adaptations multiply quickly. Teams need ownership, review dates, version pinning, approval records, and rollback paths.

A useful skill registry should show which guideline version each procedure implements. It should also record who approved the file and which evaluation suite passed before deployment.

The third signal is production behavior. Watch routing accuracy, escalation rates, error severity, response variance, and expert override frequency. Average benchmark scores will not reveal every operational failure.

Teams should also test whether improvements persist after adding retrieval, patient-specific context, and external tools. Those components can introduce conflicts that do not appear in isolated prompt evaluations.

AWS HCLS agent skills make a credible case that healthcare AI needs more than factual fluency. The release turns clinical and technical procedures into inspectable artifacts, then reports substantial gains across two agent configurations.

The 410-prompt evaluation is promising, but it remains creator-led and model-judged. Its largest lesson is not the headline win rate. It is the sensitivity of results to baseline strength and agent architecture.

Healthcare teams can act now without granting agents final authority. Start with one bounded workflow, select a current procedure, and build a local expert-reviewed evaluation. Compare the same model with and without the skill.

Then ask the question that matters beyond any benchmark: does the skill make errors easier to detect, explain, and correct? If the answer remains yes under independent review, explicit methodology could become a standard layer in healthcare AI systems.