Orca Security Discovery AI Cuts the Cost of Natural Language Security Queries

Orca Security Discovery AI now converts plain-English security questions into Sonar queries while using 93% fewer input tokens than its initial design. Orca also reports 90% lower query costs and 75% lower latency after rebuilding the system on Amazon Bedrock.

Those numbers describe more than a routine model upgrade. Orca replaced one enormous prompt with a nine-stage pipeline that decides when rules, retrieval, smaller models, or deeper reasoning should handle each request.

The result pressures a common enterprise AI strategy: place every schema, example, and instruction into one large context window. Orca’s experience suggests that more context can increase cost while weakening relevance and slowing the application.

Its alternative treats the language model as one component inside a controlled query compiler. Deterministic matching handles obvious inputs, retrieval supplies proven examples, and validation checks generated filters before returning them.

That design matters because natural language security queries operate against sensitive and highly structured information. A fluent answer has little value when the generated filter points to the wrong asset, crosses an authorization boundary, or omits a critical condition.

Orca Replaced One Giant Prompt With a Production Pipeline

The central change was architectural: Orca stopped asking one model to understand its entire security schema during every request.

Orca’s Sonar query system spans more than 900 cloud infrastructure and application models. These models represent resources such as compute instances, storage buckets, identities, vulnerabilities, and the relationships connecting them.

Before Discovery AI, users needed to construct precise JSON filters. They also had to know exact model names, navigate nested relationships, and choose among 18 filtering operators.

That requirement created a usability barrier for analysts who understood the security question but not Sonar’s full schema. Orca therefore built a natural language interface that translates an analyst’s request into a valid filter.

The first proof of concept used the most direct generative AI approach. Orca loaded more than 900 model definitions, relationship types, and example queries into one prompt for a large model.

According to the company, design partners and internal users liked the output quality. However, the design consumed roughly 2 million input tokens for a comparable workload and took around 60 seconds.

Complex requests also exposed accuracy limits. Questions involving several resource models, nested relationships, or multiple conditions often required additional refinement.

Orca and AWS then developed the production nine-stage pipeline. It has reportedly handled thousands of daily queries since March 2026, with response times near 15 seconds.

The pipeline starts by combining the request with organizational context. That context can include connected cloud providers, request metadata, and details needed to interpret the user’s environment.

Next, a deterministic matcher looks for structured identifiers. These include IP addresses, CVE identifiers, file hashes, resource names, URLs, and alert IDs.

A matching identifier can trigger a predefined filter without an expensive model call. This fast path keeps predictable work away from a probabilistic system.

Requests that need interpretation pass through a guardrail. Orca says this stage blocks prompt injection attempts, irrelevant questions, and unsupported inputs before they reach query generation.

The system then searches a library of validated examples. Amazon Titan embeddings retrieve 20 semantically similar candidates, and Cohere Rerank 3.5 reduces that set to five.

A model selector narrows the 900-plus Sonar models to those relevant for the current request. The generator receives detailed schemas only for that smaller group and their relationships.

Anthropic Claude Haiku 4.5 powers the selection and generation stages. Amazon Nova 2 Lite handles guardrail and validation tasks where Orca prioritizes faster, lower-cost inference.

The generated filter still does not automatically become the final result. Static checks resolve each model, field, and operator against the selected schemas.

Orca also checks the filter against the Sonar API. When validation fails, the error identifies the offending element and returns it to the generator for correction.

This loop turns generation into a constrained process. The model can propose a filter, but deterministic systems decide whether that proposal conforms to the available schema.

Orca Security Discovery AI Makes Context Smaller on Purpose

Orca Security Discovery AI improved its reported economics by selecting context, not by expanding it.

The initial prototype treated Sonar’s full schema as mandatory context. That choice appeared reasonable because the model needed to understand hundreds of resource types and their relationships.

The problem was that most questions touched only a small portion of that schema. Sending every definition on every request increased token use without making every definition useful.

Orca’s model selector now works from compact descriptions of the available models. Each description summarizes a model’s name, purpose, important fields, and important relationships.

The selector identifies which models a question requires. The generator then receives the full schema only for those selections, plus the relationships needed to connect them.

That change reduced reported input usage from about 2 million tokens to 140,000 per comparable workload. It represents the stated 93% reduction.

Orca attributes the 90% cost reduction to lower token volume and per-stage model allocation. Different tasks no longer default to the largest available model.

Latency reportedly fell from about 60 seconds to 15 seconds, a 75% reduction. The faster path still contains multiple stages, showing that stage count alone does not determine response time.

A long single inference can be slower than several narrow operations. Rules and retrieval can also bypass generation when the system recognizes a known pattern.

This is the core mechanism behind the reported gains. Orca reduced the amount of material requiring deep reasoning, then reserved generative work for unresolved portions of the request.

The retrieval design follows the same principle. Pure top-K semantic search was fast, but Orca found that it often returned near-duplicate examples.

Those duplicates occupied context without adding much coverage. Maximal Marginal Relevance increased diversity, but it sometimes moved the most relevant example away from the top.

Orca settled on retrieving 20 nearby examples and reranking them to five. The larger first pass emphasizes recall, while reranking focuses the final context on stronger matches.

A high-confidence match can reuse a known-good filter directly. Otherwise, the five examples show the generator how validated questions were previously translated.

This approach resembles case-based reasoning more than unrestricted generation. The model works from a compact schema and examples drawn from successful historical behavior.

The architecture also keeps model choices replaceable. Amazon Bedrock provides access to several model families through a shared service layer, reducing separate integration work.

That flexibility matters because the fastest or most accurate model for one stage may change. Orca can reportedly replace a stage model without redesigning the entire application.

The broader lesson is not that every system needs nine steps. It is that context should earn its place in a production prompt.

Schemas, examples, and conversation history impose costs even when a model supports a large context window. Irrelevant material can also compete with the evidence that matters.

Natural Language Security Queries Still Need Hard Boundaries

A conversational interface reduces query-language friction, but it cannot replace authorization, validation, or security expertise.

Natural language security queries can help an analyst move from an investigation idea to a structured filter. They do not guarantee that the question itself is authorized or complete.

Orca’s pipeline includes request context, guardrail screening, constrained generation, and two forms of validation. That sequence acknowledges the risk of treating model output as executable truth.

Amazon Bedrock guardrail evaluation can inspect prompts and responses against configured policies. Available controls include content filters, denied topics, sensitive information filters, and prompt-attack detection.

However, a guardrail is not an authorization system. It can classify an input while still lacking the user identity and resource permissions needed for an access decision.

AWS security guidance makes that distinction explicit. Its authorization controls recommend enforcing data access outside the language model and propagating the user’s identity through connected systems.

That separation is especially important for cloud security data. A query might expose asset names, vulnerabilities, account relationships, network paths, or internal configuration details.

An assistant with broad backend access can become a confused deputy. That occurs when a less-privileged user gains information through a more-privileged intermediary.

The safe architecture must constrain what data reaches the model. It must also ensure that the resulting Sonar query runs with permissions appropriate for the requesting user.

The published architecture says Orca combines requests with organizational and cloud-provider context. The post does not provide a detailed account of tenant isolation or user-level authorization enforcement.

That omission does not establish a weakness. It simply means the reported performance results should not be read as a complete security assessment.

Validation creates another important boundary. Orca checks syntax first, then verifies the generated filter through its Sonar API.

The generator receives structured errors when a check fails. It can revise the specific field, operator, or relationship instead of starting another unconstrained guess.

This approach reduces one class of hallucination: references to schema elements that do not exist. It does not establish that every valid query accurately captures the analyst’s intent.

A filter can be syntactically valid and semantically incomplete. It might select the correct model but omit an exception, relationship, time boundary, or environmental condition.

That problem becomes more serious when a concise request hides ambiguity. “Show exposed databases” can depend on definitions of exposure, database type, network reachability, and accepted risk.

A strong interface should reveal assumptions or show the generated filter for review. The AWS post focuses on filter generation and validation, not the complete review experience.

Security teams should therefore distinguish query validity from investigative correctness. The former is machine-checkable against schemas, while the latter often requires domain judgment.

The system’s deterministic exits also deserve attention. Reusing a validated example can lower latency, but the match threshold must prevent superficially similar questions from sharing an unsuitable filter.

Orca says it routes low-confidence queries to domain experts. That feedback process helps, although its effectiveness depends on sampling, review coverage, and correction quality.

Natural language access can broaden participation in security investigations. It also shifts responsibility from query authors toward the platform designers who encode these safeguards.

The Real Opponent Is the One-Model, One-Prompt Design

Orca’s most meaningful comparison is not with another vendor, but with the monolithic prototype it replaced.

Generative AI products often begin as a prompt connected to a capable model. That structure works well for proving whether a model understands the task.

It becomes harder to defend once the application faces real traffic, changing schemas, strict latency requirements, and sensitive organizational data.

Orca’s baseline exposed that transition. The prototype produced encouraging answers but sent a vast schema into every request and still struggled with complex relationships.

The production system moves several responsibilities outside the generator. Regular expressions recognize exact identifiers, retrieval finds examples, reranking reduces noise, and validators reject malformed filters.

This division produces an important reversal. Adding components reduced overall cost and latency because those components prevented unnecessary generative work.

The pipeline also challenges model-centric procurement. Choosing the largest model did not solve the production problem by itself.

Orca instead assigned models according to task requirements. Reasoning-heavy stages received Claude Haiku 4.5, while narrower screening and validation stages received Nova 2 Lite.

Amazon Titan created embeddings for semantic search. Cohere Rerank 3.5 scored the retrieved examples before generation.

That mix creates operational complexity. Teams must monitor several models, retrieval behavior, validation loops, and dependencies instead of one API call.

Yet a single-call architecture hides complexity inside a large prompt. It does not remove schema management, evaluation, authorization, or error handling.

Earlier AWS work on natural-language investigations showed a related pattern for Amazon Security Lake. An agent translated analyst questions into SQL and supported follow-up analysis over security logs.

Orca’s implementation targets its own Sonar data graph and domain-specific filters. Its distinguishing feature is the aggressive reduction of schemas and examples before generation.

This pattern can apply beyond cloud security. Enterprise systems often expose hundreds of objects, fields, relationships, and permitted operations.

Customer support platforms, analytics tools, developer portals, and knowledge systems face the same temptation to place every definition into the prompt. That approach can work during demonstrations and fail under production economics.

The alternative resembles a compiler pipeline. The system parses the request, identifies relevant structures, retrieves precedents, generates an intermediate representation, and validates it.

The language model handles ambiguity and translation. Deterministic components handle enforceable rules.

That distribution also makes failures easier to locate. A team can determine whether retrieval missed a relevant example, selection omitted a schema, or validation rejected an operator.

A single prompt often produces a less useful failure signal. The answer is simply wrong, expensive, slow, or some combination of all three.

Orca says its evaluation harness measures correctness, completeness, and semantic equivalence to reference queries. It also divides results into slices based on question and model categories.

That is more informative than one aggregate score. A system can perform well overall while failing on rare multi-cloud relationships or complex vulnerability conditions.

The design pressures other enterprise AI teams to report system-level results. Model benchmark scores alone cannot explain application latency, retrieval quality, validation rates, or correction workloads.

The Numbers Are Strong, but the Accuracy Evidence Is Incomplete

The cost and latency improvements are specific, while the public accuracy claim lacks the same level of measurement detail.

AWS and Orca provide clear before-and-after figures for tokens, cost, and response time. They do not publish the evaluation dataset, sample size, pass rate, or exact accuracy improvement.

The post says accuracy improved for complex, multi-model queries. It also describes an LLM-as-judge system that grades correctness, completeness, and semantic equivalence.

An LLM-as-judge uses another model to evaluate generated results. It can support rapid iteration, but its conclusions depend on the rubric, reference answers, and judge reliability.

Orca supplements automated scoring with internal specialists and customer feedback. Product managers, support engineers, and security researchers can flag outputs and add corrected examples.

Low-confidence production queries reportedly enter a daily labeling queue. Corrections become available to subsequent retrieval runs without requiring a software deployment.

Orca’s director of AI, Or Gabay, called that labeling interface the team’s most important long-term quality investment. The statement highlights a human process rather than another model upgrade.

That process is credible as an improvement mechanism. It still leaves several questions unanswered for outside buyers and engineering teams.

The public post does not disclose how often generation reaches the retry limit. It does not report how many requests take deterministic or example-reuse paths.

It also does not separate median latency from tail latency. A 15-second figure can describe a typical workload while concealing slower complex requests.

The phrase “comparable workload” needs more definition. Token and cost reductions depend on request mix, selected models, cache behavior, output length, and the number of retries.

The reported cost reduction also compares the production pipeline with Orca’s own initial baseline. It is not a benchmark against other cloud security products or query-generation systems.

Readers should treat the percentages as company-reported operational results. The coauthored AWS case study provides technical detail, but it is not an independent audit.

The March 2026 production date and thousands of daily queries provide a useful adoption signal. They do not reveal unique users, acceptance rates, abandonment, or analyst time saved.

Those metrics would help determine whether natural language changes investigation behavior. Lower inference costs matter, but user value ultimately depends on better and faster security decisions.

The system’s feedback loop introduces another tradeoff. Production examples improve coverage, yet corrections can encode inconsistent interpretations unless experts follow a shared policy.

A growing example bank also requires governance. Teams need versioning, quality review, removal procedures, and protection against sensitive information entering retrieved examples.

AWS guidance for generative AI applications recommends limiting retrieved data before it reaches a model. Its security controls emphasize identity propagation and access enforcement at the data layer.

These concerns do not negate Orca’s results. They define the evidence required before another organization treats the architecture as a reusable template.

The strongest public conclusion is narrow but useful. Orca reports that context selection, model specialization, retrieval, and validation dramatically improved its original design.

The evidence does not establish a universal reduction for every schema-heavy agent. Teams must measure the pattern against their own workloads and authorization requirements.

What to Watch After Orca’s Amazon Bedrock Deployment

The next test is whether Orca can preserve quality as schemas, user behavior, and model versions continue changing.

The first signal is published accuracy detail. A future technical report should disclose dataset size, query categories, pass thresholds, and human-review agreement.

Slice-level results would be particularly valuable. Multi-model relationship queries appear to be the hardest category and the strongest test of Orca’s architecture.

More transparent evaluation would strengthen the claim that the new design improved accuracy, not only efficiency. Rising correction or retry rates would weaken that conclusion.

The second signal is sustained production adoption. Orca says Discovery AI handles thousands of daily queries, but longer-term usage patterns will show whether analysts trust the interface.

Useful indicators include repeat usage, accepted filters, manual edits, time saved, and the share of questions resolved without escalation. These measures connect model behavior to security operations.

The third signal is how the pipeline handles change. Sonar’s schemas will evolve, customers will ask unfamiliar questions, and foundation models will receive new versions.

Orca’s model-agnostic design should make replacements easier. However, each change still needs regression testing across retrieval, generation, validation, and authorization boundaries.

The continuous evaluation harness is therefore as important as the initial architecture. Every prompt, model, retrieval setting, and example can alter behavior in unexpected slices.

The system also needs to remain useful against changing threat behavior. Analysts may map investigations to the cloud threat framework while attackers adopt new paths and resource combinations.

A static example library would decay under those conditions. Orca’s expert labeling loop is intended to keep the retrieval bank aligned with emerging questions.

For engineering teams, the immediate action is to examine where large prompts contain material that most requests never use. That material may belong behind selection, retrieval, or deterministic routing.

Teams should also separate three tests: whether a query is valid, whether it represents the user’s intent, and whether the user may access its results.

Orca Security Discovery AI offers a notable production case for that layered approach. Its reported efficiency gains are substantial, but its deeper contribution is a change in system design.

The question now is whether Orca publishes enough operational evidence to show that this efficiency survives evolving schemas and adversarial use. That evidence will determine whether the pattern becomes a broader standard for enterprise agents.