Sandia AI Security Guidance Puts Tri-Valley Startups on Notice

Sandia National Laboratories brought four researchers before Tri-Valley startups with a warning: rapid AI adoption is creating risks that conventional software reviews can miss. The July 29 panel put Sandia AI security methods into a business setting, where small teams often deploy models before establishing formal assurance programs.

The event did not introduce a new product, binding standard, or regulation. Its importance came from the operating model Sandia presented. Startups should examine data, models, integrations, and human decisions as one connected system, then test that system against realistic failures and attacks.

That position creates a clear conflict between deployment speed and evidence of security. Startups compete by releasing features early, learning from users, and changing direction quickly. Sandia approaches AI from high-consequence environments, where an unexplained failure can undermine an entire system.

The panel also challenged a familiar startup assumption. Using a model through a reputable provider does not transfer every risk to that provider. The company building the final application still controls its data access, retrieval layer, tools, permissions, user experience, and response procedures.

That makes AI assurance a product responsibility, not a final compliance exercise. The immediate question for founders is whether their teams can produce evidence about system behavior before customers, regulators, or attackers demand it.

Sandia AI Security Moves From the Lab to Startup Operations

The event translated national security concerns into operational questions that young companies already face.

Daybreak Labs hosted the free Lunch and Learn in Livermore, California, from 11:30 a.m. to 1:30 p.m. The nonprofit incubator supports life-sciences and deep-tech startups in the Tri-Valley region.

The panel featured Amanda Dodd, Gayle Thayer, Philip Kegelmeyer, and Jonathan Crussell from Sandia. Their backgrounds covered national security analysis, secure algorithms, adversarial data analytics, software security, and agentic AI workflows.

Dodd, Sandia’s director of Computation and Analysis for National Security, framed the discussion around vulnerabilities created by fast adoption. She compared the current AI expansion with widespread internet adoption during the 1990s.

Her comparison focused on uncertainty, not nostalgia. Internet connectivity created valuable services while exposing systems to attacks and failure modes that many organizations had not anticipated. AI now adds similar uncertainty inside systems that generate content, recommend decisions, and sometimes take actions.

According to the event program, the discussion covered speed versus security, innovation versus regulation, and risks involving human and AI teams. It also examined how government and private-sector priorities overlap.

Sandia’s subsequent panel account described three business consequences of AI threats: intellectual-property loss, operational disruption, and regulatory exposure. Those outcomes give technical weaknesses a direct financial and strategic meaning.

Thayer, manager of Sandia’s Secure Algorithms department, explained that security attention is moving beyond hardware and conventional software. Researchers increasingly examine the algorithms themselves and the behavior emerging from their use.

That change matters because an AI application can operate as designed while still producing unsafe results. A model might disclose sensitive context, accept a malicious instruction, or behave unpredictably outside its original test conditions.

The application can also fail without anyone attacking it. Ambiguous instructions, unfamiliar data, weak evaluation criteria, or an inappropriate level of autonomy can create serious errors. Conventional vulnerability scanning will not identify every one of those problems.

Sandia’s approach includes red teaming, which means actively probing a system for weaknesses before an adversary finds them. The researchers also benchmark reliability, security, and performance boundaries under defined conditions.

Testing boundaries is different from reporting one favorable accuracy score. A useful evaluation asks where the system stops behaving acceptably, which inputs trigger that change, and whether monitoring can detect it.

The panel therefore did more than list AI risks. It connected risk identification with repeatable testing, mitigation, and secure design. That sequence turns an abstract warning into an engineering discipline.

Nothing in the public account indicates that Sandia certified the participating startups or evaluated their products. The session was an exchange of approaches and concerns, not an audit.

That distinction is important. A two-hour panel cannot establish whether a particular AI system is secure. It can, however, change the questions founders ask before they ship one.

Fast AI Adoption Leaves Startups With an Assurance Gap

Startups face pressure because their release cycles are shorter than the time needed to understand every model behavior.

A young company can add a hosted model to a product without training that model. It can connect customer documents, internal records, or external tools through application programming interfaces within a short development cycle.

The resulting application inherits several layers of uncertainty. Its behavior depends on the foundation model, system instructions, retrieved data, software integrations, permissions, and user input.

A change at any layer can alter the system. A model provider can update behavior, a retrieval index can admit new documents, or a customer can enable a new integration.

Those changes create what teams often call an assurance gap. The product develops faster than the company’s evidence about security, reliability, and acceptable behavior.

The gap becomes more serious when an AI system can act. Agentic workflows are applications that let a model plan steps and call tools with limited human intervention. They can send messages, edit records, execute code, or trigger business processes.

Every additional permission increases the impact of a mistaken or manipulated response. A weak answer inside a chat window is inconvenient. The same answer passed to an execution tool can change data or expose information.

Prompt injection illustrates the difference. An attacker places instructions in user input or external content, hoping the model will follow them instead of the developer’s intended rules.

The current LLM risk list maintained by OWASP identifies prompt injection as a major application vulnerability. It also addresses sensitive information disclosure, supply-chain weaknesses, improper output handling, and excessive agency.

These risks cross organizational boundaries. A model provider secures its platform, while an application developer controls prompts, retrieval, tools, and access rules. A customer then controls user behavior, connected data, and local configuration.

No single participant sees the complete system. That makes responsibility easy to diffuse until an incident occurs.

Sandia’s panel placed responsibility back on the deploying organization. Government can contribute research, frameworks, and independent assessment. However, companies still own the risks created by their products and operating choices.

This responsibility does not require a startup to reproduce a national laboratory. It requires the company to define what safe behavior means for its own use case.

A clinical research assistant and a marketing copy generator should not share identical acceptance criteria. The first handles higher-consequence information and demands stricter evidence, access controls, and review.

Startups also need to separate model quality from system security. A model can perform well on routine tasks while failing under adversarial input. It can answer accurately while revealing information that a user should not receive.

The reverse is also possible. An application can prevent common attacks but still produce unreliable recommendations. Security and usefulness overlap, yet one does not guarantee the other.

That is why a single benchmark score provides weak assurance. Teams need tests tied to intended tasks, foreseeable misuse, access boundaries, and real deployment conditions.

They also need records. A searchable knowledge base can preserve model versions, evaluation results, incidents, and accepted risks across engineering and product teams.

Documentation is not paperwork added after development. It lets a team identify when behavior changed and whether a previous mitigation still works.

The pressure is both immediate and long term. Enterprise buyers already ask vendors about data handling, model providers, access controls, and incident response. Future rules can make that evidence more formal.

A startup that cannot answer those questions may lose deals before suffering a technical breach. In that sense, trustworthy AI becomes part of product credibility.

The Central Tradeoff Is Speed Versus Evidence

Sandia’s message does not require slower innovation, but it does require stronger evidence before AI receives greater authority.

Founders usually treat speed as a survival requirement. Shipping quickly produces user feedback, proves demand, and helps a small company compete against organizations with larger budgets.

Security teams approach the same release from another direction. They want threat models, tests, logs, access boundaries, and a response plan before a system reaches sensitive data.

Both positions contain valid concerns. Delaying every experimental feature until it meets high-consequence standards would burden many startups. Deploying an autonomous system without meaningful tests would expose customers to risks they cannot see.

The practical answer is proportional assurance. A low-impact writing assistant needs different controls from an agent that can modify financial, laboratory, or production records.

Proportional assurance begins with a defined context. The team documents who will use the system, which data it can access, which actions it can take, and which failures are unacceptable.

The company then maps credible threats and ordinary failure modes. Threats include malicious prompts, poisoned data, stolen credentials, compromised dependencies, and attempts to extract private context.

Ordinary failures deserve equal attention. A model can misunderstand a request, invent a source, select the wrong tool, or continue operating after its assumptions become invalid.

Next comes measurement. Teams build evaluation sets from realistic tasks, known edge cases, and adversarial examples. They record expected outcomes before running the tests.

A useful test has an observable pass condition. “The model should be safe” is not measurable. “The assistant must not return records outside the authenticated user’s workspace” is specific enough to test.

Sandia’s red-team approach adds an attacker’s perspective. Testers search for unexpected routes through the complete application, including its instructions, connected data, interfaces, and tools.

The broader Department of Energy laboratory community has described similar work. A multi-laboratory AI safety report says static benchmarks are insufficient for predicting behavior outside familiar conditions.

The report also identifies resource limits. Comprehensive evaluations consume computing capacity, expert labor, and time. New models arrive faster than evaluators can thoroughly assess each one.

That constraint is especially relevant to startups. Small companies cannot test every possible input or reproduce the provider’s internal evaluations.

They can still test the application they control. A focused evaluation should cover the company’s data, tools, user roles, and highest-impact workflows.

Mitigation follows testing. A team might restrict permissions, isolate untrusted content, validate outputs, require human approval, or limit which tools the model can call.

The strongest control often reduces the consequences of failure. If an assistant only drafts an action, a human can review it. If the assistant executes that action directly, the company needs stronger authorization and monitoring.

Human review is not automatically effective. Reviewers can become complacent when most outputs appear correct, especially under time pressure.

Teams should therefore identify the decisions where review matters most. They should also present enough context for a person to recognize a questionable result.

Monitoring closes the loop. Logs should show which model, instructions, data sources, and tools contributed to an important output. Without that context, incident investigation becomes guesswork.

None of these steps creates permanent assurance. Model behavior, application code, data, and attackers all change. Testing must continue throughout the system’s life.

This is where the speed-versus-evidence conflict becomes manageable. A startup can move quickly in a restricted environment, then expand authority as evidence improves.

It can begin with internal users, synthetic data, and read-only tools. Later stages can add customer data or limited actions after the system meets explicit thresholds.

That path treats security as part of product development. It avoids the false choice between blocking every experiment and releasing every capability without limits.

Frameworks Help, but They Do Not Certify a Product

Existing frameworks organize AI risk work, yet they cannot determine whether one startup’s application is safe enough.

The NIST AI Risk Management Framework gives organizations four connected functions: govern, map, measure, and manage. Governance applies across the other functions rather than appearing only at the end.

The framework is voluntary and designed for broad use. Its AI risk framework helps organizations incorporate trustworthiness into AI design, development, deployment, use, and evaluation.

For a startup, “govern” means assigning decision authority and documenting risk tolerance. Someone must decide which use cases are permitted and which evidence supports release.

“Map” means understanding context. Teams identify users, affected parties, data, dependencies, expected benefits, and possible harms.

“Measure” covers testing and analysis. It includes performance evaluation, security testing, uncertainty assessment, and monitoring against defined criteria.

“Manage” turns findings into priorities and actions. A company accepts, reduces, transfers, or avoids risks based on severity and available controls.

NIST also published a generative AI profile in July 2024. The profile adapts the broader framework to risks that generative systems introduce or intensify.

These resources provide vocabulary and structure. They do not inspect a product, guarantee compliance, or certify a model’s behavior.

Sandia’s position appears more technically demanding than merely adopting framework language. Its researchers emphasized red teaming, benchmarking, secure design, and understanding performance bounds.

That emphasis matters because governance documents can become disconnected from software. A risk register might say prompt injection is addressed while the deployed retrieval system still passes untrusted instructions to a model.

Evidence should connect each identified risk to a control and a test. The company should also know which risks remain after mitigation.

Independent assessment can strengthen that process. Sandia described federally funded research and development centers as sources of unbiased technical analysis.

However, the public panel report did not announce a general Sandia certification service for startups. It expressed interest in finding where industry can meet assurance needs and where national laboratories can address gaps.

That is an invitation to collaboration, not proof of a finished pathway. Access, scope, intellectual-property handling, and procurement arrangements will shape whether individual startups can participate.

Sandia already operates programs that connect laboratories, universities, and smaller companies. Its RAIICE program seeks to provide open datasets and technical roadmapping for AI and machine-learning work involving resilient energy.

RAIICE demonstrates a partnership model, but it should not be confused with the July security panel. The panel did not announce that attendees had entered that program.

This distinction exposes the skeptical angle in Sandia’s message. Technical expertise exists, and founders clearly need it. What remains uncertain is how broadly that expertise can reach commercial teams.

National laboratories work under specific missions, contracting rules, and resource constraints. Startups operate on compressed schedules and may need answers within days.

A collaboration model must bridge those rhythms. It must convert research methods into tests that small teams can run, repeat, and explain to customers.

Regulation creates another uncertainty. Panelists reportedly favored clear rules that let industry own AI risks responsibly, but the event produced no proposed legal standard.

Rules can clarify expectations, yet they can also become obsolete as models and applications change. Prescriptive requirements may reward checklist completion without improving real behavior.

Outcome-focused rules create a different problem. They can tell companies what result to achieve without providing affordable ways to demonstrate it.

The best near-term signal will be practical evidence. Public test methods, reusable evaluation tools, accessible datasets, and documented startup collaborations would show that the partnership vision is becoming operational.

Until then, companies should not claim that framework adoption proves safety. They should describe the controls they use, the tests they ran, and the limitations they still observe.

What Founders and Buyers Should Watch Next

The next test is whether Sandia’s risk language produces repeatable practices, credible partnerships, and clearer market expectations.

The first signal is a concrete collaboration pathway for small companies. Sandia said it wants to identify where industry can meet assurance needs and where national laboratories can fill gaps.

A useful pathway would define eligible projects, available expertise, data-handling requirements, expected deliverables, and engagement timelines. Public case studies would make the model easier to evaluate.

If those details emerge, they will strengthen the panel’s central claim. They would show that national-laboratory methods can move beyond discussion and support commercial development.

If no pathway appears, the event will remain valuable outreach but offer limited operational help. Startups would still need to assemble their own testing programs or hire outside specialists.

The second signal is technical validation tied to real applications. Founders should watch for published evaluations covering prompt injection, data leakage, model manipulation, unsafe tool use, and behavior outside normal test conditions.

The most useful work will report failure conditions and residual risks, not only favorable scores. It should explain what changed after testing and whether the mitigation survived another attack cycle.

Such evidence would strengthen the case that trustworthy AI creates a competitive advantage. Buyers can compare vendors more confidently when companies provide repeatable results.

A lack of evidence would weaken that argument. Security language alone does not distinguish one product from another, especially when every vendor makes similar promises.

The third signal is movement from broad frameworks toward clearer customer and regulatory expectations. NIST is revising AI RMF 1.0 and developing additional profiles for cybersecurity and critical infrastructure.

Founders should track whether enterprise questionnaires, procurement requirements, insurance reviews, and sector rules begin asking for similar evidence. Convergence would reduce uncertainty about which controls deserve early investment.

Fragmentation would create a heavier burden. A startup might face different evaluation formats from every customer, regulator, and partner.

Buyers have a role in shaping that outcome. They should ask vendors which models and data sources support a workflow, which actions the system can take, and how the vendor tests access boundaries.

They should also ask what happens after a model update. A previous evaluation offers limited protection if a provider changes behavior or the application gains new tools.

Developers should treat every added permission as a new security decision. Connecting an assistant to email, source code, customer records, or laboratory data changes the consequences of failure.

Knowledge workers should understand those boundaries as well. They need clear signals about which information can enter an AI system and which outputs require independent review.

Sandia’s panel did not settle these questions. It made them harder for startups to postpone.

The most durable lesson is that AI security covers the complete operating system around a model. Data, instructions, software, permissions, users, and monitoring all influence the result.

That lesson also clarifies what founders should do now. Define the system, rank its highest-impact failures, test those failures, reduce permissions, and retain evidence.

Then ask whether each new capability deserves the authority it receives. A persuasive demo is not an assurance case, and a provider’s reputation is not a substitute for application testing.

Sandia AI security guidance will matter most if companies turn that discipline into routine product work. Founders should start by selecting one consequential workflow and documenting its risks, controls, tests, and unresolved limits.