NVIDIA Nemotron 3.5 Pushes Open Agents From Data Centers to Desktops

NVIDIA released three Nemotron 3.5 models spanning 120 billion parameters, desktop hardware, and edge devices on October 15. The NVIDIA Nemotron 3.5 expansion challenges the assumption that capable, always-on agents must depend on closed cloud services.

Nemotron 3.5 Super VL handles reasoning, tool use, code, images, documents, and video within one open-weight model. Nemotron 3.5 Bolt brings smaller agentic systems to hardware with 24GB or 4GB of memory. NVIDIA says this range gives organizations a model for each deployment tier, from data centers to local devices.

The central contest is now open, controllable infrastructure versus closed frontier APIs. NVIDIA is not claiming that every local model matches the best hosted service on every task. It is arguing that specialized agents can win through customization, lower inference work, data control, and closer integration with enterprise systems.

That argument has credible supporting evidence, but it also has limits. An independent analytics test found impressive speed and competent routine analysis. The same test exposed failures involving hidden patterns, missing data, and statistical traps.

NVIDIA Nemotron 3.5 Covers Three Deployment Tiers

NVIDIA has turned Nemotron from a collection of models into a deployment ladder for always-on agents.

The Nemotron release introduces Super VL alongside two Bolt variants. Super VL and Bolt 74B-A9B became available immediately. NVIDIA scheduled Bolt E4B for October 23.

Super VL is a 120-billion-parameter mixture-of-experts model. A mixture-of-experts architecture activates only selected parts of a larger network for each token. Super VL activates 12 billion parameters per token, reducing computation while retaining access to wider model capacity.

The model combines text generation with visual understanding, reasoning, instruction following, coding, and tool use. It can inspect webpages, PDFs, tables, charts, images, and video without transferring each input to a separate vision system.

That combination matters for long-running agents. A claims-processing agent might read a scanned document, compare its fields with policy rules, and call an internal tool. Moving between separate models can increase latency and complicate context management.

Super VL instead keeps those steps within one multimodal model. NVIDIA positions it for workflows such as financial document analysis, software engineering, cybersecurity triage, web research, and video review.

The model was trained on about 26 trillion tokens across pretraining and continued pretraining. NVIDIA then used roughly 3.8 million post-training samples and reinforcement learning across dozens of environments.

The company is also releasing checkpoints, datasets, training recipes, evaluation materials, and reinforcement-learning environments. That package gives developers more than downloadable model weights. It provides ingredients for changing how the model behaves inside a specialized workflow.

Bolt 74B-A9B targets a different operating point. It contains 74 billion total parameters while activating nine billion per token. Native 2-bit quantization compresses it to fit within 24GB of memory.

Quantization stores model values at lower numerical precision, reducing memory and computation requirements. The tradeoff is that aggressive compression can damage accuracy unless training compensates for it.

NVIDIA designed Bolt 74B-A9B to run beside other applications on supported RTX systems. The company lists web research, writing, and code debugging among its intended local agent tasks.

Bolt E4B compresses the idea further into a 4GB footprint. It targets conversational and physical agents running on GeForce RTX, Jetson, DGX Spark, or related systems. Suggested uses include voice assistants, interactive characters, and embedded interfaces.

The resulting lineup does not ask one checkpoint to serve every workload. Super VL targets complex multimodal work, Bolt 74B-A9B targets everyday local agents, and Bolt E4B targets constrained edge environments.

That segmentation creates the article’s central tension. NVIDIA is betting that deployment fit can matter as much as a single leaderboard score. Closed API providers must answer with stronger economics, easier customization, or clearer reliability advantages.

Why NVIDIA Nemotron 3.5 Is an Efficiency Play

The release is less about shrinking a chatbot and more about increasing completed work per unit of infrastructure.

Agent performance cannot be judged by token speed alone. An agent might generate text quickly while wasting calls, reading unnecessary context, or choosing the wrong tools. Useful efficiency includes accuracy, response time, memory use, tool calls, and the number of steps required to finish a task.

NVIDIA says Super VL delivers up to five times the throughput of comparable open models on a single GPU. Its technical analysis reports more than 600 output tokens per second through a hosted endpoint under the evaluated conditions.

Those figures depend on NVIDIA’s chosen hardware, serving stack, workloads, and comparison set. They should not be treated as universal performance guarantees. They do show how the company wants buyers to evaluate the model.

Super VL combines several efficiency mechanisms. Its mixture-of-experts design activates 12 billion of 120 billion parameters for each token. That reduces active computation without restricting the total knowledge capacity to a 12-billion-parameter network.

Multi-token prediction lets an internal drafter propose several tokens for verification during one forward pass. Accepting several proposed tokens reduces repeated computation during generation.

NVIDIA also developed LiLiCorr, a speculative-decoding method that evaluates relationships among proposed tokens. Speculative decoding uses a smaller or lighter process to draft text before the main model verifies it.

The company says LiLiCorr can make token generation up to 2.3 times faster on DGX Spark than non-speculative decoding. Again, that measurement describes a defined NVIDIA configuration rather than every installation.

Super VL also uses NVFP4, a 4-bit numerical format designed for NVIDIA Blackwell hardware. Quantization-aware distillation trains the compressed model to imitate its higher-precision counterpart. NVIDIA reports a 99.62% median accuracy recovery across 36 evaluated benchmarks.

The published model card shows that compression does not affect every benchmark equally. Some NVFP4 results slightly exceed the BF16 checkpoint, while others decline. That variation makes workload-specific testing essential.

Efficiency also depends on serving software. NVIDIA says its Dynamo configuration can support up to eight times more users with 85% lower time-to-first-token latency. The measurement assumes an interactivity target of 50 tokens per second per user.

These optimizations connect the model directly to NVIDIA’s broader hardware and software business. Open weights attract developers, while the fastest validated path points toward NVIDIA GPUs, NIM microservices, NeMo tools, and Dynamo serving.

This alignment does not make the model less open. Developers can download and adapt its weights. However, the release shows how an open model can still strengthen a vertically integrated platform.

For enterprise buyers, that relationship has two sides. A pretested hardware and software combination can shorten deployment work. It can also deepen dependence on one vendor’s optimization stack.

Bolt sharpens that calculation. Fitting a capable agent model into 24GB permits local operation on a much wider hardware base. Yet native 2-bit quantization also raises harder questions about accuracy across coding, browsing, planning, and long conversations.

NVIDIA cites Bolt as the leading open model in its PinchBench class and second among open models on BrowseComp. Those results provide useful starting points, not a substitute for deployment tests using real tools and data.

The practical measure is completed, correct work. A slower model that notices a missing record can be more valuable than a fast model that confidently processes bad data.

Open Agents Put Pressure on Closed Cloud APIs

NVIDIA’s strongest challenge is not raw intelligence; it is giving organizations control over where an agent runs and how it changes.

Closed frontier services offer strong general performance and simple API access. Providers manage capacity, model updates, security patches, and much of the serving infrastructure. That convenience remains attractive for teams that do not want to operate models.

However, always-on agents create concerns that ordinary chat use does not. They repeatedly access internal documents, execute tools, retain context, and make decisions over extended workflows. Each additional system boundary creates another place where permissions, latency, or context can fail.

Open-weight models let organizations keep sensitive workloads within approved infrastructure. They can also post-train a model for internal terminology, tools, policies, and procedures.

NVIDIA presents financial services, insurance, healthcare, security, and engineering as early targets. These fields combine sensitive data with specialized rules. A general model may understand the language while still mishandling the organization’s actual process.

Scale AI post-trained Super VL for vendor credit processing at Skydance. NVIDIA says that deployment is expected to reduce inference cost by 40 times and latency by five times.

Ramp is adapting the model to convert receipt images into structured transaction records. According to NVIDIA, Ramp expects up to 50% higher throughput and ten times lower latency for that workflow.

AI21 and Alice used post-training for threat intelligence and vulnerability analysis. NVIDIA reports a 41% accuracy improvement in that specialized setting.

These are company and partner claims. They have not all been independently reproduced across equivalent production systems. Their value lies in showing what specialization can target, not in establishing universal gains.

The broader partner list includes Atlassian, CrowdStrike, Dassault Systèmes, Perplexity, Salesforce, and Synopsys. Their use cases span enterprise search, defensive security, engineering simulation, government workflows, and semiconductor verification.

This is where the contest with closed APIs becomes concrete. A hosted frontier model can remain more capable overall while losing a narrowly defined workflow. Post-training, tool design, data access, and serving speed can outweigh general benchmark leadership.

Open models also provide more control over versioning. A company can test a checkpoint, freeze it, and decide when to adopt an update. Closed services can change behavior through provider-managed model revisions.

Control does not remove operational responsibility. Teams must evaluate safety, monitor outputs, patch serving components, protect model endpoints, and maintain training data. Downloadable weights do not automatically produce a reliable enterprise agent.

NVIDIA tries to bridge that gap with NIM, its packaged inference microservice. NIM provides validated configurations and enterprise support, with security and performance updates scheduled every two weeks.

This creates a hybrid offer. The weights, datasets, and recipes are open, while NVIDIA sells an optimized route for operating them. Enterprises can choose greater independence or a supported NVIDIA deployment.

That choice pressures closed providers to explain what customers receive in exchange for less control. The answers can include higher accuracy, easier operations, broader safety testing, and access to larger models.

NVIDIA must prove that customization does not become an expensive engineering project. A specialized model can outperform a general service only if teams can produce training data, evaluations, and reliable tool integrations.

For knowledge-intensive agents, those evaluations should reflect the information environment employees actually use. Teams building a searchable knowledge base need tests involving real documents, permissions, revisions, and conflicting evidence.

The result is not a simple victory for open or closed models. NVIDIA has made the open route more credible by connecting model efficiency, deployment tooling, and hardware coverage. Closed services still hold an advantage when buyers prioritize convenience and broad capability.

Independent Testing Finds Speed and Blind Spots

Super VL looks fast on realistic analytics work, but speed sometimes comes from examining less evidence.

NVIDIA’s claims need independent evaluation because vendor benchmarks often reflect favorable hardware and task definitions. A third-party test from Plotly offers both confirmation and a useful warning.

Plotly evaluated Super VL on a private data-analytics exam using a synthetic wind-energy dataset. The model passed 30 of 43 questions, producing a 70% overall score. It answered a typical question in 12 seconds, the fastest result among 28 tested models.

The analytics benchmark covered routine analysis, impossible requests, ambiguous questions, statistics traps, data-quality problems, and hidden patterns. Keeping the exam private reduces the chance that answers appeared in training data.

Super VL answered 19 of 20 conventional analytics questions correctly. It handled multistep SQL operations, time-based joins, and merged intervals with strong first-attempt performance.

It also rejected all five impossible requests. That behavior matters because useful enterprise agents must recognize when available data cannot support an answer.

The model resolved four of five ambiguous questions. In one example, it defined the months counted as winter before comparing seasonal turbine performance. That response showed sensible handling of an underspecified request.

The failures were more revealing. Super VL detected none of three planted data-quality problems. It also found none of five hidden patterns that required exploratory investigation.

In one task, the model failed to notice that a dataset lacked the final day of a month. It also missed a week of absent readings while summarizing a quarter of equipment temperatures.

The model passed only two of five questions containing statistical traps. It sometimes accepted an overall relationship without checking whether another variable explained both observed effects.

These misses challenge the idea that faster task completion always indicates a better agent. Plotly found that Super VL made about four model calls and read roughly 150,000 tokens per question.

Nemotron 3 Ultra used more calls and inspected substantially more context. Super VL’s lighter approach improved response time, but it also reduced opportunities to discover concealed problems.

This is the main uncertainty surrounding the NVIDIA Nemotron 3.5 strategy. Efficiency can improve the economics of continuous agents. It can also encourage systems to stop searching before they have enough evidence.

The problem is especially serious when the user does not know what to ask. A model can answer a specific query correctly while failing to identify the missing record or suspicious pattern that changes the decision.

NVIDIA’s own evaluations provide a more favorable view. The company reports Super VL scores of 63.44% on Hermes Bench and 87.05% on LangChain evaluations. Nemotron 3 Ultra scored 56.33% and 84.3%, respectively, under the reported setups.

NVIDIA also reports 80.8% on CharXiv, a chart-reasoning benchmark. Using Python code interpretation raised that result to 84%. The model ranks near the top of the open-model OCRBench v2 leaderboard, according to the company.

Those results and Plotly’s findings can both be true. Benchmarks isolate different capabilities, agent harnesses, tool policies, and stopping behaviors. An agent can excel at chart interpretation while failing to inspect whether a dataset is complete.

Deployment teams should therefore test the entire workflow, not only the checkpoint. They need evaluations for data retrieval, tool selection, error recovery, source coverage, and final-answer verification.

High-stakes agents may require explicit inspection steps. A financial workflow can force the system to check record counts, missing intervals, duplicated rows, and conflicting values before producing a conclusion.

The same rule applies to research agents. A fast answer is less valuable if the agent skips primary documents or fails to surface contradictory evidence. Evaluation must reward complete investigation, not only fluent completion.

Super VL’s open training and evaluation materials make such testing easier to inspect. Openness does not guarantee reliability, but it gives technical teams more room to study and modify failure behavior.

Nemotron 3.5 Bolt Brings the Contest to Local Hardware

Bolt tests whether useful agent autonomy can fit beside everyday applications instead of living exclusively in a cloud cluster.

The 24GB Bolt model matters because that memory level is accessible beyond data centers. NVIDIA says it runs on GeForce RTX PCs, DGX Spark, DGX Station, RTX PRO workstations, and incoming RTX Spark laptops.

Local execution changes the relationship between an agent and its data. Documents, code, images, and application state can remain on the device rather than moving to a hosted model endpoint.

That setup can reduce network delay and support offline use. It can also simplify some data-residency requirements, although local operation does not remove security or compliance obligations.

A local coding agent could inspect a private repository, run development tools, and suggest changes without transmitting the repository to an external model provider. A research agent could organize documents stored on the workstation.

Bolt E4B extends the idea to embedded interfaces. Its 4GB target supports devices where a 24GB model would be impractical. NVIDIA describes voice assistants, interactive characters, and physical agents as intended workloads.

The smaller footprint comes with an expected capability boundary. A 4GB conversational model cannot be assumed to match a 120-billion-parameter multimodal system on complex planning.

NVIDIA’s lineup treats that difference as a routing decision. The edge model handles immediate conversation or simple actions. Super VL handles heavier reasoning, visual analysis, and multistep work.

That architecture can keep common interactions local while escalating difficult work to larger infrastructure. It also introduces coordination challenges involving permissions, context transfer, and consistent behavior across models.

Developers must decide which model sees each request. Poor routing can send private work to the cloud, overload the larger model, or assign a complex task to an insufficient edge model.

Hardware compatibility also deserves scrutiny. Super VL runs across NVIDIA’s data-center and local systems, but its most aggressive optimization paths favor recent NVIDIA architectures. Performance can differ on older GPUs or non-NVIDIA devices.

Support from llama.cpp, Ollama, LM Studio, Unsloth, and vLLM broadens access to Bolt 74B-A9B. These integrations make local experimentation easier and reduce dependence on one serving interface.

Still, the operational experience will matter more than compatibility lists. Developers will judge startup time, memory stability, sustained generation speed, tool calling, context handling, and accuracy after compression.

NVIDIA’s desktop strategy also benefits its hardware roadmap. A compelling 24GB agent gives customers another reason to buy memory-rich RTX systems. Local AI becomes an application category that can drive hardware demand.

That incentive does not invalidate the software. It does explain why NVIDIA is investing across model weights, inference libraries, developer tools, and devices at the same time.

The most important test is whether local agents remain useful after real applications consume part of the available memory. Bolt is supposed to operate alongside other software, not occupy an otherwise empty benchmark machine.

Developers should also examine thermals, power draw, and long-session stability. An always-on model has different requirements from a short demonstration. A system that throttles after sustained use may erase its early speed advantage.

Privacy claims require equal care. Keeping inference local reduces data transfer, but an agent can still expose information through logs, plugins, network tools, or poorly scoped permissions.

Local operation shifts responsibility toward the device owner. Teams must secure model files, tool credentials, stored context, and generated artifacts. They also need a dependable update process.

If Bolt handles those conditions well, it expands the market for private agents. If it only performs well in narrow benchmarks, local users will continue relying on cloud services for serious work.

Three Signals Will Decide Whether NVIDIA’s Bet Works

The next phase depends on independent reliability, sustained local performance, and visible production adoption.

The first signal is broader testing of Super VL and Bolt across independent agent benchmarks. Results should cover full task completion, not only token generation or isolated question answering.

The strongest evidence would come from repeated evaluations using different agent harnesses, hardware configurations, and tool policies. Consistent results would strengthen NVIDIA’s efficiency argument. Large performance swings would show that the serving recipe contributes as much as the model.

Tests should also measure investigation depth. Plotly’s findings show why missing-data detection and exploratory behavior belong beside speed and accuracy. An agent that completes more tasks quickly can still create risk if it overlooks evidence.

The second signal is Bolt performance on real 24GB systems after the October rollout. Developers should watch memory use, sustained speed, application coexistence, and quality under the released 2-bit configuration.

Strong local results would validate NVIDIA’s claim that everyday agentic work no longer requires a remote cluster. Weak tool use or unstable long sessions would narrow Bolt’s role to lightweight assistance.

Bolt E4B will provide another useful boundary test. Its value depends on whether a 4GB model can maintain coherent multi-turn behavior while calling tools and responding quickly on edge devices.

The third signal is production evidence from named partners. NVIDIA has identified many organizations testing or customizing Super VL. The next step is measurable deployment beyond announced experiments.

Useful evidence would include completed workload volumes, verified accuracy changes, operational latency, escalation rates, and human-review requirements. Case studies should compare the specialized model with the system it replaces under similar conditions.

Partner adoption would also reveal how difficult post-training is in practice. NVIDIA provides models, datasets, NeMo tools, and evaluation recipes. Enterprises must still supply domain data, policies, integrations, and governance.

Closed model providers will not stand still. They can reduce latency, add private deployment options, improve model routing, and offer stronger controls for enterprise data.

Their response will determine whether open agents become a distinct purchasing category or remain an option for technically advanced teams. NVIDIA has narrowed the convenience gap, but it has not eliminated it.

The NVIDIA Nemotron 3.5 release establishes a credible spectrum from multimodal data-center agents to compact local assistants. Its strongest evidence concerns efficiency and deployability, not unquestioned superiority.

Developers and buyers should now test one representative workflow from beginning to end. Measure whether the agent finds missing evidence, uses the right tools, and produces a correct result under sustained load. That evaluation will answer the question that model rankings cannot: does an open, specialized agent complete your work more reliably than a convenient closed API?