top of page

DeepSeek V4.1 Flash Runs on 12 V100s, Builder Says

3 hours ago
11 min read
Image from the source report. Source: www.tomshardware.com.

DeepSeek V4.1 Flash now runs every routed expert on twelve used Nvidia V100 cards, according to lawyer and server builder Nick Mermiges. His November 20, 2026, account reports code generation reaching 184 tokens per second. The claim puts aging server hardware against published DGX Spark results, although the comparisons use different software and model representations.

The upgrade expands his earlier eight-card machine. Its central change is memory capacity: every expert can remain on the graphics cards without additional expert compression or transfers from system memory. For someone repeatedly questioning long documents, that changes the balance between waiting, hardware complexity, and keeping work inside an office.

These remain measurements published by the builder, without independent replication established in the reviewed sources. They also describe a narrower contest than general AI performance. A machine that answers one person quickly does not automatically serve a team efficiently, produce better answers, or offer lower operating costs.

DeepSeek V4.1 Flash Crosses the Memory Threshold

The consequential upgrade is fitting every routed expert in graphics memory at the selected four-bit representation.

Mermiges describes twelve V100-SXM2 cards arranged across three boards, with four cards per board. Each card has 32 GB of memory. His earlier eight-card arrangement lacked enough usable capacity for the complete expert collection in Nvidia’s selected format.

An expert is a specialized parameter group that the model selects during processing. Keeping every expert available does not mean activating every expert for each token. It means the server avoids the particular compromise of storing some experts elsewhere or compressing them further.

According to the builder’s twelve-card architecture, all 15,360 routed experts occupy card memory. The expert collection requires 278.9 GiB in the selected representation. GiB measures memory using binary units, so readers should avoid directly comparing it with differently labeled capacity figures.

This distinction makes the story more specific than another enthusiast adding graphics cards. The additional board crosses a capacity boundary. Software tuning alone cannot create the missing storage, even when it improves how efficiently existing memory gets used.

The claim also needs a precise boundary around the word “all.” Mermiges reports every routed expert in NVFP4, Nvidia’s compact four-bit floating-point representation. His documentation identifies different representations for other model components. Describing the entire model as uniformly stored or computed in four-bit precision would overstate that account.

For readers evaluating private AI, the important question is what the memory change removes from the working path. If a request needs an expert stored outside card memory, moving that data introduces another dependency. Further compression introduces a different compromise that requires its own quality evaluation.

Neither compromise automatically makes a system unsuitable. Their importance depends on the application, acceptable latency, and evidence about output quality. Mermiges chose additional hardware because he wanted the selected expert representation resident on the cards.

That choice sharpens the main comparison throughout this story: a custom system optimized for an individual’s workload versus compact systems serving broader needs. It does not establish that additional hardware is always the best answer.

The upgrade should therefore be judged as a distinct configuration. Results from the earlier eight-card machine do not automatically describe this twelve-card version. Conversely, improvements cannot all be credited to capacity when software and operating settings also changed.

For an independent evaluator, that is the first useful dividing line. Verify what resides in memory before interpreting the speed numbers. Otherwise, two apparently similar model deployments can represent meaningfully different compromises.

The Practical Gain Is Less Waiting on Documents

For document work, the delay before an answer starts can matter more than the fastest generation headline.

Mermiges published his principal measurements as November 19 tests, with thinking disabled and 1,024 output tokens requested. Tokens are the text units a model processes and generates. The reported figures average two runs, which makes them an initial measurement set rather than a broad statistical study.

His document latency measurements report a 128K code prompt beginning its answer after approximately 28.1 seconds. With that document already cached, the reported wait falls to approximately 0.4 seconds. A cache preserves reusable processing state from material the system has already read.

Those figures describe different situations. The shorter wait does not mean the machine reads a new document almost instantly. It means a follow-up request can reuse work, subject to the cache remaining available and the request matching reusable content.

That difference matters in the workflow Mermiges describes: private first-pass analysis of his own case files. A professional often returns to the same material with successive questions. Repeatedly paying the full document-processing delay would interrupt that interaction, even if the eventual answer appeared quickly.

Consider a hypothetical sequence involving a lengthy contract collection. The first request asks for obligations, the next asks about exceptions, and another requests supporting passages. If each request reuses the document state, the user experiences a different tool than one repeatedly starting from scratch.

This is a workflow interpretation, not a measured legal productivity result. The published timing does not establish how much professional time the system saves. Reading answers, verifying references, and correcting mistakes remain part of the work.

The distinction between initial processing and answer generation also prevents a common benchmark misunderstanding. Prefill is the initial processing of an input prompt. Decoding is the subsequent generation of output tokens. A system can excel at either stage without leading at both.

Tom’s Hardware’s server upgrade coverage identifies reduced first-token waiting as a central improvement over the earlier build. It reports a 256K code prompt falling from 108.4 seconds to approximately 58 seconds. That comparison accompanies changes in power settings and software, limiting simple explanations.

For developers building document applications, the lesson is to examine the entire interaction. A short demonstration can hide initial document loading. A cold-start test can miss the benefits of repeated work on unchanged material.

The pressure on competing systems is therefore specific. Buyers need latency measurements that resemble actual sessions, including both fresh documents and follow-up questions. A single maximum generation rate leaves too much of the user experience unexplained.

Old V100s Save Memory Traffic, Not Four-Bit Math

The build’s reported speed depends on software and data movement, despite the cards lacking native four-bit computing hardware.

The V100 does not acquire newer arithmetic capabilities because a model uses NVFP4 storage. Mermiges explains that the stored expert values are unpacked into FP16, a sixteen-bit floating-point format, before the relevant computation. The compact representation reduces the bytes read; it does not create native FP4 execution.

Nvidia’s NVFP4 model documentation provides the reference for the selected expert package. That establishes an identifiable model artifact for researchers to examine. It does not independently validate this server’s implementation, performance, or application quality.

This mechanism helps explain why an older accelerator can remain useful for a newer model format. Storage and arithmetic are separate decisions. A processor can benefit from reading less data even when software must convert that data before computation.

There is a corresponding limit to the inference. Reduced memory traffic does not erase conversion overhead, communication overhead, or differences in energy efficiency. The same technique can create a useful deployment without making the older architecture equivalent to newer hardware.

The physical arrangement adds another layer. Mermiges divides the model into three pipeline stages, meaning successive groups of layers run on separate card groups. Within each group, tensor parallelism splits calculations across four cards. Both techniques distribute work, but they introduce communication and scheduling requirements.

NVLink, Nvidia’s connection between compatible accelerators, connects cards within each board. Transfers between boards follow a different path through the host’s expansion infrastructure. That makes the topology part of the result, rather than an incidental detail beneath the card count.

For someone assessing reproducibility, “twelve V100s” is therefore an incomplete specification. The arrangement, engine modifications, memory allocation, and operating settings all belong to the configuration. Another machine with the same number of cards need not produce the same timings.

The software effort also changes the meaning of affordability. A used component can have an attractive acquisition cost while demanding substantial integration work. Whether that work is worthwhile depends on the owner’s skills, available time, and tolerance for maintenance.

This is analysis of the deployment choice, rather than a measured comparison of engineering hours. The published evidence does not assign a complete labor cost to the build. Readers should resist treating an equipment total as a finished procurement budget.

For the broader private AI market, the interesting challenge is practical. Older hardware can remain competitive for selected tasks when developers target the actual bottleneck. Suppliers of newer systems still need to explain what buyers gain in efficiency, support, reliability, and broader workload coverage.

DeepSeek V4.1 Flash provides the workload in this example. The larger engineering principle is to identify which resource limits a particular deployment before assuming the newest accelerator determines the best experience.

The DGX Spark Comparison Has a Narrow Scope

The reported advantage concerns selected single-request decoding results, not a controlled ranking of complete AI systems.

Mermiges’ single-request comparison places his machine alongside published three- and four-unit DGX Spark configurations. He explicitly notes that those systems use their own engines and weights. That qualification belongs beside the claim, because software and model representation affect the outcome.

His twelve-card machine reportedly generates code at 157 to 184 tokens per second across 1K through 128K contexts. Legal text reaches 92 to 102 tokens per second, while prose reaches 83 to 95. The difference between content categories already shows why the maximum figure needs context.

The builder’s comparison also acknowledges stronger Spark results in some prompt-processing tests and advantages as serving workloads become more concurrent. These concessions narrow the story considerably. They describe different system strengths rather than a universal winner.

A person waiting for one answer has a different objective from an organization serving overlapping requests. The individual values how quickly the current task finishes. An organization must also consider queues, fairness between users, sustained throughput, and behavior during demand spikes.

Batching combines work from multiple requests to improve resource use. Its benefits depend on a serving engine designed to exploit that workload. A machine optimized around a particular sequential interaction does not automatically become a high-capacity shared service.

That makes “V100 vs DGX Spark” a useful question only after specifying the job. Is the comparison about starting an answer, finishing an answer, or handling simultaneous users? Does it involve fresh documents or cached ones? Are output quality requirements equivalent?

Those questions also matter for AI agents, applications that alternate model responses with actions or tool calls. Some interactions have sequential dependencies that prevent simply batching every step. Other applications generate enough independent work to keep a shared server busy.

The reviewed evidence does not measure every such application. Applying the benchmark to agent workflows remains a hypothesis that requires direct testing. A code-generation result alone cannot establish faster completion of an entire development task.

An appropriate V100 vs DGX Spark comparison would preserve prompts, model artifacts, generation settings, and measurement boundaries wherever possible. Where configurations must differ, the evaluator should explain why. Otherwise, the test compares assembled software systems without isolating the contribution of the hardware.

This does not make the published numbers meaningless. They identify an implementation worth investigating and give other builders a concrete target. They also challenge simplistic assumptions that product age alone predicts responsiveness.

However, the available results do not establish purchasing superiority. A buyer choosing a system needs evidence covering their actual workload. The relevant pressure on Spark configurations is to demonstrate their strengths under matched conditions, rather than answer a headline with another unmatched headline.

Power, Privacy, and Quality Remain Separate Tests

Fast local generation answers only part of the question a professional buyer needs resolved.

The machine’s operating requirements are substantial. Mermiges says the cards can draw up to 2,100 watts under his 175-watt limits, excluding other system components. He also describes noise and a placement away from occupied working space.

That creates an obvious practical constraint. A server can be useful for private work while being unsuitable beside a desk. Space, cooling, electrical capacity, and ongoing maintenance belong in the assessment alongside responsiveness.

His acquisition-cost argument also needs careful interpretation. The headline calculation reflects what he paid for used components over time. He separately provides a higher estimate for acquiring the hardware at current prices. Historical purchases do not establish what another buyer can reproduce.

Neither total alone settles ownership economics. A comparison would need consistent treatment of supporting equipment, installation, energy, maintenance, and downtime. The reviewed material does not establish a standardized ownership-cost advantage covering all those factors.

Privacy requires a similarly precise boundary. Mermiges says his case documents remain inside his office while he uses the local server. That is an operator statement about his configuration. It is not independent evidence that every comparable setup keeps every category of data local.

An organization would still need to inspect the surrounding application behavior. Logging, integrations, backups, and administrative access can matter even when model inference runs on premises. These are evaluation questions, not allegations about this particular machine.

The distinction becomes especially important when a familiar interface connects to a different backend. Changing the model endpoint does not by itself document the behavior of every surrounding service. The reviewed evidence does not provide a complete audit of that broader environment.

Answer quality is another independent issue. Keeping experts at a selected representation reduces one particular compression compromise. It does not establish that generated answers are correct, sufficiently grounded, or suitable for professional use.

Likewise, numerical checks against another configuration do not replace evaluation on realistic tasks. A system can preserve expected computational behavior and still produce an incorrect interpretation. A fast answer creates little value when verifying it requires more effort than doing the original task.

For document users, information organization also remains distinct from model speed. A searchable knowledge base addresses how material becomes available for retrieval. It does not, by itself, validate an underlying model or establish this server’s privacy properties.

The evidence therefore supports separate decisions. Assess responsiveness, operational fit, data handling, and task quality on their own terms. Combining them into a single “private AI” label would conceal the questions that matter most.

Three Results Would Make the Claim More Useful

Independent replication, realistic serving tests, and task-quality evidence should determine how far this result travels beyond one office.

The first signal to watch is independent reproduction of the reported latency and generation bands. Another operator should identify the exact model artifacts, engine version, hardware topology, and operating limits. Matching those details would make agreement more meaningful.

Fresh-document and cached-document tests should remain separate. Reporting both would clarify whether replication confirms initial processing, follow-up responsiveness, or both. Results should also distinguish code, legal material, and prose rather than treating the fastest category as representative.

Successful reproduction would strengthen confidence that the configuration delivers a repeatable advantage for its stated workload. Materially different results would not automatically discredit the builder. They would identify dependencies that the current public account has not fully isolated.

Reproduction also requires attention to what is publicly available. The repository describes engine changes and measurements while identifying components that are not shipped as a complete launchable package. That limits how easily another operator can reproduce the exact environment.

The second signal is a realistic concurrent-serving comparison. The useful test would follow requests from arrival through completion, including time spent waiting. Summed generation rates alone can miss delays introduced before generation begins.

A matched exercise should include a professional working alone and a small group sharing the server. It should preserve equivalent document and output requirements. That would reveal where the custom configuration remains responsive and where a different serving approach becomes preferable.

If newer batching work improves shared use without damaging single-request latency, the build’s practical scope would expand. If concurrency introduces substantial waiting, the original individual-user positioning would remain the more defensible interpretation.

The third signal is task-quality evidence tied to representative documents. Speed measurements should sit beside evaluations of answer correctness, supporting evidence, and omissions. Those assessments need clearly stated criteria rather than impressions that a model feels comparable to another product.

For private legal work, a useful evaluation would test whether answers accurately represent supplied material and identify relevant passages. Similar principles apply to engineering records or internal research. The exact tasks should reflect the intended users, rather than a convenient demonstration.

Strong quality results would support broader adoption discussions. Weak results would limit the value of the speed advantage, even if every timing reproduced. Hardware performance and professional usefulness overlap, but they are not interchangeable.

DeepSeek V4.1 Flash on twelve V100s is therefore a specific reported result worth testing. It suggests that carefully arranged older hardware can remain relevant when memory capacity and individual responsiveness dominate the workload. It does not settle the broader contest between custom servers and compact AI systems.

Before treating this machine as a buying template, ask for the workload that resembles your own. Examine the cold-document wait, repeated questions, output quality, and behavior when someone else starts working. Compare the maintenance burden with the value of keeping that workflow local. Then inspect the evidence behind each claimed advantage, including the parts another operator can actually reproduce. The next useful question is not simply whether twelve older cards can run the model. It is whether an independently reproduced configuration can deliver dependable answers, acceptable operating requirements, and consistent responsiveness for the work you need completed.

Get your work moving in remio

See what needs your attention and take the next step in Workspace.

remio currently supports Windows 10+ (x64) and Macs with Apple silicon.

Your AI Partner at Work
Get more done with remio

Plan. Create. Deliver.
All in one place.

bottom of page