top of page

DeepSeek V100 Server Trades Speed for Precision

3 days ago
11 min read
Illustrative photograph. Photo: panumas nikhomkhai / Pexels.

A lawyer's DeepSeek V100 server has traded some generation speed for higher numerical precision across frequently used parts of its model. The machine still uses eight Nvidia V100 graphics processors, but its builder reports bringing most measured expert-routing activity closer to an NVFP4 reference. NVFP4 is Nvidia's four-bit floating-point format, which stores model numbers compactly with shared scaling information. The qualification matters: most routing activity does not mean most model components have moved to that format.

The project belongs to Nick Mermiges, who uses local AI for first-pass work on legal case files. His upgrade targets a practical tension: preserving more of a model's numerical behavior while keeping its working components inside limited graphics memory. The earlier build favored aggressive compression and faster output. The newer configuration spends more memory on frequently selected components, accepting slower generation in exchange for closer reference results.

That places the experiment between familiar alternatives: hosted assistants such as Claude, newer local hardware, and heavily compressed models running on older equipment. It does not establish that an improvised server matches a managed legal AI service. It does offer a documented example of an owner choosing numerical fidelity over headline speed. The important questions concern what improved, which measurements belong together, and whether those changes survive a complete working day.

What changed in the DeepSeek V100 server

The upgrade changes where the machine spends its limited graphics memory, rather than replacing its eight main processors.

Tom's Hardware reported the update on November 14, 2026. Its coverage described system memory increasing from 256 GB to 1 TB and an RTX 3090 joining the machine. It also reported code generation falling from an earlier 160–215 tokens per second to 150–180. Tokens are the text fragments a model processes and produces. Those figures describe reported coding workloads, rather than every task the server handles. The publication's server upgrade coverage traces the changes to the builder's public account.

The server runs DeepSeek V4.1 Flash with a legal harness, meaning the surrounding workflows and checks that organize the model's work. That distinction separates the model from the application Mermiges actually uses. A model generates responses; the surrounding system decides what material enters, what task follows, and what requires review. Changing the underlying numerical representation affects only part of that larger process.

The hardware additions also need separate treatment. More system memory does not automatically become more graphics memory. The added RTX 3090 does not automatically contribute to every published speed result. Treating the entire parts list as an explanation for performance would skip the actual configuration and software decisions.

For readers following the project, the substantive development is a change in priorities. The previous approach compressed the routed model components aggressively enough to keep them available on the cards. The update preserves that availability while allocating additional space to selected components at higher precision.

That is a different objective from maximizing tokens per second. A fast response is useful only when its content supports the task. Conversely, a closer numerical match offers limited practical value if the system becomes too slow or unreliable to use.

The update therefore belongs in a discussion about workload design, rather than a contest between processor generations. Its relevance comes from a particular combination of model, memory budget, software, and legal work. Buyers should resist turning that combination into a general purchasing rule.

The visible additions make the machine more elaborate. The less visible change, deciding which components deserve scarce memory, explains why this update matters.

Most expert traffic is not most of the model

The headline percentage measures activity through selected model components, not the proportion of the entire model running at full precision.

DeepSeek uses a mixture-of-experts design, which directs computation through selected model components called experts. During decode, the stage that generates subsequent tokens, different experts receive different amounts of traffic. A component selected frequently can account for more activity than its share of the total component count suggests.

According to the builder's expert routing update, the earlier default promoted 5,949 of 15,360 routed experts to NVFP4. Measured single-slot coverage reached roughly 80–86% of decode routes. The stated 78–85% coverage for the two-slot default was predicted, rather than directly measured. The post also says expert selection used public prompt sets alongside private legal traffic.

Those statements support a narrower claim than “80% of DeepSeek runs at full precision.” They describe a workload-dependent distribution of activity. They do not establish that the same share applies to an unfamiliar user's documents, languages, or requests.

Consider the underlying allocation problem. If some experts receive frequent traffic, preserving their reference weights can affect many routing decisions. Giving equal treatment to every expert would consume memory differently. The project's approach instead favors components that its measurements identify as useful.

That makes DeepSeek mixed precision a form of workload specialization. Here, mixed precision means storing different model components with different numerical representations. The benefit depends partly on whether future requests resemble the traffic used to choose those components.

The distinction matters when assessing legal workloads. Contract review, factual chronology, correspondence drafting, and source comparison can demand different behavior. A selection tuned around one mixture of requests needs testing before its measured coverage becomes a claim about another.

Nvidia's NVFP4 format explanation also helps separate storage precision from hardware acceleration. The format combines four-bit values with shared scaling to limit representation error. Nvidia introduced native acceleration for that format with Blackwell. Keeping reference-format weights on an older processor does not give it Blackwell's execution hardware.

“Near-NVFP4” consequently needs a named reference and a defined measurement. It should not serve as a synonym for uncompressed arithmetic, guaranteed correctness, or newer hardware performance.

The interesting result is the allocation strategy. The machine attempts to preserve higher fidelity along frequently traveled computational paths while retaining compressed coverage elsewhere. Whether that remains effective outside the measured traffic is a question for additional testing.

Better reference agreement does not establish better legal answers

Closer numerical behavior is evidence about the implementation, while legal usefulness requires a separate evaluation.

The project's public model card reports next-token agreement of 95.1% with its NVFP4 reference, compared with 92.2% for the earlier compressed build. That measures whether both implementations select the same next text fragment under the comparison conditions. It does not mean either system answers legal questions correctly at those rates.

The documentation also contains a later configuration correction. A November 13 update reduced the two-slot default from 5,949 to 5,936 NVFP4 experts after increasing reserved memory. The same mixed precision package distinguishes earlier agreement tests from later serving options whose quality had not been established. The documented context limit is 524,288 tokens, meaning the configured maximum input and output capacity for a request.

These details prevent several tempting shortcuts. Agreement with a reference is not agreement with the underlying documents. A reference model can make a factual mistake, and a more faithful implementation can reproduce that mistake.

Likewise, a disagreement does not necessarily produce a worse answer. Different wording can preserve the same meaning. A small numerical difference can also change a critical qualification. The consequence depends on the task and output, not simply the number of matching tokens.

For a DeepSeek legal workflow, evaluation should therefore start with completed work products. Did the output preserve the distinction between an allegation and an established fact? Did it attach a conclusion to the right document? Did it retain uncertainty when the evidence remained incomplete?

These are examples of relevant questions, not findings from an independent assessment of this server. They illustrate why a numerical comparison cannot substitute for task review.

A long configured context also needs careful interpretation. Capacity tells users how much material the system accepts under its settings. It does not certify that every relevant detail remains accessible or correctly connected throughout the response.

Imagine a first-pass chronology assembled from a case file. The valuable result is a usable sequence with traceable supporting material. A server can process the file successfully yet omit an important event. It can also generate fluent text that needs substantial correction.

The practical pressure falls on whoever maintains the workflow. Higher numerical fidelity creates a reason to retest the application, rather than a reason to remove review. The most useful follow-up would compare outputs before and after the upgrade using the same documents and evaluation criteria.

That comparison would connect the engineering change to the reason the machine exists. Until then, the evidence supports closer reference behavior within published tests. It does not support a quantified improvement in legal accuracy.

Speed depends on which build and workload you measure

The public results describe several configurations, so combining their best numbers would create a server that was never actually tested.

The project's later repository documents a November 13 close-out build with code output of roughly 143–175 tokens per second. It lists 176.2 tokens per second in aggregate with three simultaneous requests, compared with the earlier reported 192.3. Its later test also reached a first token after a 256K prompt in 86.7 seconds, versus 108.4 in the earlier run.

The benchmark configuration details disclose different power settings between those runs. They also warn that quality was not established for every slot profile. A slot is the serving capacity reserved for a request's state. These remain developer-reported measurements, rather than independently reproduced benchmarks.

That disclosure changes how the upgrade should be described. The earlier headline figures remain useful as dated results. They should not become permanent specifications for every later revision.

Aggregate throughput also needs an explicit denominator. It counts total output across requests over the measured interval. It does not tell an individual user that their own answer will arrive at that combined rate.

For an office, the distinction is practical. Several background tasks might benefit from greater combined output. An attorney waiting for a particular response cares about that request's completion time. A configuration can improve shared utilization while making each active session slower.

Prompt processing introduces another distinction. Prefill is the initial stage that processes input before generation begins. A long document can create a noticeable wait before the first visible response, even when subsequent output arrives quickly.

That matters for interactive work. A user might tolerate a longer wait for a comprehensive document analysis than for a short follow-up question. An evaluation that blends both into a single speed number hides the difference.

Comparisons with Claude or another hosted assistant need the same discipline. Mermiges describes moving work from a Claude-based harness to his local system. That account establishes his motivation and experience, rather than a controlled comparison between services.

A fair assessment would hold the task, source material, expected output, and review standard steady. It would also include waiting time and corrections. Otherwise, the comparison risks measuring different workflows under a shared label.

The DeepSeek V100 server's slower generation is therefore a real tradeoff, but it needs the right frame. The central question is whether greater fidelity reduces enough correction work to justify the additional waiting and maintenance.

Published output rates cannot answer that alone. They help identify the boundary between a workable implementation and an application whose full value still needs measurement.

Reliability and local control carry their own workload

The builder's stability improvement is encouraging, but the suspected hardware cause remains a diagnosis rather than a proven finding.

Mermiges reports that crashes stopped after changes to power delivery. His account describes nine successful long chats afterward and identifies a possibly faulty power supply as the suspected cause. Earlier errors had pointed him toward PCIe, the connection system linking hardware components. The distinction between those observations and a confirmed root cause should remain visible.

The same account says the RTX 3090 was physically installed, while moving the small draft model onto it remained unfinished. A draft model proposes candidate text that another model checks, a method known as speculative decoding. Its planned placement is a future software change, rather than an established explanation for existing results.

Together, those details show why a working local system involves more than fitting model weights into memory. Hardware diagnosis, configuration management, and software integration consume effort even when generation speed looks adequate.

A run of successful sessions supports the statement that the observed behavior improved. It does not isolate the failed component. Several changes made around the same period can complicate attribution.

That uncertainty is normal in an evolving personal project. It becomes more consequential when someone treats the project as a template for dependable office infrastructure. Another builder needs enough documentation to distinguish a transferable fix from a machine-specific improvement.

Local control raises a related distinction. Mermiges says his case documents remain in his office. That describes his intended operation and reported practice. It is not an independent audit of the system's data handling.

An organization evaluating a similar approach would still need to understand where documents, logs, backups, and administrative access reside. These are deployment questions, rather than evidence of a flaw in this particular machine. The available account does not establish a complete answer.

The comparison with hosted services therefore concerns responsibility as well as location. Running hardware locally gives its owner direct control over many decisions. It also places more maintenance and troubleshooting work with that owner.

A managed service distributes those responsibilities differently. Neither arrangement automatically establishes better output quality, appropriate access, or lower total effort. Those judgments require evidence about the actual system and its use.

The surrounding knowledge workflow remains relevant whichever deployment wins. A searchable knowledge base addresses the related task of organizing source material for retrieval. It cannot replace model evaluation or verify the privacy claims of a separate installation.

For the DeepSeek legal workflow, the demanding question is whether local control produces dependable access to useful work. That requires a repeatable operating process alongside the optimization work. A successful demonstration and a maintainable daily service are related achievements, but they require different evidence.

Three follow-ups will determine how far this travels

The next useful update should connect the engineering improvements to reproducible behavior, rather than add another isolated headline speed.

First, watch for a unified evaluation of the current serving configuration. The stronger evidence would combine measured routing coverage, reference agreement, latency, and completed-task quality on the same build. That would reduce the need to reconcile figures collected under different settings.

This signal would strengthen the project's central claim if the current default retained closer reference behavior during representative use. It would weaken the claim's practical reach if fidelity dropped sharply with unfamiliar traffic or additional requests. Either outcome would be more informative than another maximum output rate.

The most valuable task comparisons would preserve the input documents and scoring criteria. Public examples can support reproducibility without exposing private case files. They would also let other users examine whether the chosen tests resemble their own work.

Second, watch for the completed RTX 3090 draft-model move, followed by a controlled before-and-after measurement. Installing a processor establishes availability. Demonstrating its contribution requires showing what changed in the running system.

A useful result would explain whether the move freed memory, changed waiting time, affected simultaneous requests, or introduced new instability. The comparison needs the same workload and a clearly identified configuration. Without those controls, an improvement could come from another software revision.

This signal would strengthen the allocation strategy if the added processor created usable headroom while preserving output behavior. It would weaken the expected benefit if integration overhead erased the gains. The result matters because the proposed role remains part of the project's unfinished work.

Third, watch for sustained operation and reproduction outside the builder's own machine. An independent attempt would test whether the documented configuration transfers to another installation. Longer operation would also provide better evidence about the stability improvement.

Reproduction need not match every number exactly to be useful. Differences in hardware and settings can explain some variation. The important question is whether another operator can understand those differences and obtain a dependable result.

This signal would strengthen the case for broader adoption if the instructions produced comparable behavior without extensive undocumented intervention. It would narrow the case if success continued to depend on machine-specific knowledge. That would still leave a valuable experiment, but a less transferable office solution.

The deeper issue is who can afford the attention required to operate this approach. Developers gain a concrete optimization example. Enterprise buyers gain questions to ask about evaluation and maintenance. Knowledge workers gain a reminder that local execution, numerical fidelity, and useful answers require separate evidence.

The DeepSeek V100 server deserves attention because its builder documented a deliberate compromise and exposed several limits along the way. It gives readers something more useful than a claim that older hardware can run a large model. It shows how choosing a workload changes what fits, what runs faster, and what remains uncertain.

Before treating this build as a model for your own work, examine the measurements attached to the configuration you would actually use. Separate reference agreement from task accuracy, combined throughput from personal waiting time, and planned upgrades from completed changes. Ask for examples with traceable source material and visible review requirements. Compare the whole workflow with your current process, including the effort spent correcting and maintaining it. The useful next question is not simply whether this server can generate text locally. What evidence would show that its higher precision helps you finish dependable work?

Get your work moving in remio

See what needs your attention and take the next step in Workspace.

remio currently supports Windows 10+ (x64) and Macs with Apple silicon.

Your AI Partner at Work
Get more done with remio

Plan. Create. Deliver.
All in one place.

bottom of page