QEngine RTX 5090 Benchmark Reports 504 Tok/s, With a Cache-Shaped Lead

The QEngine RTX 5090 benchmark reports 504 tokens per second for Qwen3.8-Flash-Next, but that peak comes with conditions that change the competitive picture. A LocalLLaMA user tested QEngine 1.3.1 on two RTX 5090 cards and reported a substantial lead in single-stream decode, meaning one conversation’s generation speed. The same comparison puts QEngine roughly level with Strata on an agent prompt-latency test, while Strata processes the longer benchmark prompt faster.

The tester, posting as Relative-Ant-9249, credits developer bpx03 and explicitly says QEngine is someone else’s project. Their local inference benchmark compares several engines on the same workstation. That makes the report useful, although it remains a user-run comparison without independent replication.

The central tradeoff is already visible: the fastest output stream does not automatically produce the shortest wait throughout an entire conversation. Repeated prompts favor QEngine’s adaptive caching, long inputs favor Strata in the reported test, and retaining idle conversations changes latency again. Developers choosing between these engines need to identify which delay actually interrupts their work.

What the QEngine RTX 5090 benchmark actually measured

The 504 tok/s result belongs to a particular workload, memory configuration, and software version. It is neither an average across the comparison nor a guarantee for every response.

The workstation contains two RTX 5090 cards, each with 32 GB of video memory, alongside a Threadripper PRO 3975WX processor. The tester reports 251 GB of system RAM and a configured context of approximately 262,000 tokens. Tokens are the text units a model processes and generates, rather than a direct count of words.

QEngine used its INT4 AutoRound build of Flash-Next, a compressed model representation using four-bit integer weights. The tester says they changed the CUDA compilation target from architecture 86 to 120, using nvcc 13.3 and g++-13. They report no other source changes and a 53-second interval between launch and serving requests.

That is an encouraging portability report, although successful compilation does not establish general hardware support. It demonstrates that this user got this build running on this machine. It does not establish equivalent behavior across different motherboards, drivers, memory arrangements, or operating environments.

The test separates narrative output from code output, and the distinction matters throughout the results. With the default configuration, QEngine reportedly generated narrative text at 370 tok/s and code at 439 tok/s. Moving its conversation cache into system RAM raised those measurements to 406 and 504 tok/s, respectively.

Strata’s ISTA IQ3_S configuration reportedly reached 214 tok/s for narrative output and 293 tok/s for code. Its UD-IQ4_XS configuration returned 180 and 239 tok/s. These are different compressed model builds, which prevents treating the comparison as a pure measurement of engine implementation.

The tuned bucko vLLM configuration returned 135 tok/s for narrative output and 141 tok/s for code. Its shipped configuration measured 120 and 118 tok/s. The tester also lists an Infernix configuration at 14–18 tok/s, without a corresponding prompt-processing measurement.

Those figures establish a substantial reported difference on this workstation. They also establish the limits of the headline: 504 tok/s is the highest listed code-generation result. The same engine’s narrative result is lower, and its default memory configuration is slower than its RAM configuration.

The report says all engines used the same benchmark script and prompts. The linked community serving recipes provide context for the testing ecosystem, though linking a benchmark project does not independently validate a submitted result.

The most defensible description is therefore specific. QEngine 1.3.1 led the configurations this user tested for single-stream generation, with its highest reported rate occurring on code. Broader claims about the fastest available engine require a more complete field and repeatable independent runs.

Repeated prompts make the generation lead harder to generalize

QEngine’s reported advantage is meaningful, but the benchmark also rewards behavior that ordinary traffic might not reproduce. The tester identifies this limitation directly, making it central to interpreting the result.

According to the post, QEngine’s expert cache adapts to recent use. An expert cache retains frequently needed portions of a model so the engine can access them without repeatedly fetching them elsewhere. Repeating the same prompts gives that mechanism a favorable workload.

This matters because the benchmark repeats its prompts across runs. Recent use then becomes a useful predictor of future use, which can reduce the movement of model data. The resulting generation rate describes a favorable operating condition rather than every possible conversation.

The tester contrasts this with Strata’s cache, which they say already holds approximately 99.5% of routed expert mass. In their account, repeating a prompt therefore provides less additional benefit to Strata. That explanation comes from the tester and has not been independently established here.

The post also supplies an earlier observation from varied traffic: approximately 300–400 tok/s for a single stream. Requests through the tester’s agent workload reportedly ranged from 330–470 tok/s. Those figures remain fast, but they make the 504 tok/s headline look more like a favorable case than a typical expectation.

The title describes an approximately 1.5–1.9 times advantage over the next-best engine tested. The body uses a somewhat broader estimate of roughly 1.4–1.9 times Strata after considering workload differences. Neither range should become a universal multiplier applied to unrelated hardware or prompts.

This is where QEngine vs Strata becomes a workload question. A developer generating similar code repeatedly might benefit from a different cache pattern than someone alternating among unrelated documents. The source does not establish how long either pattern must persist before the measured advantage changes.

A useful independent comparison would preserve both conditions. Repeated prompts reveal what the engine can do when recent activity predicts future demand. Varied prompts reveal how well it handles a less cooperative sequence of requests.

Neither condition deserves automatic preference in every evaluation. Repetition can represent a real application with shared instructions and recurring tasks. Diversity can represent a personal assistant that moves between coding, research, document review, and unfamiliar projects.

The practical problem arises when a favorable condition disappears from the headline. A reader who sees only 504 tok/s might expect that rate immediately after changing tasks. The available evidence does not support that expectation.

The QEngine RTX 5090 benchmark therefore pressures competing engines to explain their behavior under comparable cache conditions. It also pressures benchmark authors to expose those conditions clearly. Without that disclosure, a performance ranking can conceal the reason the ranking exists.

For buyers and developers, the relevant question is how often their actual work resembles the repeated benchmark. That answer determines how much weight to place on the peak. A slightly lower rate with consistent behavior might serve some workflows better than a higher rate with larger swings.

Strata stays competitive before the answer starts

QEngine’s generation lead does not translate into a clear lead across the entire request. The reported prompt-processing results give Strata a stronger position as inputs grow longer.

Prefill means processing the input before the model starts generating its response. At approximately 10,000 input tokens, QEngine’s default configuration reportedly processed about 5,700 tok/s. Its RAM-cache configuration reached approximately 6,200 tok/s at that input length.

Strata’s ISTA IQ3_S configuration reportedly processed approximately 5,100 tok/s at the shorter input length. At approximately 90,000 tokens, however, it reached about 10,000 tok/s. QEngine’s default configuration measured approximately 7,300 tok/s at that longer input length.

The post does not give a matching 90,000-token prefill figure for QEngine’s RAM configuration. That missing result should remain missing rather than being inferred from its shorter-input performance. The available measurements support a split result, with QEngine ahead on the shorter prompt and Strata ahead on the longer one.

This distinction has a direct application to coding assistants. An assistant working from a small recent exchange spends a different share of time reading than one reviewing a large repository context. Faster generation helps both, but it does not erase the time spent processing their inputs.

The agent test adds another important boundary. It covers a 12-turn tool-use sequence with context growing to approximately 35,000 tokens. With prefix caching working, QEngine 1.3.1 reportedly read the final turn’s prompt in 1.75 seconds.

Prefix caching means reusing computation for the unchanged beginning of a prompt. Strata’s corresponding result was 1.7–1.9 seconds, placing QEngine inside that reported range. The tester appropriately describes the two engines as level on this measurement.

The tuned bucko configuration took 4.36 seconds in the same reported comparison. That establishes a difference between those tested configurations, but it does not establish a result for every vLLM deployment. Configuration, model representation, and workload remain part of the comparison.

It also matters that the reported number measures a prompt-reading stage. It does not measure the complete time needed to execute tools, inspect results, generate subsequent instructions, and finish a task. A fast stage can improve an agent without determining the agent’s overall completion time.

QEngine vs Strata therefore has no single winner across the disclosed measurements. QEngine leads generation, while Strata remains competitive on cached agent prompts and stronger on the reported long-input prefill test. That is a more useful conclusion than collapsing everything into output tokens per second.

For teams maintaining local assistants, the pressure is operational. They need to establish whether users mostly wait for input processing, answer generation, or external tools. An engine change addresses only the portions of that waiting time the engine actually controls.

The result also challenges how local inference demonstrations are presented. A stream of rapidly appearing code is easy to show and easy to understand. The quieter delay before that stream begins can matter just as much during repeated tool interactions.

Keeping idle conversations in RAM changes a different wait

The most practical result might be the recovery of older conversations, rather than the highest generation rate. QEngine’s RAM configuration reportedly preserved useful state through intervening traffic.

A KV cache stores attention-related keys and values, allowing a model to reuse calculations from earlier tokens. In this test, QEngine could keep that cache in pinned system RAM, meaning host memory reserved for predictable transfers. The tester reports a 13.5 GB pool representing approximately one million tokens of aggregate cache capacity.

That pool size is distinct from an individual conversation’s configured context limit. Aggregate capacity describes how much conversation state the pool can retain across sessions. It does not mean the benchmark tested a single conversation containing one million tokens.

The idle-session experiment began with eight conversations of approximately 58,000 tokens each. The tester then sent approximately 400,000 tokens of other traffic before returning to the original conversations. This sequence examines whether earlier work remains ready after the machine serves something else.

With the cache in RAM, all eight conversations reportedly returned warm, meaning their retained state remained reusable. Their time to first reply ranged from 1.56 to 2.01 seconds. That measures a delay users encounter when reopening work, rather than the eventual speed of the output stream.

With the cache in video memory, only two conversations reportedly remained warm. The other six required input processing again, taking 8.7–9.4 seconds. The tester reports a cold-start reference of 8.85 seconds, making the practical consequence easy to see.

A developer moving between ongoing projects would recognize this pattern. One conversation holds a debugging investigation, another contains an implementation review, and another tracks a separate feature. The relevant benefit is preserving the work already processed while attention moves elsewhere.

The same distinction matters in a searchable knowledge base. Finding the right material and resuming a model’s previous computation are separate problems, although both affect how quickly someone returns to useful work.

The source reports that answers remained correct when the video-memory configuration had to process the old input again. Its observed penalty was latency, rather than a documented loss of answer correctness. That distinction prevents treating every cache miss as a reasoning failure.

There is also a concurrency tradeoff, where concurrency means serving multiple streams at once. With eight concurrent streams at 64,000-token contexts, the RAM configuration reportedly produced 124 tok/s per stream. The video-memory configuration reached 139 tok/s per stream.

The tester summarizes that as approximately 10% slower under this concurrent workload, despite a single-stream improvement of approximately 10%. They also disclose one boot per mode and one pass. That is enough to identify a tradeoff worth examining, but insufficient to establish its full distribution.

The broader technical issue extends beyond this engine. The hybrid cache documentation describes Flash-Next-specific handling and limits its stated validation to text generation. Model support, state retention, and workload coverage require explicit attention rather than assumptions based on a product name.

For an individual assistant, preserving idle work might outweigh lower concurrent output speed. For a shared service, the balance could reverse. The reported results make both possibilities worth evaluating without establishing either as the default choice.

Quality checks and competing engines limit the victory claim

The report supports further testing of QEngine, not a declaration that it produces better answers or beats every alternative. Its own quality checks and discussion thread establish those limits.

The assistant test reportedly returned 44 successful tasks out of 44 on QEngine 1.3.1. Earlier 1.3 runs returned 42 out of 44 at low effort. The tester treats the small difference cautiously rather than presenting it as an established quality improvement.

The coding comparison requires even more careful version attribution. A release-candidate build returned 26 successful tasks out of 30 on the tester’s coding set. Strata configurations returned 24 and 25, while repeated results for one configuration ranged from 24 to 26.

The tester explicitly says the coding check had not been rerun on 1.3.1. It therefore cannot serve as a direct quality certification for the exact version behind the headline. It supplies context about an earlier build and the observed variability of the comparison.

The reasonable takeaway is that these limited checks did not reveal an obvious coding disadvantage for QEngine’s compressed model. They do not establish equal quality across all tasks. They also do not show that a generation-speed improvement preserves every behavior a production application needs.

Quantization, the compression of numerical model weights, complicates the engine comparison further. Different compressed builds can alter both performance and outputs. A benchmark comparing complete deployment configurations answers a practical question, but does not isolate every contributing cause.

The competitive field is incomplete as well. In the discussion, Basalt’s developer suggests testing an engine designed specifically for NVIDIA Blackwell hardware. The developer reports performance from a different machine and benchmark, so those figures cannot be inserted into the original ranking.

Still, the existence of Blackwell inference support matters when interpreting the phrase “fastest engine.” QEngine led the configurations the poster measured. An untested alternative remains untested, regardless of whether its developer reports promising numbers elsewhere.

The source thread also contains enthusiastic user reactions and requests for configuration help. Those responses indicate interest, but neither enthusiasm nor an anecdote independently reproduces the benchmark. A second workstation needs a documented test before its owner’s experience can substantiate the same performance claim.

Version drift creates another verification problem. At the time of checking, the QEngine source code repository describes version 1.3.2 and additional Flash-Next functionality. Those current descriptions should not be retroactively attached to the tested 1.3.1 build.

The repository calls the project source-available and documents a focus on RTX 3090 hardware. Its build notes still identify architecture 86 and describe other GPU models as untested. That makes the poster’s 5090 compilation useful evidence of an individual result, rather than a general compatibility commitment.

An independent reproduction should therefore preserve the actual software revision, model files, compiler choices, and launch configuration. Updating everything to the newest version might produce an interesting result, but it would answer a different question. Reproducibility depends on retaining what was measured.

The pressure on QEngine is now to turn a promising user report into a dependable operating pattern. The pressure on competitors is to respond with comparable workloads and transparent configurations. Neither task is satisfied by posting a larger number from unrelated hardware.

Three results would make the next comparison more useful

The next useful evidence is a more reproducible comparison across workloads, rather than another isolated peak. Three specific signals would strengthen or weaken the interpretation of this report.

First, watch for independent reproduction on another two-card RTX 5090 workstation. A useful result would identify the exact QEngine revision and model build, then report both repeated and varied prompts. Reproducing a material generation lead under both conditions would strengthen the case that the advantage travels beyond the original machine.

A smaller advantage on varied prompts would narrow that case without making the original measurement meaningless. It would show that recent workload history accounts for more of the observed result. That information would help users choose an engine based on their own traffic.

Independent testing should also preserve the distinction between narrative output and code output. Combining them into one average would conceal a difference the original report already exposes. Reporting their separate results gives readers a better chance of matching the benchmark to their application.

Second, watch for a same-machine comparison that includes a Blackwell-focused alternative such as Basalt. The original poster’s result does not establish where that engine belongs in the ranking. Testing it under the same conditions would address a concrete gap raised in the discussion.

A credible comparison would disclose differences in compressed model builds and quality settings. It should also include prompt-processing latency and resumed-session behavior alongside generation speed. Otherwise, a new winner would inherit the same ambiguity as the old headline.

If QEngine retains its lead in that expanded field, the competitive claim becomes stronger. If another engine matches generation while improving long-input or resumed-session latency, the choice becomes more dependent on application behavior. Either outcome would be more informative than comparing incompatible screenshots.

Third, watch for repeated mixed-workload testing of idle-session retention and concurrent generation. The reported RAM result is attractive because it addresses the interruption between work sessions. Its concurrent slowdown shows why that benefit needs evaluation under sustained use.

Repeated runs would reveal whether warm-session recovery remains consistent as conversations accumulate and activity changes. They would also show how often the engine processes old input again, encounters failures, or reduces per-stream speed. Those measurements would test the practical value behind the memory configuration.

Consistent recovery with acceptable concurrent performance would strengthen the argument for RAM-backed conversation state. Frequent variability or a larger slowdown would narrow its appeal to lighter workloads. Neither outcome requires pretending that every local assistant serves the same pattern of requests.

For developers considering a change, the QEngine RTX 5090 benchmark supplies a credible reason to investigate, alongside unusually explicit limitations. Its strongest contribution is showing that generation speed, prompt latency, and conversation recovery can move in different directions.

Before replacing an existing engine, compare the waits your users actually experience. Preserve a representative long conversation, introduce unrelated traffic, and measure what happens when the original work resumes. Include varied tasks alongside repeated prompts, and retain quality checks that reflect the work you need completed. Record the tested version and configuration so later updates do not erase the comparison’s meaning. The QEngine RTX 5090 benchmark makes a persuasive case for that evaluation, while leaving the final choice dependent on evidence from your workload. Does your assistant spend more time generating an answer, reading a growing prompt, or recovering a conversation you expected it to remember?