Miles Blackwell RL Trades BF16 Uniformity for Native MXFP8 and NVFP4

Miles Blackwell support now includes two native low-precision reinforcement learning recipes, tested across six configurations on eight NVIDIA B200 GPUs. One runs MXFP8 through rollout and training. The other applies per-token NVFP4 to mixture-of-experts weights while retaining higher precision elsewhere.

The result challenges a familiar assumption about reinforcement learning, or RL. Lower precision does not necessarily require accepting an obviously weaker learning curve. In Miles’ Qwen3-30B-A3B ablation, all five low-precision configurations closely followed the BF16 baseline’s raw rewards.

That finding carries an important qualification. The experiment was a controlled recipe ablation, not a fully tuned training benchmark or a general accuracy guarantee. Miles also retained BF16 in sensitive layers, maintained an extra BF16 weight copy, and observed occasional NVFP4 gradient spikes.

The real contest is therefore not simply four bits against sixteen bits. It is a consistent, selective precision contract against a uniform BF16 pipeline. Miles argues that Blackwell-native formats become useful when training, sampling, conversion, and live weight updates quantize the same tensors in the same way.

Miles Blackwell Support Now Spans the Full RL Loop

Miles has moved MXFP8 and NVFP4 beyond isolated kernels by connecting them across the complete reinforcement learning path.

The Miles team published the recipes on July 29, 2026. Its low-precision results cover checkpoint conversion, Megatron training, SGLang rollout, and live weight export. The implementation also depends on components contributed across TransformerEngine, FlashInfer, cuDNN frontend, and related projects.

The first recipe uses MXFP8 throughout the main computational path. MXFP8 is an eight-bit microscaling format where each block of 32 E4M3 values shares an E8M0 scale. Rollout, forward propagation, weight-gradient matrix multiplications, and data-gradient multiplications can all use it.

That broad coverage matters because an RL system contains two closely linked policies. The training policy calculates updates, while the rollout policy generates the responses used to compute rewards. If they quantize weights differently, they no longer represent exactly the same model.

Miles previously supported a DeepSeek-V3-style FP8 design based on larger scaling blocks. That route remains relevant, especially on Hopper hardware. However, it applies scaling in software around the Tensor Core path rather than using Blackwell’s native microscaling hardware.

The second recipe takes a more selective approach. It applies NVFP4 to the routed expert weights and activations inside mixture-of-experts, or MoE, layers. NVFP4 stores E2M1 values in four bits, with an E4M3 scale for every 16-value block.

A second FP32 scale covers a larger scope. Miles calculates that scale separately for each token instead of sharing one across a tensor or batch. This detail aims to prevent one token’s representation from changing with batch composition.

The rest of the NVFP4 model stays in BF16 unless configuration rules select another format. This design concentrates four-bit computation where MoE models store much of their weight data. It also avoids forcing attention and other sensitive components into the narrowest available representation.

NVIDIA describes NVFP4 scaling as a two-level system designed for Blackwell Tensor Cores. The 16-value blocks provide finer local adaptation than formats using 32-value groups. The tensor-level FP32 scale extends the usable range.

Miles changed the larger scale’s scope from per tensor to per token. That choice separates its RL recipe from a straightforward inference deployment. It also shows why adding an FP4 data type is not enough to create a stable RL system.

Both recipes support high-precision and dequantized backward modes. High-precision backward uses the original BF16 operands for backward matrix multiplications. Dequantized backward instead reconstructs BF16 operands from the exact low-precision values used during the forward pass.

The second option sacrifices some numerical detail but preserves closer agreement with the forward policy. Neither NVFP4 mode executes its backward matrix multiplications in FP4. Miles is therefore testing selective low precision, not claiming a universally four-bit training loop.

The Reward Curves Shift the Burden of Proof

The most notable result is not peak Tensor Core throughput, but the absence of an obvious reward penalty in the tested workload.

Miles evaluated Qwen3-30B-A3B with synchronous, GRPO-style RL on the dapo-math-17k dataset. The system used eight B200 GPUs, divided evenly between rollout and training. Each prompt received eight rollout samples, with responses capped at 8,192 tokens.

The study compared one BF16 baseline with five low-precision configurations. These included end-to-end MXFP8 and two backward modes for each low-precision format. The NVFP4 variants limited four-bit operation to the MoE expert path.

All five low-precision raw reward curves closely tracked the BF16 curve. That does not establish equivalence across tasks, seeds, or longer training runs. It does suggest that lower-precision rollout did not overwhelm the learning signal in this particular experiment.

That is a meaningful threshold for RL. Supervised training can average optimization signals across a large and relatively stable dataset. RL often works with noisier rewards and smaller policy updates, making additional quantization error harder to separate from genuine learning.

Miles also reported that MXFP8 and NVFP4 reduced rollout time compared with BF16. The published article presents the comparison graphically but does not provide a single percentage suitable for a universal headline. The direction of the result is clearer than its portability.

The rollout improvement matters because generation often dominates reinforcement learning workloads. A policy must produce complete responses before rewards and updates can be calculated. Long contexts and multiple samples make this stage especially expensive.

MXFP8 also improved training time relative to BF16 in the measured setup. NVFP4’s training side moved in the opposite direction. Both NVFP4 backward-override variants were slower than BF16 during training, despite faster rollout.

Miles attributes that gap to its current integration rather than to Blackwell’s FP4 Tensor Cores. The measured TransformerEngine path performs per-token FP32 scaling as a separate PyTorch operation. Fused cuDNN frontend kernels exist, but their TransformerEngine integration remains pending.

This split result keeps the analysis grounded. NVFP4 is not automatically faster simply because its values are smaller. Kernel fusion, data layouts, scaling operations, and framework boundaries decide whether theoretical throughput reaches the complete application.

The low-precision configurations also showed greater train-rollout mismatch than BF16. NVFP4 began with a higher reference KL measurement, which compares policy distributions. However, Miles calculated that diagnostic against a BF16 Megatron reference model.

The metric therefore contains the format difference from the outset. Its coefficient was set to zero, so it did not act as an optimization penalty. Miles cautions against treating the measurement as a standalone sign of learning failure.

The reward curves create a useful reversal. Precision mismatch increased, yet observed rewards did not separate materially from BF16 during the ablation. That combination supports further testing while leaving the long-run stability question open.

A Shared Quantizer Is the Real Mechanism

Miles treats low-precision RL as a distributed consistency problem, not merely a request for smaller numbers.

Each part of the RL stack can quantize tensors independently. Megatron owns training, while SGLang and FlashInfer handle rollout operations. Checkpoint conversion and live weight updates introduce two more opportunities for values or layouts to diverge.

Even small differences can accumulate across repeated policy updates. A training worker may optimize one quantized representation while rollout workers sample from another. Rewards then describe behavior from a policy that the optimizer never sees exactly.

Miles addresses this problem through a bit-exact quantizer contract. FlashInfer tests compare its output byte for byte against a TransformerEngine-style reference. Test inputs include random values, boundary cases, zero tensors, and maximum representable values.

The team also disables a FlashInfer fast-math option for the relevant FP4 quantization path. Approximate math can be reasonable for ordinary serving, where minor differences do not feed back into future weights. RL turns those differences into part of the learning loop.

MXFP8 presents another layout issue. Blackwell Tensor Cores expect microscaling blocks to follow the matrix reduction dimension. Forward and backward operations can use different orientations, so one quantized copy cannot always serve both paths correctly.

The MXFP8 documentation explains that TransformerEngine creates row-wise and column-wise copies from the original high-precision input. That consumes more memory, but avoids dequantizing and requantizing an existing low-precision copy.

Miles accepts that memory cost in its full MXFP8 path. Its high-precision and dequantized backward alternatives avoid retaining the second quantized copy. The choice becomes a balance among numerical agreement, memory use, dequantization work, and matrix multiplication speed.

NVFP4 introduces a different consistency problem. Sharing one activation scale across multiple tokens makes a token’s quantized value depend on its neighbors. Changes in rollout scheduling, sequence length, or batch packing can then alter the policy representation.

Miles computes one FP32 activation scale per token online. FlashInfer fuses this calculation into the rollout activation-quantization kernel. The same operation emits packed FP4 activations, block scales, and token scales.

Training and rollout must also use matching expert tensor-parallel partitions. Otherwise, each system sees different pieces of the tensor when calculating the per-token scale. Identical formulas cannot produce identical policies from different inputs.

SwiGLU expert layers add another special case. Their gate and up projections commonly enter one fused matrix multiplication, although checkpoints may store them separately. Miles quantizes each gate-and-up pair together so both receive one consistent larger scale.

These implementation details explain why the Miles Blackwell work stretches across several repositories. No single library controls every representation between a saved checkpoint and a generated rollout. The precision contract must survive each handoff.

That systems work also distinguishes the recipe from post-training quantization. A static inference checkpoint can be calibrated once and served repeatedly. Reinforcement learning changes weights continuously, so every live export effectively creates a new quantization event.

Teams evaluating similar systems will need traceable evidence for those events. An internal engineering knowledge base can connect configurations, kernel versions, evaluation curves, and incident notes. That record becomes important when a numerical regression appears several updates later.

Per-Token NVFP4 Pressures the Uniform BF16 Default

The Miles Blackwell approach pressures teams to justify BF16 everywhere, while still treating selective BF16 as a stability tool.

BF16 remains the simpler baseline. It offers a wider numerical range, fewer quantization contracts, and easier comparisons across training and rollout. Its weakness is that every expert weight and activation consumes more memory bandwidth than narrower formats require.

MoE models make that tradeoff more visible. They contain many expert parameters, although each token activates only a subset. Moving expert weights through memory during rollout can become more restrictive than raw arithmetic capacity.

Miles therefore targets the expert path first. The strategy resembles a financial budget rather than an ideological commitment to four-bit training. It spends precision on the tensors most likely to influence stability and compresses the largest repeated structures.

The experiment retained the final 15 percent of model layers in BF16 for every low-precision configuration. Miles says this choice reduced train-inference mismatch and improved gradient stability. Keeping early layers in BF16 did not provide a similarly meaningful reduction.

Shared experts also remained higher precision. Unlike routed experts, they process every token. Their quantization errors would therefore propagate through every MoE block rather than only through selected routes.

Certain multi-head latent attention projections received BF16 exceptions as well. Their contraction axes can change with execution mode, while MXFP8 uses one-dimensional scaling blocks. A changed axis can regroup values under different scales.

These exceptions are central to the recipe, not incidental cleanup. A headline describing an entirely FP4 model would misstate the work. Miles instead built fine-grained controls that preserve each exception through conversion, training, rollout, and live updates.

That creates pressure on conventional BF16 pipelines in two ways. First, the reward ablation suggests that narrower rollout paths deserve evaluation on Blackwell. Second, the configuration system provides an alternative to choosing one format for every layer.

It also pressures earlier FP8 approaches designed around Hopper. DeepSeek-V3 uses 128-by-128 blocks for weights and 1-by-128 tiles for activations. Miles describes that design as effective, but it cannot use Blackwell’s microscaling hardware in the same native manner.

The comparison does not make the older method obsolete. Hopper clusters remain widely deployed, and block-scaled FP8 has a longer operational record. Blackwell-native formats also create compatibility boundaries for teams supporting mixed GPU generations.

NVIDIA’s TransformerEngine guide now supports FP8, MXFP8, and NVFP4 through optimized building blocks. Yet framework support alone does not settle which layers should use each format. Model architecture and workload shape still govern that decision.

The most direct beneficiaries are organizations running large MoE reinforcement learning jobs on Blackwell. They can test faster rollout without immediately moving every backward operation into low precision. That staged route lowers the cost of collecting evidence.

Smaller teams face a different calculation. Reproducing the setup requires eight B200 GPUs, several coordinated libraries, and careful configuration parity. The engineering burden may outweigh rollout savings at modest scale.

The primary competitive line is therefore selective consistency versus uniform simplicity. Miles offers more control and a path toward lower memory traffic. BF16 offers fewer interfaces where an unnoticed mismatch can enter the learning process.

What the Qwen3 Ablation Does Not Establish

The published experiment supports a promising recipe, but it does not establish broad quality parity or complete four-bit efficiency.

The most immediate limitation is scope. Miles tested one model, one math-oriented dataset, one hardware arrangement, and one principal workload configuration. Closely tracking rewards there cannot predict behavior on coding, tool use, dialogue, or multi-agent tasks.

The fixed setup used synchronous RL and a four-GPU split for each of rollout and training. Asynchronous systems can introduce larger policy lag between data generation and optimization. That lag might interact with quantization mismatch differently.

The published reward plot also covers a recipe ablation rather than a fully tuned training run. Miles explicitly describes it that way. Readers should not interpret the graph as a benchmark victory against optimized BF16 systems.

Raw reward can hide behavioral changes. Two policies may reach similar scores while using different reasoning patterns, response lengths, or failure modes. Stronger validation would include held-out evaluations, multiple random seeds, and task-specific error analysis.

NVFP4’s occasional gradient spikes remain another concern. The high-precision-backward variant showed spikes during the reported run. Dequantized backward reduced the largest examples but did not remove them.

This is consistent with broader concerns around four-bit training. Research on NVFP4 outliers has found persistent sensitivity in specific architectural components. That work concerns pretraining, not Miles’ RL recipe, but it reinforces the need for layer-level monitoring.

The current memory story is also incomplete. Megatron still preserves an additional BF16 weight copy, despite executing training and rollout through low-precision recipes. That copy limits how much model memory the system actually recovers.

Blackwell-native low-precision parameter gathering is still maturing. Miles notes that the relevant NVFP4 parameter-gather path does not yet support its one-dimensional 1-by-16 weight layout. Removing the BF16 copy depends partly on this infrastructure.

NVFP4 training performance presents a second unfinished area. Rollout improved, but training became slower in the tested implementation. Pending fused TransformerEngine integration must close that gap before NVFP4 can claim a clearer end-to-end advantage.

Live weight movement also remains complicated. Serving backends often pad, shuffle, or swizzle weights into layouts optimized for particular kernels. Training systems usually maintain a different canonical tensor representation.

Those transformations can obstruct low-latency updates or remote direct memory access. Each backend-specific layout adds another step that must remain verifiable. A fast kernel provides limited value if every policy update triggers expensive repacking.

Hardware specificity creates a commercial risk. MXFP8 and NVFP4 receive native acceleration on Blackwell, but many organizations still operate Hopper hardware. Adopting the new recipes can split infrastructure support across multiple precision paths.

There is also no independent replication in the published evidence. The results come from the team that designed and implemented the recipes. That is appropriate for an initial engineering report, but third-party reproductions would strengthen confidence.

None of these limits erases the observed result. They define what the result means. Miles has shown that carefully controlled low precision can follow BF16 rewards in one demanding setup while reducing rollout time.

The next standard is harder. The recipes must remain stable across longer runs, varied tasks, asynchronous updates, larger models, and different parallelism layouts. They must also preserve their gains after all framework overhead enters the measurement.

Three Signals Will Decide Whether the Recipe Travels

The next three signals are fused NVFP4 training, independent reward replication, and removal of the extra BF16 weight copy.

First, watch the pending TransformerEngine integration for fused per-token NVFP4 scaling. Miles already uses fused scaling in FlashInfer’s rollout path, but its measured training path performs a separate operation. Integration should reveal whether NVFP4 can improve training time as well as rollout time.

A faster end-to-end result would strengthen the case for the format. A continued training slowdown would narrow its appeal to rollout-heavy workloads. Either outcome would clarify where four-bit computation produces practical value.

Second, watch for independent reproductions across different models and tasks. The highest-value tests would include coding, long-context reasoning, tool use, and asynchronous RL. Multiple seeds would help separate format behavior from ordinary reward variance.

Similar reward curves on those workloads would support Miles’ consistency thesis. Divergence in particular layers or tasks would instead identify where BF16 exceptions need to expand. Either result would be more useful than a single universal precision rule.

Third, watch whether Megatron can remove the extra BF16 weight copy while supporting the recipe’s weight layout. That change would expose the memory benefit currently hidden by compatibility requirements. It would also test whether the low-precision policy can become the system’s primary stored representation.

The Miles Blackwell work has crossed an important engineering boundary. MXFP8 and per-token NVFP4 now participate in one connected RL loop, rather than appearing only in isolated inference or training tests.

The sharper question is no longer whether Blackwell can execute four-bit and eight-bit operations. It is whether teams can preserve one policy across every system that touches those values. Track the integrations, reproductions, and memory changes before treating the Qwen3 curves as a general result.