DeepSeek-V4 Flash Reinforcement Learning Training Reaches AMD Instinct MI355X GPUs via Miles Framework

DeepSeek-V4 Flash completed its first verified reinforcement learning runs on AMD Instinct MI355X GPUs using the Miles framework. The 284 billion parameter mixture-of-experts model activates 13 billion parameters per token and finished more than 100 optimizer steps across four eight-GPU nodes.

The test combined SGLang for rollout generation and Megatron for policy updates while Miles handled asynchronous loops and weight synchronization. Training and rollout log-probability differences stayed within acceptable bounds, and online reward scores rose steadily throughout the run.

AMD has sought stronger footholds in large-scale training clusters that currently favor NVIDIA hardware. This verification marks one concrete instance where an open ROCm stack supported a full reinforcement learning cycle for a frontier-scale model.

The outcome places immediate pressure on competing GPU vendors and on software teams still waiting for broader multi-node validation.

Verified multi-node run meets stated targets

The four-node cluster delivered end-to-end execution without the three previously documented failure modes: model misalignment between SGLang and Megatron, stale quantization states during weight updates, and node-to-node connection drops under sustained load.

Miles maintained consistent weight versions across rollout and training processes. Offline AIME-2024 scores recorded after the run showed measurable gains that aligned with the upward trend seen in the online reward signal.

No external benchmark claims were released beyond these internal metrics. The company reported the results through LMSYS channels without independent third-party confirmation of the cluster scale or sustained throughput.

AMD gains one concrete data point against established training stacks

NVIDIA systems hold the majority of current reinforcement learning workloads at the largest labs. AMD has published ROCm updates and partner benchmarks, yet few public records exist of comparable scale reinforcement learning runs on Instinct hardware.

The Miles framework attempt therefore functions as a direct comparison on the specific question of whether an alternative software layer can close the operational gap. Miles addressed synchronization and state consistency that had blocked earlier ROCm deployments for this workload.

The test does not prove broad superiority. It shows that one previously unsolved combination of components now completes without immediate collapse under the reported conditions.

Miles handled three documented integration challenges

SGLang and Megatron maintain separate model representations. Miles inserted an alignment layer that kept token probabilities matched during transfer. Quantization states were refreshed on each weight sync rather than held static, reducing drift that had appeared in prior internal trials.

Multi-node stability required explicit heartbeat and restart logic inside the asynchronous loop. The run passed more than 100 optimizer steps without triggering the restart path, according to the reported summary.

These fixes remain tied to the Miles implementation and have not yet been shown to transfer automatically to other frameworks or model sizes.

Remaining questions center on scale and repeatability

The published record covers four nodes and a fixed step count. No data addresses behavior at eight nodes or beyond, sustained week-long runs, or performance variance across different reward models.

Independent labs have not reproduced the result on equivalent hardware. AMD has not released per-node utilization figures or interconnect bandwidth numbers from the test.

Without those measurements, it stays unclear whether the configuration generalizes or remains a narrow successful case.

What to watch in the next three months

Labs running similar reinforcement learning pipelines will release hardware utilization reports that either include or omit MI355X clusters. Any sustained public runs above four nodes would strengthen the current observation.

AMD’s next ROCm point release will indicate whether the Miles fixes were upstreamed or kept framework-specific. Watch for mentions of SGLang-Megatron alignment in the changelog.

Additional model releases from the DeepSeek series trained under the same stack would test whether the measured stability holds across architecture variants.