Black Forest Labs FLUX 3 Action Challenges Nvidia With Open Robot Weights

Black Forest Labs released FLUX 3 Action on September 23, putting an open-weight, 7-billion-parameter model directly into the race for general-purpose robot control. The company says its model leads a prominent simulation benchmark despite using fewer parameters than Nvidia’s competing Cosmos 3 Nano policy.

That comparison gives the release its significance. Black Forest Labs FLUX 3 Action is not simply an image generator adapted for another demo. It predicts video and robot actions together, then converts visual observations and natural-language instructions into a sequence of physical commands.

The initial results look promising, but they remain narrower than a general robotics breakthrough. The leading score comes from simulated tabletop tasks, while a small external hardware test used only 30 attempts. The open weights also come with license conditions, substantial computing requirements, and explicit safety responsibilities for anyone deploying them.

Black Forest Labs FLUX 3 Action Changes the Robotics Contest

The important change is that a prominent visual AI developer has released both usable robot weights and the code needed to adapt them.

FLUX 3 Action is a world action model, or WAM, that predicts future visual states and robot commands within one architecture. It accepts camera frames, the robot’s current state, and a text instruction. It then produces the next chunk of actions alongside predicted video frames.

Black Forest Labs released three main components. The base repository contains the action-pretrained model and shared encoders. Separate DROID and SO-101 checkpoints provide policies adapted to two established robot configurations.

DROID is a large robot-manipulation dataset built from real-world demonstrations across many environments. The released DROID policy targets a Franka robot configuration with three camera views. The SO-101 version targets a smaller, lower-cost robot arm commonly used with Hugging Face’s LeRobot framework.

The model has 7 billion parameters. According to the published model documentation, one DROID inference call returns 32 actions covering about two seconds of movement.

That action horizon matters because robot controllers must repeatedly observe, plan, and act. A longer useful prediction window can reduce how often the full model must run. However, longer action chunks can also amplify errors when the environment changes after a plan begins.

The release includes full fine-tuning, parameter-efficient adaptation, inference, checkpoint conversion, and evaluation tools. Developers can begin with the base model, adapt it to another robot, or run one of the prepared policies.

This is broader than publishing a model card and a demonstration video. The public inference repository includes data preparation, distributed training, checkpoint management, export tools, and robot-specific examples.

The company developed the release with Nvidia and Hugging Face. That cooperation complicates the competitive framing. Nvidia helped enable the model and supplies much of the hardware stack, yet its Cosmos policy is also the most important open benchmark rival.

FLUX 3 Action grew from the larger FLUX 3 program. Black Forest Labs originally built its reputation around image generation. Its latest architecture extends that visual foundation into video, audio, and action prediction.

The connection is not superficial. A video model must estimate how objects move, collide, deform, and respond over time. A robot policy needs related information, but it must also select commands that produce a desired outcome.

Black Forest Labs is betting that these two problems belong inside one shared model. That bet now has downloadable weights, working checkpoints, and benchmark results that other teams can test.

A Smaller Model Now Leads RoboLab-120

FLUX 3 Action’s strongest early claim is a 42.92 percent success rate on RoboLab-120, compared with 36.8 percent for Nvidia’s larger Cosmos3-Nano-Policy.

RoboLab-120 is a simulation benchmark containing 120 tabletop manipulation tasks. Each policy faces visual, procedural, and relational challenges across several difficulty levels.

Examples include identifying the correct object, understanding spatial relationships, and completing multi-stage instructions. The benchmark runs in Nvidia Isaac Sim on a DROID-style Franka robot setup.

The RoboLab paper describes a system for generating realistic scenes and tasks without binding the test to one model architecture. RoboLab-120 uses ten trials per task, producing a larger evaluation set than a short demonstration reel.

Black Forest Labs reports that its DROID-tuned model reached 42.92 percent overall success. OASIS WAM reportedly scored 39.0 percent, while Nvidia’s Cosmos3-Nano-Policy recorded 36.8 percent under the default language setting.

Physical Intelligence’s π0.5 reached 28.0 percent in the same comparison. DreamZero scored 25.7 percent, while Nvidia’s smaller Cosmos3-Edge-Policy reached 22.9 percent.

Those figures make FLUX 3 Action the current leader in the published comparison. They do not mean it completes most tasks reliably. A 42.92 percent success rate still represents failure on more than half of evaluated attempts.

The parameter comparison is also notable. FLUX 3 Action uses 7 billion parameters, while Cosmos 3 Nano uses 16 billion. That gives Black Forest Labs a smaller model with a 6.12 percentage point lead on the reported leaderboard.

Parameter count is not a direct measurement of cost, latency, or intelligence. Architecture, precision, memory movement, sampling steps, and encoder overhead all affect actual deployment performance.

Black Forest Labs also offers several inference variants. The base policy uses four denoising steps with guidance. Another checkpoint removes external guidance, while a step-distilled version produces results in one step.

The company reports faster inference than Cosmos 3 Nano across tested hardware configurations. The precise advantage varies with the checkpoint, numerical precision, and GPU.

These optimizations address a real robotics constraint. A model that plans impressive actions but responds too slowly cannot recover from movement, interference, or perception errors.

Still, the headline benchmark requires context. RoboLab evaluates simulated manipulation, not unpredictable work around people. Simulation can standardize comparisons, but it cannot represent every sensor error, mechanical fault, collision, or environmental change.

The benchmark also emerged from Nvidia’s robotics research stack. That does not invalidate the results, but it makes independent reproduction particularly valuable.

A public issue concerning Cosmos 3 previously reported difficulty reproducing some published RoboLab results. The underlying Cosmos report presents scores across different instruction specificity levels, illustrating how benchmark outcomes depend on the test configuration.

For buyers and developers, the correct interpretation is narrow but meaningful. FLUX 3 Action has established a credible benchmark position using a smaller model. Independent teams must now determine whether that advantage survives different hardware, instructions, and physical environments.

Why Joint Video and Action Prediction Matters

Black Forest Labs is treating robot control as a visual prediction problem with actions embedded inside the same evolving scene.

Traditional vision-language-action models connect visual input and language instructions directly to robot commands. A world action model adds an explicit prediction of how the observed world should change.

FLUX 3 Action jointly denoises video and action tokens. Denoising means the system begins with noisy candidate outputs and iteratively refines them into a coherent prediction.

The model uses a diffusion transformer with two synchronized output streams. One stream represents future visual frames. The other represents robot actions associated with those moments.

Both streams share the same noise level for each training sample. This design encourages the model to connect a commanded motion with its expected visual consequence.

A frozen video autoencoder converts frames into a compact representation. A frozen Qwen3-VL-4B encoder processes the text instruction. The trainable action model combines those signals with the robot state.

For the DROID policy, three camera views are arranged into one visual canvas. The system also receives joint positions and gripper state. It returns 32 commands, each containing seven joint targets and a gripper value.

This is different from generating a video and asking a separate controller to imitate it. The video and action sequences emerge from the same model pass and remain temporally aligned.

The mechanism gives Black Forest Labs a plausible route from generative media into physical AI. Video training exposes a model to motion, contact, object permanence, and cause and effect. Robot data then teaches it how particular machines can influence those scenes.

The company’s earlier FLUX-mimic collaboration offered a preview. That system connected the FLUX 3 visual backbone with robotics expertise from mimic, including work around industrial manipulation.

FLUX 3 Action expands the idea into public weights and reusable adaptation tools. It also covers more than industrial arms in the company’s experiments.

Black Forest Labs reports training versions for two video games and an indoor drone. The game policy used one set of weights across separate driving environments, with a text caption identifying the active game.

For the drone experiment, the model received a 256-by-256 onboard camera view and generated four control values. Its training set contained 800 scripted flights created in Isaac Sim.

The company says the drone navigated rearranged rooms and followed paraphrased instructions that did not exactly match its training sentences. Those results are demonstrations from the developer, not a standardized independent evaluation.

Even so, they illustrate the architectural claim. A shared model can represent commands for a robot arm, a drone, or a virtual vehicle when each embodiment receives suitable inputs and output heads.

This does not make the model universally interchangeable. Every machine has different cameras, action dimensions, units, timing, and safety limits. Adaptation still requires data representing the target body and task.

The main benefit is a reusable visual foundation. Developers may not need to train scene understanding from scratch for every new machine. They can focus more training effort on the robot’s observations and controls.

That approach resembles the foundation-model strategy that reshaped language and image software. Robotics presents a harder test because wrong outputs can damage hardware or injure people.

The model’s predicted video offers another possible advantage. Engineers can inspect what the model expects to happen, not only the numerical command it sends. That visual forecast may support debugging, although it is not a formal safety guarantee.

Open Weights Do Not Mean Unrestricted Robotics

FLUX 3 Action is inspectable and adaptable, but its license, hardware demands, and deployment safeguards limit what “open” means in practice.

Black Forest Labs calls the release open weights rather than fully open source. The model parameters are available, and the associated inference and training code is public. The weights use the FLUX Kommunity License, while portions of the software repository use a conventional open-source license.

The model license permits noncommercial use and some commercial use by qualifying users. Larger organizations or deployments outside those conditions may need a separate agreement.

That distinction matters for robotics teams evaluating long-term dependency risk. A research laboratory can experiment with the weights, while a commercial manufacturer must examine whether its proposed use qualifies.

The model’s resource requirements present another boundary. The DROID checkpoint uses about 32 GB of GPU memory in bfloat16 on an Nvidia H200, according to the published hardware guidance.

FP8 quantization and text-encoder offloading can fit the system onto a 24 GB card. Quantization reduces numerical precision to save memory and improve speed, while offloading moves part of the model away from the primary GPU.

That requirement is accessible compared with some frontier models, but it is not lightweight edge inference. A production robot may still need a nearby GPU server, an expensive onboard computer, or a carefully engineered communication path.

Latency is only one operational concern. The policy outputs target joint positions, but it does not enforce joint velocity, force, collision, or workspace limits.

The model card explicitly tells deployers to add those controls at the application level. It recommends simulator validation, active robot safety limits, human oversight, and an accessible hardware stop.

These warnings expose the difference between a learned policy and a complete robot-control system. A factory deployment also needs state monitoring, emergency behavior, fault detection, access control, maintenance procedures, and responsibility boundaries.

Action chunks create an additional control question. FLUX 3 Action can return 32 commands in one prediction. A controller must decide how many to execute before observing the environment and planning again.

Executing the entire sequence can improve efficiency when the world behaves as expected. Replanning earlier can help when objects move, a grip slips, or a person enters the workspace.

The SO-101 example reflects this tradeoff. Its control loop executes 32 actions from a longer predicted sequence, discards the remainder, and then plans again.

Developers must also preserve the camera ordering, normalization, joint units, timing, and state conventions associated with each checkpoint. Mixing these details can produce plausible-looking but incorrect outputs.

This is why downloadable weights do not eliminate robotics engineering. They shift some work from learning a policy toward integration, verification, and safety enforcement.

For enterprise buyers, the most useful question is not whether the model is open. It is whether the complete system remains testable, maintainable, and safe under the organization’s actual operating conditions.

The Nvidia Comparison Is Real but Incomplete

FLUX 3 Action pressures Nvidia’s model strategy, yet the release also depends heavily on Nvidia’s benchmark, simulation tools, and computing platform.

The clearest competitive comparison is FLUX 3 Action against Cosmos3-Nano-Policy. Both predict future visual states and actions, both offer accessible weights, and both target general robot manipulation.

Black Forest Labs reports better RoboLab performance with fewer parameters. It also claims higher inference speed across several tested GPUs.

That combination matters because robot developers often face a three-way constraint among capability, response time, and memory. A smaller model that improves task success could reduce infrastructure demands without accepting weaker behavior.

However, model size alone does not establish deployment efficiency. FLUX 3 Action includes a frozen visual autoencoder and text encoder. The complete memory and latency profile depends on how those components run.

Checkpoint choice also changes the comparison. Four-step inference can preserve quality while requiring more compute. A single-step checkpoint is faster but may trade away some success.

Nvidia remains central to the release. RoboLab runs in Isaac Sim, the reported inference tests use Nvidia GPUs, and the model was built with Nvidia support.

Black Forest Labs is also a member of Nvidia’s broader open-model collaboration. The relationship resembles competition within a shared platform more than a simple challenger attacking an incumbent.

Hugging Face plays another important role. Its LeRobot framework packages robot datasets, policies, and hardware integrations so researchers can reproduce workflows more consistently.

FLUX 3 Action’s SO-101 checkpoint arrives with saved preprocessing and normalization information. That reduces a common source of error when moving a policy between a repository and a physical arm.

The broader competitive field includes Physical Intelligence’s π models, Nvidia GR00T, DreamZero, OpenVLA, OASIS, and other vision-language-action systems. Each makes different choices about training data, model architecture, openness, and supported hardware.

Some focus on direct action prediction. Others add world modeling or planning components. Several publish weights but retain restrictions around commercial use or training data.

FLUX 3 Action’s distinct position combines generative-video pretraining, joint future-frame prediction, robot actions, and public adaptation tools. Its benchmark lead strengthens that package but does not settle the architectural debate.

A direct action policy can be smaller and simpler because it does not predict video. A world action model spends computation representing possible future scenes, which may improve physical reasoning but also increase latency.

The decisive evidence will come from controlled comparisons under the same data, hardware, safety constraints, and real-world tasks. Public leaderboards rarely capture that entire system.

The release nevertheless changes expectations. Robotics teams can now ask why a larger or closed model performs worse than an available 7-billion-parameter alternative on a relevant benchmark.

Competitors must answer with stronger results, faster deployment, broader embodiment support, clearer licenses, or evidence from real installations. That pressure is more consequential than the leaderboard position alone.

What Developers and Buyers Should Watch Next

The next three signals are independent reproduction, broader real-robot testing, and sustained adaptation across new machines.

The first signal is reproduction of the RoboLab-120 result. Independent teams should run the released checkpoint with pinned versions, documented prompts, and identical evaluation settings.

A reproduced score near 42.92 percent would strengthen the claim that FLUX 3 Action has a genuine benchmark advantage. Large deviations would indicate sensitivity to configuration, software versions, or unpublished details.

Reproduction should include more than one checkpoint. The base, guidance-distilled, step-distilled, bfloat16, and FP8 variants trade speed, memory use, and task success differently.

The most useful reports will publish full hardware details and failure distributions. An overall success rate can hide weakness on complex procedures, spatial relationships, or vague instructions.

The second signal is larger real-robot evaluation. A third-party test cited in coverage involved ten DROID tasks, three attempts each, and a Franka arm.

FLUX 3 Action reportedly completed 28 of 30 attempts. Cosmos 3 Nano completed 27, DreamZero completed 20, and π0.5 completed 13.

Those results offer encouraging external evidence, but 30 trials remain too few for deployment conclusions. One additional failure would materially change the percentage.

Future tests should include hundreds of trials, unfamiliar objects, lighting changes, camera disturbances, shifted work surfaces, and human interruptions. They should also document interventions and unsafe motions, not only final task completion.

Success in simulation becomes more persuasive when a policy preserves its advantage across physical sites and hardware maintained by different teams. If the lead disappears, the model may be benefiting from benchmark alignment.

The third signal is adaptation to genuinely new embodiments. Black Forest Labs provides a base model, full fine-tuning support, and a parameter-efficient SO-101 workflow.

Developers should watch how much task-specific data and computing a new robot requires. A useful foundation model should reduce the adaptation burden, not merely relocate it.

The strongest evidence would come from outside teams adapting the model to different arms, mobile manipulators, drones, or industrial tools. Those projects should compare training time, data volume, reliability, and control latency against established policies.

Licensing will shape that adoption. Researchers can explore the model now, but commercial users must determine whether the FLUX Kommunity terms fit their organization and deployment.

Hardware economics will matter as well. A 24 GB inference path broadens access, but reliable production operation includes spare capacity, monitoring, control hardware, and safety systems.

Developers working through these evaluations need disciplined records of prompts, checkpoints, camera layouts, datasets, and failures. A searchable engineering knowledge base can help teams preserve that context across experiments.

Black Forest Labs FLUX 3 Action has earned attention by connecting a clear technical mechanism with public weights and measurable results. It has not established general-purpose robot intelligence, and its current benchmark score leaves substantial room for failure.

The immediate opportunity is practical experimentation. Teams can inspect the code, run recorded observations without a robot, and evaluate the checkpoint in simulation before approaching physical hardware.

The harder question comes afterward. Can developers reproduce the lead, transfer it to unfamiliar machines, and maintain safe behavior when the environment stops matching the benchmark?

Those results will determine whether FLUX 3 Action becomes a widely used robotics foundation or remains an impressive reference point. For now, its most important contribution is giving the field a concrete, inspectable model to test.