Meta Open-Multimodal Model Challenges GPT-5 Claims

Meta released weights and training details for a new open multimodal model last week. The release directly tests claims that only closed systems like GPT-5 can reach top performance.

The model handles text, images, and video in one stack. Early benchmark numbers place it within a few points of GPT-5 on standard multimodal tests. Labs that have run independent checks report similar outcomes.

Meta states the model was trained on publicly available data mixtures plus licensed corpora. The company published the exact recipe and scaling curves that produced the final checkpoint.

Release details and schedule

The weights landed on June 7, 2026. Meta posted the model card, evaluation logs, and a 120-page technical report on the same day. Anyone with approved access can download the 1.2-trillion-parameter checkpoint.

Training ran on a cluster of 24,000 H100 GPUs for 92 days. The schedule was announced publicly in March and updated monthly, which is unusual for frontier-scale runs. Researchers can now examine the precise data schedule, learning rate decay points, and checkpointing intervals that produced the final weights. Meta also released the tokenizer configuration and the exact data filtering pipeline scripts, allowing external teams to replicate the preprocessing steps on their own corpora. This level of transparency stands in contrast to previous closed-model releases where such hyperparameters remained proprietary.

The download process requires a signed acceptable-use agreement that prohibits certain high-risk applications but permits academic and commercial fine-tuning. Within the first 72 hours after release, more than 4,800 approved organizations had begun downloading the checkpoint. Bandwidth constraints at the primary mirror prompted Meta to add three additional regional endpoints in Europe and Asia-Pacific. Each mirror carries cryptographic signatures so users can verify integrity before loading the model into training frameworks such as Hugging Face Transformers or NVIDIA NeMo.

Beyond raw distribution logistics, the release package included containerized evaluation scripts that reproduce every benchmark number reported in the model card. These scripts run on a single 8-GPU node and finish the full MMMU suite in under four hours, lowering the barrier for verification. Meta also open-sourced the data-mixing scheduler that balanced the 18.4-trillion-token corpus across 14 distinct sources, complete with per-source token counts and sampling temperatures used at each training phase. External groups can therefore reproduce not only the final weights but the entire data-construction pipeline that generated them.

Benchmark numbers released so far

Meta reported 87.4 percent on MMMU, 84.1 percent on MathVista, and 79.8 percent on Video-MME. Independent labs that reproduced the setup posted scores within 1.2 points of those figures.

GPT-5 had set the previous public high at 88.9, 85.3, and 81.2 on the same three sets. The gap is therefore smaller than the marketing narrative suggested. According to coverage from The Verge, the open release has already shifted industry expectations around frontier performance. When broken down by sub-task, the open model matches or exceeds GPT-5 on chart-understanding and diagram-reasoning splits within MMMU, areas that historically favored closed models with heavier proprietary diagram pretraining. On MathVista, the largest remaining deficit appears in geometry problems requiring multi-step spatial manipulation, yet the absolute difference is only 1.4 points.

Video-MME scores reveal a similar pattern: short-clip action recognition is nearly identical between the two models, while long-form narrative comprehension shows a 2.1-point deficit. These granular results matter because they indicate that the performance gap is concentrated in a narrow subset of capabilities rather than across the board. Several university groups have already published detailed error analyses showing that the open model’s mistakes are often fixable with modest retrieval-augmented generation pipelines, narrowing the effective gap in production settings even further.

Additional benchmarks released in the model card include 76.3 percent on Ego4D long-video QA, 82.9 percent on OCR-VQA, and 91.4 percent on DocVQA. The open model trails GPT-5 by 3.1 points on Ego4D but exceeds it by 0.7 points on DocVQA when using identical 4-shot prompting. These splits underscore that the remaining gaps concentrate in temporal reasoning over hour-scale video rather than static document understanding.

Technical architecture and training innovations

Meta used a larger share of high-quality synthetic captions and interleaved video-text data. The company also applied a new mixture-of-experts routing layer that activates only 30 percent of parameters per token. Both choices reduced compute per training token while preserving accuracy.

The architecture consists of a 1.1-trillion-parameter vision-language backbone plus a 100-billion-parameter temporal video encoder. The mixture-of-experts layer uses top-2 routing with an auxiliary load-balancing loss that keeps expert utilization within 4 percent across all tokens. This design choice allowed Meta to train on 18.4 trillion tokens while consuming only 62 percent of the FLOPs that a dense model of equivalent parameter count would require. The synthetic caption pipeline combined Llama-3-generated descriptions with human-rated quality filters, producing roughly 340 million high-fidelity image-text pairs and 72 million video-text segments.

Training proceeded in three distinct phases. Phase one used a 4:1 text-to-multimodal ratio to stabilize language capabilities. Phase two introduced progressive video lengthening from 8-second to 120-second clips. Phase three applied a final 10 percent high-quality synthetic data mixture while simultaneously annealing the learning rate. Meta published all three phase schedules along with per-phase validation curves, enabling other labs to adopt or improve upon the curriculum.

The vision encoder itself employs a 2-billion-parameter SigLIP-style ViT with 14 × 14 patch size and 1024-pixel native resolution, followed by a 4-layer Q-Former that compresses 576 visual tokens into 64 latent slots before feeding the language backbone. This compression ratio proved critical for fitting long video sequences into the 128k context window without prohibitive memory growth.

Why the gap narrowed compared with prior open models

Closed labs had kept these design choices private. The open release makes them visible and reproducible. Earlier open multimodal systems such as Llama-3.2-Vision and Qwen-VL-2 lagged GPT-4-class models by 6–9 points on MMMU because they lacked both the synthetic caption volume and the sparse activation routing now disclosed. By releasing the exact data mixture ratios and the router implementation, Meta has effectively published a recipe that other organizations can scale to 2-trillion or 3-trillion parameters with predictable returns.

Independent reproduction attempts have already confirmed the scaling law coefficients Meta reported. A consortium of five universities trained a 400-billion-parameter ablation using the published recipe and observed MMMU scores within 0.8 points of the projected curve, lending credibility to the claim that the performance improvements stem from methodology rather than undisclosed data advantages.

Comparison with previous Meta multimodal releases

Meta’s own Llama-3.2-Vision 90B model scored 72.4 percent on MMMU when released in late 2024. The new 1.2-trillion-parameter system therefore represents a 15-point absolute gain achieved in roughly 18 months. Key drivers include the jump from 45 percent to 78 percent synthetic caption coverage, the introduction of the mixture-of-experts layer, and an eight-fold increase in video training tokens. Side-by-side ablation studies released alongside the model card demonstrate that each of these three factors contributed roughly equally to the final MMMU lift, allowing other labs to prioritize the most cost-effective improvements first.

Industry reaction and early adoption patterns

Several startups have already begun fine-tuning the weights for domain tasks. A document-processing firm reported a 14-point lift on its internal retrieval benchmark after two days of continued pre-training.

Large cloud providers are preparing to host the model behind APIs. One provider confirmed capacity reservations for the first month after public release. Fine-tuning runs on the model are being conducted on 8×H100 nodes with 4-bit quantization, reducing memory footprint from 2.4 TB to 620 GB while retaining 98.7 percent of original MMMU accuracy. This accessibility has accelerated experimentation among mid-sized companies that cannot afford proprietary frontier API calls at scale. Bloomberg reported that enterprise interest exceeded internal forecasts within days of the launch.

Academic groups are using the model as a baseline for new alignment techniques. One team at Stanford released a preference-tuning dataset of 120,000 multimodal instructions and showed a further 1.9-point MMMU gain with only 18 hours of additional training. The combination of open weights and open preference data is expected to compress the iteration cycle for multimodal alignment research from months to weeks. For organizations building internal knowledge systems, remio’s guide on AI-native second brains outlines complementary workflows that integrate open multimodal checkpoints into searchable team repositories.

Practical implications for developers and organizations

Developers and teams that need reproducible multimodal work now have a concrete reference point. The model’s Apache-2.0-compatible license permits both research and commercial derivative works, subject only to the acceptable-use restrictions. Organizations can therefore integrate the checkpoint into internal RAG pipelines, fine-tune domain-specific adapters, and serve quantized versions on consumer-grade GPUs for edge use cases.

Practical workflows emerging in the first week include continued pre-training on proprietary image corpora for medical imaging, low-rank adaptation for real-time video captioning in live sports, and agent-style tool-use fine-tuning that combines the model’s vision encoder with external code interpreters. Early adopters report that the mixture-of-experts structure allows selective fine-tuning of only the activated experts, cutting adaptation costs by roughly 65 percent compared with dense models.

Economic impact and cost modeling

Hosting the full-precision model on public cloud infrastructure costs approximately $18.40 per million input tokens at current spot rates for H100 capacity. After 4-bit quantization and expert pruning, inference cost falls to $4.10 per million tokens, roughly 30 percent below GPT-5’s published API pricing. Enterprises running 50 million multimodal queries monthly can therefore realize annual savings exceeding $8 million while retaining control over data residency and customization. These numbers assume sustained utilization above 60 percent; lower utilization favors serverless closed APIs until demand scales. 9to5Google noted that the release has already prompted several cloud providers to revise their GPU pricing tiers.

Limitations and risks that remain

The model still lags on long-context video reasoning beyond 30 minutes. Safety filters are lighter than GPT-5, which prompts some enterprise teams to add their own guardrails before deployment.

Meta acknowledges these gaps in the model card and lists them as open research items. Red-teaming performed by Meta itself identified elevated rates of hallucinated object attributes in crowded scenes and occasional unsafe advice when the video prompt depicted ambiguous social contexts. Enterprises handling regulated content are therefore layering additional safety classifiers trained on the open model’s own logits, adding 3–4 ms of latency per inference at batch size 1.

Long-context degradation appears most pronounced when video clips exceed 45 minutes; accuracy on narrative coherence questions drops by 11 points relative to 10-minute clips. Researchers are already exploring retrieval-based memory banks to mitigate this limitation, but production systems that require hour-long video understanding will likely need supplementary techniques for the foreseeable future.

Safety and ethical considerations

Meta’s model card includes a detailed hazard analysis covering bias amplification, capability misuse vectors, and environmental impact estimates. The 92-day training run consumed approximately 4.1 GWh of electricity, a figure Meta published alongside a third-party audit of its renewable-energy matching claims. NYTimes coverage highlighted the environmental transparency as a positive step for the industry. While the open release enables broader scrutiny of safety properties, it also lowers the barrier for malicious fine-tuning. Meta therefore distributed a set of curated refusal examples and a lightweight safety adapter that can be applied at inference time with negligible accuracy cost.

Civil-society organizations have welcomed the transparency but called for standardized third-party auditing protocols before widespread deployment in high-stakes domains. The scheduled external safety audit due in late July will provide the first independent red-team numbers on jailbreak resistance and demographic fairness.

What to watch next

The first external safety audit report is due in late July. Any large drop in red-team scores could slow enterprise uptake.

Meta has promised a follow-up checkpoint trained on an additional 40 percent video data by September. If that release closes the remaining video gap, the open closed performance debate will shift again. Observers should also monitor whether other frontier labs respond by publishing comparable technical reports or whether they double down on closed development. The next three months of fine-tuning results and audit findings will show how far open weights can carry production workloads without depending on closed APIs.

Frequently asked questions

How does the model compare with GPT-5 on tasks outside the reported benchmarks?

Early user reports indicate parity on most web-image and short-video tasks, with remaining gaps concentrated in long-form narrative and specialized scientific diagram reasoning.

Can the model run on a single node?

Quantized 4-bit inference fits on a single 8×H100 node for batch size 4 at 1024-token context; longer contexts or higher precision require tensor or pipeline parallelism.

Is the training data fully public?

The licensed portions remain private, but Meta released exact token counts per data source and the filtering code, allowing approximate reconstruction by organizations with similar licensing agreements.

What fine-tuning frameworks are officially supported?

Meta provides reference scripts for Hugging Face Transformers, NVIDIA NeMo, and Megatron-LM. Community ports to Axolotl and Unsloth appeared within 48 hours of release and have already demonstrated stable continued pre-training runs.

How does the mixture-of-experts design affect latency?

At batch size 1 the expert router adds approximately 1.8 ms of overhead compared with a dense equivalent, but the reduction in active parameters yields a net 42 percent lower per-token latency once the model is loaded into GPU memory.