Gemini 4 Argon ARC-AGI-3 Score Hits 97.5%, but the Standard Test Tells a Different Story

Google’s Gemini 4 Argon ARC-AGI-3 score reached 97.5% under ARC Prize’s provider adapter harness. The same model scored 49.8% under the standard harness. That 47.7-point gap is the real story.

ARC Prize published the verified results after testing six configurations across its benchmark series. Gemini 4 Argon also scored 98.5% on ARC-AGI-1 and 91.3% on ARC-AGI-2 at high reasoning. Those results place Google DeepMind among the leading labs on all three generations of the test.

Yet ARC-AGI-3 measures an interactive system, not an isolated model answering static questions. Its harness controls what information survives between actions and how long histories are managed. OpenAI’s GPT-6 Astra remains the most relevant comparison because it also crossed 95% under a provider adapter.

That distinction changes how developers and enterprise buyers should read the headline. The 97.5% result supports Google’s claims about sustained, long-horizon reasoning. It does not show that Argon delivers the same performance through every interface or deployment architecture.

The Gemini 4 Argon benchmark is therefore less a simple victory than a demonstration of where model capability now resides. Increasingly, it lives in the combined system of model, memory, context management, API behavior, and agent runtime.

Gemini 4 Argon ARC-AGI-3 Results Split Across Two Harnesses

ARC Prize verified an exceptional result, but it verified two sharply different versions of the same system experience.

The verified scorecard lists Gemini 4 Argon at 97.5% on ARC-AGI-3 with low reasoning and the provider adapter harness. Medium and high reasoning both produced 97.3% under that configuration.

The standard harness generated a different pattern. Low reasoning scored 49.8%, while medium reasoning scored 46.3%. ARC Prize did not publish a comparable high-reasoning point on its chart because that run completed too few tasks.

This result is unusual in two ways. First, increasing the named reasoning level did not improve the provider adapter score. Low reasoning narrowly led the other two configurations.

Second, the provider adapter produced almost twice the standard score. A difference that large cannot be treated as ordinary evaluation noise. It points to a structural change in what the tested system can remember and reuse.

ARC Prize defines the standard harness as a provider-neutral interface. It lets a model carry forward notes that the model chooses to preserve throughout an environment.

The provider adapter can preserve opaque reasoning state between requests. It can also compact a long interaction when the history approaches a context limit. The model can then reuse earlier work without recreating its full reasoning from visible notes.

That matters because ARC-AGI-3 does not present a familiar question with a single response. The model enters an unfamiliar interactive environment, observes a state, chooses an action, and sees the result. It must infer the rules and goal while avoiding wasteful moves.

A failed hypothesis is still useful if the system remembers why it failed. A successful experiment matters only if the relevant observation survives long enough to influence later decisions.

The standard harness asks the model to externalize that continuity through selected notes. The provider adapter keeps more continuity inside the provider’s system. Both configurations use Gemini 4 Argon, but they expose different amounts of the model’s ongoing cognitive state.

ARC Prize says API issues also affected the standard evaluation. At low reasoning, runs completed on 42 of 55 semi-private tasks. Medium reasoning completed 46 tasks, while high reasoning completed only 19.

The reported scores still covered all 55 tasks. Incomplete tasks counted against the result, although partial credit already earned was retained. The 49.8% score is therefore a real published result, but it also reflects an imperfect test run.

That qualification does not erase the harness gap. The medium configuration completed more tasks than the low configuration and still scored only 46.3%. Meanwhile, every provider adapter configuration remained above 97%.

The correct headline is not that one score is authentic and the other is invalid. ARC Prize verified both. The correct conclusion is that the Gemini 4 Argon ARC-AGI-3 result depends heavily on how the surrounding system maintains reasoning across actions.

Why ARC-AGI-3 Makes Memory Part of the Test

ARC-AGI-3 turns context management from an implementation detail into a measured capability.

ARC-AGI-1 and ARC-AGI-2 use static visual reasoning tasks. A system receives examples that establish an implicit transformation, then applies that inferred rule to a new grid. These tests emphasize abstraction and generalization without a long sequence of environmental actions.

ARC-AGI-3 replaces that passive format with unfamiliar interactive worlds. The agent receives no detailed manual. It must experiment, interpret feedback, revise its beliefs, and eventually discover how to win.

The official ARC-AGI-3 benchmark focuses on adaptation under uncertainty. Humans can often transfer ordinary intuitions about objects, movement, causality, and goals into these worlds. An AI system must reconstruct comparable knowledge from observations.

ARC Prize’s technical design places an action budget on that exploration. Solving an environment is not enough if the agent spends an excessive number of moves probing randomly. The benchmark therefore evaluates both progress and action efficiency.

The technical report describes context management as a central challenge. Environment frames use 64-by-64 grids, and repeatedly carrying every observation can quickly consume available context.

A competent agent needs more than a large raw context window. It must identify which changes matter, connect actions with effects, preserve hypotheses, and discard irrelevant detail. That is close to the problem faced by agents operating browsers, codebases, or business systems.

Consider a coding agent investigating an intermittent test failure. It may inspect logs, change a configuration, rerun tests, compare outputs, and return to an earlier hypothesis. Losing the causal chain between those actions can make the agent repeat work or accept a false explanation.

An enterprise research agent faces the same problem across documents. It must distinguish established facts from tentative interpretations while preserving their sources. Long input capacity alone does not guarantee that the right evidence remains influential.

ARC-AGI-3 compresses these problems into controlled environments. The worlds are artificial, but the need to learn from sequential feedback is recognizable. That makes the benchmark more relevant to agent behavior than a one-turn exam.

It also makes comparisons harder. In a static benchmark, the prompt and generated answer define much of the tested system. In an interactive benchmark, orchestration decisions accumulate across dozens or hundreds of steps.

The system may need to decide what enters the next prompt, what becomes a durable note, and when an older history should be summarized. It may also need to handle failed API calls without corrupting its state.

These decisions are often made outside the base model. Yet they directly influence whether the model appears capable of sustained reasoning.

Google designed Argon for long trajectories. Its model announcement says the model supports an output limit of one million tokens and targets complex, multi-step work. Google also describes applications in software engineering, research, and defensive cybersecurity.

Those remain company claims unless independently evaluated in each domain. Still, the provider adapter result is consistent with the narrower proposition that Argon benefits from preserved reasoning state.

The standard harness exposes the other side. A model built around persistent internal trajectories can lose much of its advantage when forced through a thinner, provider-neutral interface.

That is why the ARC-AGI-3 harness comparison matters. It asks whether the capability transfers with the model name or depends on a particular delivery stack.

The Main Contest Is Provider-Native State Versus Portable Agents

The most important competition is no longer only Google versus OpenAI. It is provider-native continuity versus portable orchestration.

Provider-native state gives a model access to features designed alongside its inference system. Retained reasoning can preserve internal work that would be expensive or incomplete to express as ordinary text. Compaction can reduce a long history while keeping information selected by the provider’s machinery.

This approach can deliver higher performance with less application-side engineering. A developer sends actions through the supported interface, while the provider manages more of the long-term reasoning process.

The tradeoff is dependence. If an agent’s performance relies on opaque state, developers cannot fully inspect, export, or reproduce that state through another provider. A workflow may become tied to one API even when the visible prompts look portable.

The standard harness represents a more transferable architecture. The model can write notes that remain available across turns, but the interface does not depend on provider-specific internal state. Developers can inspect those notes and apply a similar pattern to another model.

That transparency does not guarantee equal performance. Gemini 4 Argon’s standard score suggests that visible notes did not preserve enough of what Argon needed for these environments. The model may have lost hypotheses, causal relationships, or plans during repeated interactions.

An explicit notebook can also become noisy. If the agent records too much, later prompts fill with stale observations. If it records too little, important discoveries disappear.

Provider compaction can address that balance, but its decisions are less visible. A high score tells users the complete system worked under the evaluation conditions. It does not reveal every detail of what the compaction process retained.

ARC Prize’s testing policy acknowledges that the two harnesses answer different questions. The organization reports them separately instead of treating them as interchangeable implementations.

The standard harness asks how a model behaves through a minimal, provider-neutral interface. The provider adapter asks how it performs when paired with context-management features offered by its creator.

Neither question is trivial. Portability matters for buyers who want bargaining power, auditability, and fallback providers. Native performance matters for teams that need the best available result on difficult workflows.

OpenAI’s GPT-6 Astra provides the clearest competitive reference. Its provider-adapter configuration remains ahead of Argon on the ARC-AGI-3 leaderboard. Its standard score is also higher than Argon’s published standard result, although the comparison must account for Argon’s incomplete runs.

The official leaderboard reinforces the importance of separating configurations. ARC Prize labels standard and provider-adapter results and explains that they preserve different forms of state.

That structure pressures every frontier lab. A provider can no longer publish one agent score without explaining the runtime that produced it. Buyers should ask whether a result used native state, external tools, repeated sampling, private memory, or task-specific scaffolding.

The pressure also reaches independent agent platforms. Their value proposition often rests on making models interchangeable while adding memory, tools, and workflow control above the API.

If provider-native adapters keep producing gaps of this size, generic orchestration layers face a difficult choice. They can remain portable but accept lower peak performance, or build deep integrations that reduce practical interchangeability.

Developers should not assume the choice is permanent. Capabilities that begin as specialized harness features can move into standard APIs. Explicit memory systems can also improve as teams learn what agents need to retain.

For now, Gemini 4 Argon makes the division unusually visible. The model looks near-saturated under one interface and far from saturation under another.

That gap is more informative than a single ranking. It shows that model selection and agent architecture have become inseparable purchasing decisions.

What the 97.5% Score Does Not Establish

A verified benchmark score is strong evidence about a defined test, not a general certificate of autonomous intelligence.

ARC Prize verifies frontier scores on semi-private tasks to reduce direct overfitting. Test items are sent to provider APIs under data-retention safeguards, and the results include task-level evaluation information.

That process is more credible than a self-reported score on a public test. It limits a lab’s ability to tune directly against the complete evaluation set.

Verification still does not expand the scope of the claim. Gemini 4 Argon achieved 97.5% under a named harness, reasoning setting, scoring method, and action budget. Changing those conditions can change the result, as the standard score demonstrates.

ARC-AGI-3 environments also occupy a specific domain. They test exploration and adaptation inside grid-based interactive worlds. Real business workflows add ambiguous instructions, changing permissions, unreliable websites, conflicting documents, and consequences for mistaken actions.

A model can excel at the benchmark and still struggle with an enterprise workflow whose success criteria remain unclear. Conversely, an agent optimized for a stable business process may create value without scoring highly on abstract environments.

The incomplete standard runs create another uncertainty. ARC Prize attributed the missing coverage to API issues, but the public result alone cannot tell buyers how often similar interruptions will occur in production.

Long-running agents need recovery behavior. They must know whether a failed request can be retried safely, whether state survives that retry, and whether an interrupted tool action produced an external change.

These operational properties are not secondary. An agent that reasons correctly but loses state after a network failure can be less useful than a weaker model with predictable recovery.

The 97.5% result also does not prove that higher reasoning effort always helps. Argon’s low setting slightly outscored its medium and high provider-adapter configurations. That narrow difference might reflect task dynamics rather than a broad law, but it warns against equating more inference with better outcomes.

Teams should test reasoning settings against their own latency, reliability, and quality requirements. The most expensive or longest trajectory is not automatically the best configuration.

ARC Prize’s original benchmark framework also distinguishes general model ability from task-specific harness engineering. Its technical report warns that a harness designed around known public environments can inflate performance without transferring to unseen domains.

The provider adapter is not presented as an ARC-specific hand-coded solver. It uses general context-management features. Even so, readers should avoid turning a high interactive score into a claim that the model has reached human-level general intelligence.

The strongest defensible interpretation is narrower. Gemini 4 Argon can perform extremely well on semi-private ARC-AGI-3 tasks when its reasoning state persists through Google’s adapter design.

A second defensible conclusion is that the same capability does not transfer cleanly through ARC Prize’s standard interface. Whether that reflects a model weakness, a mismatch in note-taking behavior, API instability, or several factors remains unresolved.

Enterprise evaluations should reproduce both sides of this test. Teams can compare a provider’s native agent interface with a portable implementation that stores plans and evidence explicitly.

They should measure task completion, repeated work, recovery after failure, and the quality of retained evidence. For knowledge-heavy workflows, a traceable AI knowledge base can complement model memory by keeping source material inspectable.

That does not replace reasoning state. It creates a separate record that users can audit when the model’s internal continuity is opaque.

The 47.7-point difference is therefore a reason to run better evaluations, not abandon benchmarks. It reveals a deployment variable that a single score would have hidden.

Three Signals Will Show Whether the Result Transfers

The next evidence must show whether Argon’s adapter advantage survives broader testing, operational failures, and real deployments.

The first signal is a completed standard-harness retest. ARC Prize should be able to rerun all 55 semi-private tasks after the reported API problems are resolved.

A full run would establish how much of the 49.8% result came from missing task coverage. If Argon remains near that level, the case for a genuine architecture gap becomes stronger.

If the standard score rises sharply, the original comparison will need revision. It would suggest that service reliability, rather than memory design alone, widened the published difference.

The second signal is performance under comparable harnesses from other labs. OpenAI already supplies the main reference, but more verified results would show whether large adapter gains are common across frontier models.

If Anthropic, xAI, or another provider shows a similar pattern, provider-native state will look like a general feature of advanced agents. The industry would then need to treat harness choice as a standard reporting field.

If other models remain stable across both harnesses, Argon’s gap will look more model-specific. Google would then face pressure to improve portable note-based continuity or document the adapter’s requirements more clearly.

The third signal is evidence from long-running production workflows. Google says Argon is being tested in software engineering, research, and cybersecurity. Those domains can reveal whether preserved reasoning improves outcomes beyond controlled games.

Useful reports would include multi-step completion rates, intervention frequency, recovery after interrupted calls, and the percentage of actions users must undo. Average benchmark scores cannot substitute for those operational measurements.

Developers should also watch whether Google exposes more control over compaction and retained reasoning. An interface that lets users inspect summaries, set retention policies, or export durable state would reduce some portability and governance concerns.

Without such controls, buyers receive performance but must trust a hidden memory process. That arrangement may be acceptable for low-risk tasks and harder to approve for regulated or security-sensitive work.

The Gemini 4 Argon ARC-AGI-3 score should therefore change evaluation practice before it changes anyone’s definition of intelligence. A model name alone no longer describes the tested product.

Teams comparing agent systems should record the model version, reasoning level, harness, memory policy, context-compaction method, and failure-recovery behavior. They should also preserve the evidence used for consequential decisions.

The 97.5% score shows what Argon can do when Google’s preferred state-management path is available. The 49.8% score shows how much performance can disappear when that path changes.

Which result matters more depends on the system being purchased. The practical next step is to test the exact interface your application will use, then ask whether its memory remains accurate, inspectable, and recoverable across a full workflow.