Schema Harness Hits 98.98% on ARC-AGI-3 Public, but the Real Test Is Still Hidden

Schema Harness says it scored 98.98% on the ARC-AGI-3 public set, despite using existing frontier models without changing their weights. That Schema Harness ARC-AGI-3 result nearly matches the benchmark’s human-efficiency target across all 25 public environments.

The result creates a startling contrast. ARC Prize independently measured GPT-5.6 Sol at 13.33% on the public set and 7.78% on its semi-private evaluation. Schema Harness reports 95.35% when wrapping that same model family inside a structured reasoning and verification loop.

This is not evidence that ARC-AGI-3 has been solved. The Schema scores remain self-reported, and the public environments are open to inspection and repeated development. However, the released trajectories make the claim more substantial than an isolated screenshot or social post.

The central story is therefore not a new model beating an old one. It is a specialized harness apparently extracting radically different behavior from models that ARC Prize already tested.

That shifts attention from model weights toward the machinery surrounding them. It also raises a harder question about what public benchmark performance measures once developers can repeatedly study every environment.

Schema Harness ARC-AGI-3 Results Reach the Human Range

Schema Harness reports near-human action efficiency across the complete public set, not merely a few successful game demonstrations.

The project’s Schema Harness results report two separate evaluations over ARC-AGI-3’s 25 public environments. One used Claude Opus 4.8 and Claude Fable 5, while the other used GPT-5.6 Sol.

The Claude and Fable collection reportedly completed all 183 available levels. Its mean Relative Human Action Efficiency, or RHAE, reached 98.98%.

The GPT-5.6 Sol collection completed 182 of 183 levels across 24 winning trajectories. It recorded a mean RHAE of 95.35%.

RHAE does not simply count victories. It measures completed levels and compares the agent’s action count with a human baseline for each level.

An inefficient victory therefore receives less credit than an efficient one. Missing a level also imposes a completion penalty, preventing partial runs from gaining a misleadingly high average.

ARC Prize introduced this structure because ARC-AGI-3 tests learning through interaction. An agent receives no written rules, stated objective, or tutorial for each environment.

It must discover what objects matter, infer how actions change the scene, identify the goal, and execute a successful strategy. The score reflects how much interaction the agent needed during that process.

The benchmark’s scoring methodology caps each level at 115%, allowing exceptional efficiency to offset weaker performance elsewhere. Each complete game remains capped at 100%.

A 98.98% mean therefore indicates more than broad task completion. According to the project’s scoring artifacts, the Claude and Fable system usually acted with efficiency close to the human reference.

The team also released 50 gameplay trajectories through an independent data-hosting platform. The evaluation traces include run metadata, event logs, snapshots, scoring inputs, and per-environment results.

Those artifacts let outside researchers inspect the runs and execute the included scoring utility. They provide useful transparency about the reported arithmetic and agent behavior.

However, reproducible arithmetic is not equivalent to independent benchmark verification. The runs still cover environments that developers can inspect, replay, and optimize against.

The distinction matters because ARC-AGI-3 launched as a test of adaptation to unknown environments. Once an environment becomes public, repeated development can introduce knowledge that a first-time test taker would never possess.

That limitation does not make the result meaningless. It changes the question the result can answer.

Schema Harness offers evidence about how effectively a structured system can master known benchmark environments without retraining its base models. It does not yet establish comparable adaptation to unseen ones.

The Same Models Look Radically Different Inside the Harness

The largest reported gain comes from reorganizing inference, memory, and verification around the model rather than training a stronger model.

ARC Prize announced ARC-AGI-3 in March 2026 with a deliberately severe baseline. Humans could solve every environment, while frontier AI systems averaged only 0.51%.

The benchmark launch described hundreds of turn-based environments designed around exploration, goal discovery, planning, and continual learning. Its public demonstration set contains 25 environments.

Model performance improved after launch, but ordinary frontier-model configurations remained far below humans. GPT-5.6 Sol provided the clearest recent example.

ARC Prize reported that Sol at maximum reasoning effort averaged 13.33% across the public environments. It was also the first directly tested frontier model to win an entire public game.

On the semi-private set, its verified score was 7.78%. The official GPT-5.6 evaluation characterized Sol as unusually capable of orienting itself within unfamiliar scenes and revising failed hypotheses.

Schema Harness reports 95.35% with Sol. That creates an 82.02 percentage-point gap against ARC Prize’s public measurement of the base agent configuration.

The comparison is not a clean scientific ablation. Different tools, prompts, execution limits, context management, and development histories can all affect the result.

Still, the size of the difference makes the surrounding system impossible to dismiss. A model’s benchmark score is increasingly a property of the model-harness combination.

A harness is the software layer that manages an agent’s context, tools, state, execution cycle, and recovery behavior. It determines what the model sees and what it can do next.

This layer can prevent familiar agent failures. Models often lose track of earlier evidence, mistake an observation for a rule, or continue acting after their assumptions become inconsistent.

A simple chat loop places the burden on the model’s prose reasoning. The model must remember observations, maintain hypotheses, plan actions, and detect contradictions within an expanding conversation.

Schema Harness changes that arrangement. It asks the model to maintain an editable program representing its current theory of the environment.

That program becomes a symbolic world model, meaning an explicit representation of objects, states, actions, and transition rules. The agent can inspect and revise it as evidence changes.

The approach reportedly addresses state attribution and mechanism discovery together. State attribution identifies which hidden variables explain the current observation.

Mechanism discovery asks how actions transform those variables. Treating both problems within one executable representation reduces the gap between describing a scene and predicting its next state.

This distinction helps explain why a capable model can perform poorly without the harness. A language model may recognize visible patterns while failing to maintain a consistent causal account across many interactions.

The harness does not add new knowledge to the model’s weights. It constrains the model to externalize, test, and repair the knowledge formed during a run.

That is less glamorous than announcing a larger model. It may be more relevant to teams deploying agents inside real workflows.

An agent handling software, research, or operations must preserve state across many steps. Fluent reasoning is insufficient when one mistaken assumption can corrupt every later action.

An Editable World Model Changes the Agent’s Job

Schema Harness turns environment understanding into a program-testing problem, giving the model a concrete object that can fail and be repaired.

ARC-AGI-3 confronts an agent with visual environments that conceal their rules. A movement may change an object, unlock a route, consume a resource, or produce no immediately visible effect.

The agent must separate observation from explanation. It must also avoid treating its first plausible theory as established fact.

Schema Harness reportedly uses analysis by synthesis. Under this approach, the system explains observations by constructing a mechanism capable of reproducing them.

The agent writes a transition program, commonly described as a step() function. That function predicts how the environment should change after a selected action.

The harness then checks the program against the recorded interaction history. If predicted states disagree with observed states, the current theory has failed a concrete test.

The model must then revise its variables, rules, or object assignments. This loop converts vague reflection into an executable debugging process.

Consider a grid containing a movable piece, a gate, and several colored cells. A chat-based agent might conclude that touching a colored cell opens the gate.

That explanation can sound reasonable after one interaction. Yet a second attempt might show that the gate opens only after two cells activate in a particular order.

An editable program exposes the missing condition. Its predicted state will diverge from the recorded frame, giving the agent a precise reason to update the mechanism.

Once the transition model fits the available history, the system can use it for planning. It reportedly searches possible action sequences within the simulated world before committing actions to the benchmark.

This planning step matters because RHAE penalizes unnecessary interaction. Random exploration might eventually reveal a goal, but excessive actions can reduce the final score dramatically.

The benchmark squares the relationship between human and agent action counts at the level-scoring stage. An agent using ten times as many actions can receive only a small fraction of the human reference score.

Schema’s reported performance therefore depends on both model accuracy and disciplined exploration. The system must learn enough while avoiding aimless action.

The approach resembles work on executable world models elsewhere in the ARC-AGI-3 community. Researchers have already shown that explicit programs and structured state exploration outperform unconstrained agent conversations.

One published baseline built executable Python models and planned through them without environment-specific code. Another used graph-based exploration to track states, transitions, and untested actions.

Schema Harness appears to push this family of ideas further. It puts a single editable representation at the center of observation, explanation, validation, and planning.

Its reported Claude Code baseline adds another useful comparison. According to the project, a simpler coding-agent setup reached 42.83% on the same public set.

That baseline already exceeds many early public results. Yet it remains more than 56 percentage points behind the Claude and Fable Schema configuration.

The comparison suggests that access to code execution alone is not enough. What matters is how the harness directs code toward causal modeling and continuous consistency checks.

This is a meaningful design lesson for agent developers. More tokens or broader tool access do not automatically produce better long-horizon behavior.

Agents need a durable representation of what they believe. They also need a mechanism that forces those beliefs to confront accumulated evidence.

The same principle applies beyond abstract games. A software agent should reconcile its plan with repository state and test results.

A research agent should connect claims to retrieved evidence and revise conclusions when sources conflict. An operations agent should compare expected system changes with observed outcomes.

In each case, the harness can transform memory from a transcript into an editable model. That model gives the agent something concrete to verify.

For teams managing large bodies of technical evidence, a searchable engineering knowledge base can support the human side of that process. Still, retrieval alone does not provide Schema’s executable causal layer.

The wider implication is not that every agent needs a game simulator. It is that reliable agents need representations aligned with the state changes they must control.

A 98.98% Public Score Is Not a Verified AGI Result

The public score tests a developed system on visible environments, while ARC-AGI-3’s strongest claim depends on performance under controlled first exposure.

The project has published detailed evidence supporting its reported score. Yet ARC Prize has not listed Schema Harness as a verified semi-private result at the time of writing.

That gap should remain central to any interpretation. ARC-AGI-3 was designed to measure adaptation when the environment, mechanics, and goals are initially unknown.

Public environments cannot preserve that condition indefinitely. Developers can inspect their files, replay interactions, compare failures, and refine a harness around recurring structures.

Even without hard-coded solutions, repeated iteration can introduce benchmark-specific assumptions. Tool descriptions, prompts, planning routines, or state representations can encode lessons learned from the public set.

ARC Prize addresses this problem with semi-private and fully private collections. The technical report describes 25 public environments, 55 semi-private environments, and 55 fully private environments.

The semi-private set supports controlled evaluation through external model APIs. The fully private set provides the stricter holdout used for the competition.

ARC Prize’s verification policy also monitors agreement between public and semi-private scores. It expects ARC-AGI-3’s semi-private score to be higher because the public demonstration set is harder.

Scores within 15 percentage points represent good agreement under that policy. Schema’s public results would therefore become far more significant if a controlled evaluation produced a comparable semi-private score.

The comparison with GPT-5.6 Sol shows why this test matters. ARC Prize measured Sol at 13.33% publicly and 7.78% semi-privately under its own agent configuration.

Schema Harness reports 95.35% publicly with Sol. That result could reflect a much better inference architecture, extensive public-set adaptation, or some combination of both.

Only a fresh holdout can separate those explanations convincingly. The released public traces cannot resolve the generalization question by themselves.

There are additional uncertainties. Schema uses two Anthropic models across its strongest collection, rather than one fixed model handling every environment.

The routing logic and model selection process therefore deserve examination. Researchers need to know whether selection occurred before each run, after exploratory attempts, or through another evaluation protocol.

Compute also matters even when prices remain outside the discussion. An agent that repeatedly writes programs, replays histories, and searches action sequences may use substantially more inference than a direct model call.

ARC-AGI-3 measures action efficiency inside the environment. It does not necessarily capture the full amount of computation performed between actions.

That is not a flaw unique to Schema. Test-time computation is a central variable across modern reasoning benchmarks.

However, comparisons should report model calls, tokens, wall-clock time, failed attempts, and selection procedures. Otherwise, an efficient action trace can conceal an expensive internal search.

Reproducibility presents another distinction. The trajectory release lets researchers verify outputs and scores, but a complete reproduction requires the full harness implementation and evaluation procedure.

A trace shows what happened during a successful run. It does not always reveal how many alternative configurations or discarded runs preceded the published collection.

None of these caveats invalidate the reported 98.98%. They define what evidence is still needed before treating it as broad progress on adaptive intelligence.

The responsible conclusion is narrow but important. Schema Harness appears to master the visible ARC-AGI-3 environments with striking efficiency, using explicit world modeling rather than weight updates.

Whether the same method generalizes to unseen environments remains unanswered. That question is the benchmark’s actual center of gravity.

The Result Pressures Model Makers and Benchmark Designers

Schema Harness shifts competitive pressure toward system design while forcing benchmarks to distinguish model capability from harness-assisted performance.

Frontier model providers typically present reasoning progress through improvements in model releases. Schema’s reported result suggests that comparable or larger gains can emerge after deployment architecture changes.

That creates pressure on model makers in two ways. First, customers may stop treating raw model rankings as sufficient evidence for agent performance.

Second, providers may need to offer better primitives for persistent state, code execution, simulation, verification, and controlled context management.

A model that writes persuasive explanations can still fail when its internal assumptions drift. A harness that catches drift can outperform a nominally stronger model operating inside a weaker loop.

Developers should therefore evaluate complete systems. The relevant unit includes the model, tools, memory policy, state representation, verifier, planner, and recovery behavior.

The result also pressures agent-platform vendors. Generic loops built around tool calling and conversational memory may struggle against systems designed around task-specific executable representations.

That does not mean every application needs a specialized harness. It means the harness must reflect the structure of the work.

For ARC-AGI-3, the key structure is a changing world governed by hidden transition rules. A symbolic simulator directly addresses that structure.

For coding, the corresponding structure might be files, dependencies, tests, and runtime behavior. For research, it might be claims, sources, dates, and contradictory evidence.

This makes harness design a form of product engineering rather than prompt decoration. The system must define what state exists, how it changes, and how incorrect beliefs become visible.

Benchmark designers face a related challenge. Public sets support research, debugging, and comparison, but they become weaker measures of first-exposure adaptation over time.

ARC Prize already recognizes that distinction. Its private evaluation layers are not administrative details; they protect the scientific meaning of the benchmark.

The Schema Harness ARC-AGI-3 result illustrates both sides of public evaluation. Open environments let researchers discover an effective architecture and publish inspectable traces.

The same openness prevents the final score from proving that the architecture transfers to unseen environments. Progress and overfitting can produce similar public numbers.

Community comparisons reinforce the point. Earlier systems used graph exploration, executable Python models, continual learning, memory files, and multi-agent orchestration.

Those approaches produced meaningful gains without approaching Schema’s reported average. The field has steadily moved away from asking a model to solve each environment through unsupported reasoning alone.

Schema consolidates that trend into a clearer thesis. Agents improve when they can construct, execute, falsify, and repair an external model of their world.

If that thesis survives private testing, model providers will face a new competitive landscape. Better base reasoning will still matter, but harness compatibility will matter alongside it.

If the private score collapses, the lesson will be different. Schema would remain an effective public-set solver, while its broader claim about general mechanism discovery would weaken.

Either outcome would improve understanding. The current public result is most valuable as a strong, testable hypothesis about agent architecture.

Three Signals Will Determine What Schema Harness Actually Proved

The next evidence must test transfer, reproducibility, and resource use rather than produce another optimized public score.

The first signal is a controlled semi-private ARC-AGI-3 evaluation. This is the most direct test of whether Schema’s world-modeling loop transfers beyond environments used during development.

A score near the reported public range would strengthen the claim that the harness discovers mechanisms during first exposure. A large collapse would point toward public-set specialization.

The second signal is a complete, reproducible harness release with a fixed evaluation protocol. Researchers need the prompts, tools, routing rules, retry limits, model-selection process, and scoring pipeline.

Independent teams should be able to run the same configuration without manually correcting its world models. Reproduction across different model providers would make the architectural claim stronger.

The existing traces are a meaningful start. They expose 50 runs and enough scoring data to examine the reported results.

However, full reproduction requires the machinery that generated those traces. It also requires disclosure of failed runs and any selection criteria applied before publication.

The third signal is an efficiency report covering computation between environment actions. RHAE measures action efficiency relative to humans, which is central to ARC-AGI-3’s design.

It does not describe every model call, simulation step, or search branch. Those measurements determine whether the approach is practical and whether comparisons use similar inference budgets.

A harness may deserve credit for trading internal computation for fewer risky external actions. That trade can be valuable in software deployment, laboratory control, or expensive operational workflows.

Researchers still need to see the trade clearly. Reporting only final environment actions leaves a major dimension hidden.

These signals matter more than another headline score on the same 25 environments. The public set has already shown that Schema can organize frontier models into an effective solver.

The unanswered question is whether it has captured a reusable process for learning unfamiliar systems. That is a much higher bar than replaying known game structures efficiently.

For developers, the immediate lesson is practical but restrained. Do not assume a model’s default agent score defines its usable capability.

Test whether an explicit state model, executable predictions, historical validation, and planned recovery improve the workflow you actually care about. Measure failures, not just successful outputs.

For benchmark readers, keep the evaluation boundary visible. A 98.98% public score is an impressive systems result, but it is not a verified 98.98% score on unseen ARC-AGI-3 environments.

Watch the semi-private evaluation first, the reproducible release second, and full inference accounting third. Together, those results will show whether Schema Harness found a general agent mechanism or an exceptional public-set solution.