Google Harness2 Open-Sources Agent Self-Improvement, but Verification Remains the Test

Google has open-sourced Google Harness2 after reporting gains of up to 11.5 points from changing an agent’s surrounding software instead of its model. That distinction creates the central tension. The framework lets an agent revise prompts, tools, memory, and other operating instructions, yet it must judge those revisions without known correct answers.

Called Harness² by its authors, the project comes from researchers affiliated with Google Cloud AI Research and Northwestern University. Its code appeared alongside an October 9, 2026, research paper and uses the Apache 2.0 license. The release turns a research method into software that developers can inspect, run, and modify.

The project also challenges a familiar assumption about AI agents. Better results do not always require a newer or larger model. They can come from changing the harness, meaning the software and instructions that turn a language model into an acting system.

That argument puts fixed agent frameworks under pressure. OpenCode, Codex, Claude Code, and similar systems combine models with prompts, tools, memory, and orchestration. Google Harness2 asks whether those surrounding components should remain fixed after deployment.

The answer matters beyond benchmark rankings. A self-editing harness can adapt to unfamiliar legal, office, web, and coding work. It can also preserve a bad assumption, select a weaker revision, or spend substantially more compute reaching an answer.

Google Harness2 Changes the Software Around a Frozen Model

The release moves recursive improvement away from model training and into eight editable parts of the agent runtime.

An AI agent is more than the language model answering its prompts. It also depends on instructions, available tools, stored context, delegation rules, and software controlling each action. Collectively, those components form the agent harness.

Google Harness2 exposes eight components for revision: the system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, and middleware plugins. Each component is represented as a file mounted into the runtime. The underlying model weights remain frozen.

That separation is important. Training or fine-tuning a model requires specialized infrastructure, prepared data, and a separate deployment process. Editing a prompt, skill, or tool instruction can happen while the agent is working on a task.

The open-source repository packages the method as a Python project for Linux. It supports Python 3.12, parallel and sequential refinement, and integrations with OpenCode and Codex executors. Its included demonstration uses deterministic editors and local Python execution, so developers can inspect the workflow without model credentials.

Model-backed evaluations require more infrastructure. The repository specifies Node.js, Google Cloud authentication, model access through Vertex AI, and benchmark-specific dependencies. WorkBuddy-Bench requires Docker, while the legal benchmark requires document-processing utilities.

That setup signals the project’s current audience. This is research software for teams prepared to run controlled evaluations, not a consumer feature that silently upgrades every agent. The release supplies the machinery for experimentation, but adopters still need tasks, execution environments, judges, and operating limits.

The framework starts each incoming task from a common base harness. It does not simply carry the latest task-specific configuration into the next assignment. Instead, it refines the current harness temporarily and records reusable lessons in a separate improvement library.

That decision addresses a common failure in adaptive systems. An instruction that helps with contract analysis can harm a spreadsheet task. Directly accumulating every successful edit would make the harness increasingly specialized and difficult to reason about.

Harness² instead tries to retain evidence about why an edit helped. A future task can draw on that experience without inheriting the entire earlier execution configuration.

The project therefore represents two forms of software. One harness performs the work, while another harness improves the system performing that work. This second layer is why the authors describe the method as recursive agent harnessing rather than ordinary prompt optimization.

For developers already maintaining agent instructions manually, the release formalizes a recognizable workflow. Teams inspect failures, revise instructions, rerun tasks, and preserve useful techniques. Google Harness2 assigns much of that loop to agents and records the resulting changes as structured artifacts.

How Google Harness2 Refines an Agent Without Correct Answers

Its core mechanism compares behavior under matched conditions instead of asking a perfect verifier to identify the right output.

The framework calls its main task-level process Propose-Probe-Compose. A proposer first studies the original execution and suggests targeted harness edits. A probe reruns the same task with those candidates. A composer then combines changes supported by differences between the resulting trajectories.

A trajectory is the recorded sequence of model responses, tool calls, observations, and generated files. It gives the system more evidence than a final answer alone. The editor can see whether one revision found missing inputs, invoked a suitable tool, or verified a document before submission.

The system divides proposals into two scopes. An invariant editor focuses on general execution behavior, such as checking outputs after writing them. A domain editor focuses on task-specific knowledge, formats, or procedures.

This separation reduces the chance that every problem receives the same generic fix. It also lets the composer combine a broadly useful process rule with specialized guidance when both produce helpful behavioral changes.

Crucially, the online process does not receive the benchmark’s answer key. The research paper describes an open-world setting where tasks arrive after the harness has been designed. There may be no reliable ground-truth verifier available during execution.

Instead of declaring one candidate correct, the system looks for contrasts. Suppose the base agent overlooks several source files. One revision inventories every input, while another adds spreadsheet reconciliation guidance. Their trajectories reveal whether those interventions changed the agent’s process.

The composer uses that evidence to construct another harness and run the task again. The approach does not eliminate judgment. It shifts judgment from comparing an output with a known answer to comparing observable behavior across executions.

Developers can spend additional test-time compute in two ways. Parallel recursion explores multiple candidate revisions at once, then composes evidence from those attempts. Sequential recursion applies another refinement after each previous revision.

The variable k controls that exploration budget. A larger value gives the system more opportunities to find and test edits, but each opportunity adds model calls, tool use, and execution time.

After the task finishes, a second recursion level updates the continual improvement harness. A reflector stores the adopted harness and an experience file describing the intervention, evidence, and result. Later proposals can consult that library.

This design makes experience more portable than a finished prompt. A stored lesson might say that document-generation tasks benefit from reopening final files and checking required sections. That process can apply across domains even when the exact documents differ.

The distinction also connects the framework to everyday knowledge work. An agent might need to combine meeting notes, spreadsheets, policies, and project documents before producing a decision memo. The difficult part is often tracing claims across sources, not generating fluent prose.

Teams can already reduce that burden by building a searchable knowledge base. Harness² extends the idea into execution policy: the agent can revise how it gathers, checks, and uses that material for a particular task.

The mechanism remains experimental, however. Behavioral contrast is evidence, not truth. Two candidates can share the same mistaken premise, and a polished trajectory can still lead to a bad deliverable.

The Benchmark Gains Put Fixed Agent Harnesses Under Pressure

The reported results suggest that harness design can change performance as much as swapping models, especially for weaker starting systems.

The researchers evaluated Harness² on three professional benchmarks. WorkBuddy-Bench covers coding, web, and office tasks. JobBench tests professional work, while Harvey’s Legal Agent Benchmark measures legal-task performance.

Tests covered four combinations of two models and two starting harnesses. The models were Gemini 3.5 Flash and GLM-5. The initial execution harnesses were OpenCode and Codex, while Claude Opus 4.8 served as the default editor in the main experiments.

A single refinement step improved every tested model and harness combination across all three benchmarks, according to the authors. The largest reported gains reached 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on the legal benchmark’s partial pass rate.

The largest WorkBuddy-Bench change appeared with GLM-5 under OpenCode. Its average score rose from 64.2 to 75.7 after one step. Gemini 3.5 Flash under OpenCode rose from 72.4 to 77.5.

Those numbers support a practical inference. The harness can narrow part of the gap between models by correcting execution behavior that raw model capability does not solve.

Results varied by configuration. GLM-5 under Codex gained only 1.2 points on JobBench, moving from 23.6 to 24.8. Gemini under OpenCode improved by 4.5 points on the same benchmark.

This variation matters more than the highest headline figure. The method did not produce one uniform improvement across every model, harness, and task. Its value depended on the starting system and domain.

The researchers also increased the recursion budget. With Gemini under OpenCode, moving from one refinement step to parallel recursion at k=5 raised WorkBuddy-Bench performance further. Code reached 80.8, web reached 81.7, and office reached 83.4.

The paper reports that both parallel and sequential scores increased as the budget rose from one to five on selected WorkBuddy-Bench domains. Competing methods showed smaller or less consistent changes in those tests.

More exploration was not the entire explanation. In an ablation study, removing the improvement library reduced average WorkBuddy-Bench performance by 3.1 points. Removing scoped editor roles cost 2.9 points, while skipping composition cost 1.6 points.

JobBench showed a similar pattern. Removing the library reduced its score by 1.8 points, generic editors lost 3.7 points, and omitting composition lost 1.9 points. These results indicate that retained experience, specialized proposals, and evidence-based combination each contributed.

A JobBench case study illustrates the mechanism. The base agent failed to reconcile information across source files and did not reopen its final output for verification. Its score was 43.1.

One proposed intervention required an inventory of every input and explicit source tracing. Another added domain guidance for schedule, venue, labor, and budget reconciliation. The composer retained all four edits introduced by those interventions, and the rerun scored 77.8.

That example strengthens the argument for agent harness improvement because the failure was procedural. The model lacked neither basic language ability nor access to the files. It lacked a reliable operating method.

Fixed harnesses still performed differently across domains. The paper’s comparison found that Claude Code led among fixed systems on code and office tasks, while Codex led on web tasks. No unchanged harness led everywhere.

Starting from OpenCode with a larger recursion budget, Harness² exceeded the strongest fixed alternative by 4.6 points on code, 5.8 on web, and 4.9 on office. The authors explicitly note that this adaptive result used additional test-time compute.

The competitive pressure is therefore directed at the fixed-harness approach, not one particular product. A vendor can improve its model, but users may still receive inconsistent results if the surrounding instructions and tools cannot adapt.

Self-Improving Agent Harnesses Still Need Independent Checks

Harness² can improve an agent’s process, but its own evidence shows that recursive edits sometimes select worse behavior or spread a mistaken assumption.

The benchmark results come from the project’s authors and have not yet established production reliability across independent deployments. The paper reports means across three runs, which helps measure variability but does not replace external replication.

The method also spends extra compute before delivering its final output. Each proposal, probe, composition, and rerun consumes tokens and execution time. Parallel processing can reduce elapsed time when capacity is available, but it does not eliminate the underlying resource cost.

The paper places evaluated configurations on an empirical compute-performance frontier for JobBench and WorkBuddy-Bench. That result means the measured gains compared favorably with tested alternatives. It does not mean recursion will be economical for every production workload.

A low-value task may not justify several model executions. A legal memo or operational decision might justify more checking, but those domains also demand stricter controls than a benchmark runner provides.

The most important warning comes from the authors’ own failure analysis. They examined nine regressions and grouped them into candidate selection failures, shared incorrect premises, and evaluation or execution uncertainty.

In one coding example, the framework selected an output scoring 55 over an alternative scoring 95. In a legal task, a narrower revision reduced coverage and dropped the score from 54 to 26.

Other failures began with an incorrect assumption shared across candidates. Those errors produced losses ranging from 2 to 11 points. If every proposed edit accepts the same false premise, comparing their behavior does not expose the original mistake.

The paper also identifies missing judge decisions and ordinary run-to-run variability as attribution problems. A changed score may reflect the edit, stochastic execution, or evaluation noise. More probes help, but they increase cost.

Repository activity has surfaced implementation risks as well. An open cache identity issue reports that resumed experiments can reuse earlier outputs after task instructions or source files change while task identifiers remain constant.

The issue’s author reproduced the behavior without model calls and noted that input content is absent from parts of the saved run identity. The report does not quantify any effect on published benchmark scores, and no maintainer resolution was attached when reviewed.

Still, the issue illustrates the broader operational challenge. A recursive system needs reliable lineage for tasks, source material, harness revisions, cached outputs, and evaluation settings. Otherwise, apparent improvement can reflect stale evidence.

Security creates another concern. The eight editable components include guardrails, scripts, plugins, and tool instructions. Those files can influence what an agent accesses and which actions it performs.

An autonomous editor should not receive unrestricted authority to weaken access controls, execute unreviewed scripts, or persist sensitive task details. Production use needs component-level permissions, immutable security boundaries, audit logs, and rollback.

Persistent experience deserves similar caution. A useful lesson from one customer, department, or legal matter may expose confidential information when retrieved for another. An improvement library therefore needs data separation and retention policies, not just semantic relevance.

Teams should also distinguish performance optimization from safety validation. A benchmark score rewards task completion under its evaluator. It does not automatically measure privacy, authorization, factual accuracy, or resistance to malicious input.

Google Harness2 itself demonstrates why this distinction matters. Its composer can prefer a behaviorally convincing candidate that is objectively weaker. The absence of ground truth is the problem the framework addresses, but it remains a limit the framework cannot remove.

A sensible production design would let the system modify lower-risk guidance while protecting core policies. Prompt wording and workflow checklists might be editable. Credential rules, network boundaries, approval gates, and destructive-action limits should remain outside the agent’s authority.

Human review also remains valuable for proposed changes that persist across tasks. The framework can produce evidence explaining why an edit was adopted. Operators can use that record to approve, reject, or restrict the change before it enters a shared library.

Recursive Agent Harnessing Shifts the Agent Engineering Road Map

The next test is whether organizations can reproduce the gains while controlling compute, regressions, and persistent memory.

The first signal to watch is independent benchmark replication. The repository exposes code, configurations, prompts, and evaluation commands, giving outside teams a practical starting point. Reproductions should report both final scores and total model usage.

Successful replication across different providers would strengthen the central claim. It would show that adaptive harnessing is not tightly coupled to the tested models, editor, or Google Cloud environment.

A failure to reproduce the gains would weaken the case for generality. Researchers would need to identify whether the difference came from model behavior, benchmark versions, execution environments, or evaluation settings.

The second signal is performance on real task streams rather than isolated benchmark runs. Harnessing-level recursion promises cumulative value because lessons from earlier tasks guide later proposals.

The paper offers initial evidence for such transfer. An experience library built on JobBench improved one-step refinement on WorkBuddy-Bench by 0.6 points for code, 0.5 for web, and 2.9 for office.

Those gains are smaller than the strongest task-level results, and one analysis used later-task experience retrospectively. A live deployment should show that useful lessons accumulate in chronological order without importing future information.

Operators should measure regression rates alongside average performance. They should also track intervention frequency, latency, model consumption, and the percentage of persistent edits that survive human review.

The third signal is a stronger verification and governance layer. The authors recommend probes that challenge shared assumptions, repeated execution to separate edits from variability, and better selection methods balancing coverage with precision.

Repository fixes can supply another indicator. Content-aware cache identities, pinned benchmark revisions, and dependable invalidation would make experiments easier to reproduce. Clear security boundaries for editable components would make deployment safer.

The broader research field will also test alternative forms of harness adaptation. Meta-Harness searches for configurations before deployment. ReasoningBank retrieves useful reasoning experience while leaving the harness fixed. Harness Scaling revises a harness for individual tasks.

Harness² combines online task adaptation with a persistent improvement mechanism. That combination is its main distinction, but it also concentrates risk. A bad task-level revision affects one result, while a bad retained lesson can influence later tasks.

For developers, the practical message is not that every agent should rewrite itself immediately. It is that prompts, memory, tools, and orchestration deserve evaluation as a connected system.

Teams can begin with bounded experiments. They can select representative tasks, preserve an immutable base harness, allow revisions only in a sandbox, and compare output quality against total compute. Every persistent change should remain attributable and reversible.

Knowledge workers should care because harness quality shapes visible agent behavior. A model can appear unreliable when its tool instructions are weak, its memory is noisy, or its workflow omits verification. Conversely, a better procedure can improve results without changing the model.

The open release makes those hypotheses testable. It gives researchers a common implementation for studying how agents diagnose failures, revise operating instructions, and retain experience.

Google Harness2 ultimately moves the self-improvement debate from abstract model intelligence into inspectable software artifacts. That is a useful shift because teams can review files, trace revisions, and measure executions.

It does not solve the hardest question: who verifies the improver when no answer key exists? The framework replaces blind editing with structured behavioral evidence, but evidence can still mislead.

Over the next several months, watch for independent replications, live cross-task results, and stricter controls around editable components. Those signals will determine whether recursive agent harnessing becomes a dependable engineering layer or remains a promising research workflow.

For teams evaluating AI agents now, the immediate action is straightforward. Test the model and its harness separately, record every change, and require clear evidence before retaining an automated revision. Google Harness2 provides a concrete framework for that experiment, while its reported failures show why oversight still matters.