AllSpark Iris Search Agent Leads Its Weight Classes, With a Harness-Shaped Catch

Xiaohongshu’s AllSpark team has released the AllSpark Iris search agent in two open-weight versions, with 35B and 397B total parameters. The team says both models lead comparable open systems across several demanding search benchmarks. That claim matters, but the most revealing result is not a leaderboard position. It is the size of the performance gain supplied by the surrounding context-management system.

Iris-mini and Iris-pro arrive with downloadable weights and an open evaluation harness. AllSpark has also described its data pipeline and training process in a detailed technical paper. The team says more training assets will follow, giving researchers a path toward reproducing more than a polished demo.

The release pressures other open search-agent projects on two fronts. Iris reports strong peer-scale scores, while also showing how much an inference wrapper can influence those scores. That makes the project both a model release and an argument about what the industry should measure.

The AllSpark Iris Search Agent Is More Than Two Checkpoints

AllSpark has released a paired search system whose behavior depends on trained weights, tools, and explicit context controls.

Iris-mini is post-trained from Qwen3.6-35B-A3B. It contains 35 billion total parameters but activates about 3 billion for each token. Iris-pro starts from Qwen3.5-397B-A17B, with 397 billion total parameters and about 17 billion active.

Both use a mixture-of-experts architecture. This design routes each token through a selected subset of specialized parameter groups instead of activating the entire model. Total parameter counts therefore describe model capacity, while active counts better indicate the computation required during each generation step.

Each version supports a 256,000-token context window. That capacity matters because a search agent accumulates queries, returned pages, extracted passages, intermediate reasoning, and tool messages during a long investigation. Even a large context window can fill before the agent resolves a difficult multi-hop question.

The Iris-mini weights and Iris-pro weights are available under the Apache 2.0 license. That permits broad use, modification, and redistribution under the license terms. The release therefore gives developers direct access to both model scales instead of restricting Iris to a hosted interface.

AllSpark also published the evaluation code, including configurations for serving the models and running supported benchmarks. The harness uses an OpenAI-compatible tool-calling interface, making it possible to connect the agent to search and page-reading functions.

A search agent differs from a conventional chatbot because it controls an iterative evidence loop. It decides what to search, interprets the returned material, changes its query when evidence is incomplete, and determines when it can answer. The final response depends on every decision within that loop.

AllSpark reports results from a single ReAct agent. ReAct is a pattern that alternates reasoning with tool actions, allowing a model to revise its approach after each observation. The reported configuration does not use sub-agents or a separate test-time verification team.

That detail narrows what the benchmark results represent. Iris is not obtaining its main scores by launching a large parallel organization of agents and merging their work. However, it still relies on an inference harness that manages tools, context, retries, and answer formatting.

The release is consequently more useful than a bare collection of model files. Researchers can inspect the operating assumptions around the checkpoint. They can also test which gains survive when Iris runs under a different search provider, context policy, or deployment budget.

For developers, the smaller model is the more accessible experimental target. A 35B mixture-of-experts checkpoint remains substantial, but its 3B active parameter count makes its behavior particularly interesting. It asks whether targeted post-training can produce competitive search behavior without activating hundreds of billions of parameters for every token.

The 397B version tests the same recipe at a much larger capacity. Comparing the two provides evidence about which failures respond to scale and which depend more heavily on inference mechanics.

That distinction creates the central tension around Iris. The weights are open, but reproducing the advertised agent requires more than loading those weights. It also requires reconstructing the search environment in which the reported behavior emerged.

Why Peer-Scale Search Agents Are Now Under Pressure

Iris raises expectations for what an open-weight model should disclose alongside a leading benchmark claim.

AllSpark places Iris-mini against open-weight systems in roughly the 30B to 35B range. Its published comparison includes MiroThinker-1.7-mini, FORT-Searcher, Apodex-1.0-mini, Nex-N2-mini, Agents-A1, and XYZ-Aquila-mini.

Under Iris’s headline context setting, Iris-mini scores 82.2 on BrowseComp and 84.8 on BrowseComp-ZH. It records 86.9 on DeepSearchQA and 52.3 on the text-only subset of Humanity’s Last Exam.

Iris-pro scores 88.6 on BrowseComp, 85.1 on BrowseComp-ZH, 92.9 on DeepSearchQA, and 56.4 on Humanity’s Last Exam. AllSpark describes these as the strongest overall open-source search-agent results in their respective parameter ranges.

These benchmarks examine different parts of the research process. BrowseComp emphasizes difficult web browsing across dispersed evidence. BrowseComp-ZH applies a related challenge to Chinese-language web retrieval. DeepSearchQA measures comprehensive research answers, while Humanity’s Last Exam emphasizes expert-level knowledge and reasoning.

The scoring methods are not identical. DeepSearchQA uses an F1 measure, while the other reported tasks use accuracy. AllSpark evaluates Humanity’s Last Exam on its 2,158-question text-only subset, so that result should not be read as a score for every version of the benchmark.

The comparisons also do not constitute a perfectly controlled model tournament. AllSpark notes that baseline figures come from public reports, where projects may use their own context strategies. Some rival scores were reproduced by another team rather than by the Iris researchers.

That limitation does not erase the results. It changes their meaning. Iris presents a strong peer-scale package, but the table combines model quality with differences in agent architecture and evaluation procedure.

This is exactly why other open projects face pressure. A model card that reports only a top-line score now looks incomplete when an alternative release exposes managed and unmanaged results. Developers increasingly need to know whether a gain came from post-training, more tool calls, context compression, retries, or a stronger judging setup.

The competitive target is therefore shifting from a checkpoint to an entire reproducible agent. A useful release needs weights, prompts, tool contracts, search behavior, context policies, and evaluation settings. Missing any of these elements can prevent an outside team from matching a claimed result.

Commercial research products face a related challenge. Their closed systems can combine proprietary models, search indexes, orchestration layers, and verification loops. They may deliver better end-to-end reliability, but outsiders cannot easily isolate which component produced the advantage.

Iris gives open developers a concrete system to examine. They can change its search backend, remove context resets, restrict its turn budget, or test it on private documents. Those experiments matter more for deployment decisions than a single public score.

The release also strengthens the case for evaluating agent economics. A system that answers correctly after one bounded search has different operating characteristics from one that restarts several times. Accuracy without tool-call counts, token use, latency, and failure rates gives buyers an incomplete comparison.

For teams building research assistants, the near-term pressure is practical. They must explain not only whether their agent finds an answer, but how consistently it gathers defensible evidence. They also need to show what happens when pages disappear, search results change, or a task exceeds the nominal context budget.

Iris does not settle those questions. It makes them harder for competing projects to avoid.

SFT-RL Climbing Trains the Search Loop, Not Just the Answer

The main technical contribution is a training pipeline designed around difficult evidence chains and repeated interaction with live search.

The Iris research paper describes a data pipeline that begins with the hyperlink structure of a web corpus. It treats pages as nodes and links as edges, then constructs local graphs around selected seed pages.

The system uses those graphs to create multi-hop questions. A multi-hop question requires combining evidence from several locations rather than retrieving one sentence containing the answer. This construction targets the sequence of decisions that makes web research difficult.

AllSpark rewrites non-answer entities into descriptive references. That step removes obvious names that a model could enter directly into a search box. The goal is to prevent shallow string matching from solving tasks intended to measure investigation.

Candidate questions then face two tests. A reference model must fail when answering from its stored knowledge alone. The same model must succeed after receiving the supporting evidence. This filter tries to retain questions that genuinely require retrieval while remaining answerable from identified sources.

A strong teacher model turns the accepted questions into search trajectories. A trajectory records the sequence of reasoning steps, queries, observations, and conclusions produced during a task. AllSpark filters these examples at both the complete-trajectory level and the individual-turn level before supervised fine-tuning.

Supervised fine-tuning, or SFT, teaches the model from selected examples of desired behavior. In Iris, those examples cover more than final responses. They expose how an agent chooses a query, processes a page, and decides whether more evidence is necessary.

The model then receives reinforcement learning against live search. Reinforcement learning adjusts behavior using reward signals rather than copying a fixed target response. AllSpark serves its reward judge and observation summarizer inside the training cluster.

The team alternates SFT and reinforcement-learning rounds in a process it calls SFT-RL climbing. Successful but difficult trajectories discovered during reinforcement learning return to the next supervised stage. Efficient solutions are also favored, encouraging the model to preserve useful behaviors before another round of exploration.

This cycling addresses a common problem in agent training. Static demonstrations can teach recognizable patterns, but they cannot cover every failure encountered on a changing web. Pure reinforcement learning can explore new strategies, yet it may produce noisy or inefficient behavior. Alternating the stages lets each one correct weaknesses in the other.

Long rollouts introduce another infrastructure problem. A search request may generate enough tool traffic and tokens to exceed practical training limits. AllSpark says it interrupts overlong rollouts at the request level and later resumes them from a committed prefix.

The reported training setup used two supervised epochs, a global batch size of 64, and a maximum sequence length of 262,144 tokens. Both models were initialized from Qwen mixture-of-experts checkpoints before this search-specific post-training.

AllSpark also says it blocked access to benchmark-hosting pages during evaluation. Pages under relevant Hugging Face dataset and Space paths were removed from search results, rejected during scraping, and checked after tool use. This defense is intended to reduce direct benchmark-answer leakage.

None of these methods guarantees uncontaminated evaluation. Training corpora, base-model pretraining, and mirrored benchmark discussions can still complicate leakage analysis. However, publishing the safeguards gives independent evaluators a specific procedure to challenge.

The data recipe is especially important because search performance cannot be reduced to memorized factual knowledge. An agent must recognize missing evidence, formulate a useful query, and recover after an unproductive result. Those behaviors emerge from the quality and diversity of trajectories used during post-training.

This mechanism also explains why Iris could matter outside public web search. Similar evidence loops appear in technical support, legal review, market research, and internal knowledge discovery. A team might adapt the agent to traverse approved repositories rather than the public internet.

For example, an engineering group could ask an agent to connect an incident report, a design document, and a code change. The model would still need to decide where to search and whether the evidence supports its conclusion. A well-organized searchable knowledge base becomes part of the agent’s effective environment.

The transfer is not automatic. Web-trained querying habits may perform poorly on private naming conventions or incomplete internal documents. Enterprises would need domain-specific tests, access controls, and source-level citations before trusting the system with consequential research.

Still, Iris offers a concrete hypothesis: better search agents come from training the entire evidence-gathering loop. Larger base models help, but they are only one input to that loop.

The Benchmark Lead Depends on Deliberate Forgetting

Iris’s most consequential result is that discarding context can improve a search agent more than many reported differences between competing models.

AllSpark evaluates Iris both with and without context management. Its headline configuration uses a method called discard-all. Once the prompt crosses a defined threshold, the harness resets the conversation to the original question.

The name sounds destructive because the agent loses its accumulated tool transcript. Yet long search histories contain duplicated page text, failed queries, and reasoning that no longer helps. Removing that material restores space for further investigation.

The harness can preserve useful progress outside the full conversation. A related retry setting restarts an episode that failed to produce a parseable answer and carries forward a short summary of eliminated possibilities. This turns forgetting into an active search policy rather than an accidental loss.

The smaller model shows the clearest effect. Without context management, Iris-mini scores 64.7 on BrowseComp. With discard-all, it reaches 82.2, a gain of 17.5 points.

BrowseComp-ZH moves from 72.3 to 84.8 under the same comparison. DeepSearchQA rises from 81.0 to 86.9, while Humanity’s Last Exam increases from 43.2 to 52.3.

A configuration combining discard-all and retry pushes Iris-mini to 85.9 on BrowseComp, 85.1 on BrowseComp-ZH, 89.9 on DeepSearchQA, and 52.4 on Humanity’s Last Exam. AllSpark does not use that more aggressive configuration for its headline comparison.

The larger Iris-pro also benefits, although the effect is generally smaller. The paper argues that Iris-mini needs more steps to resolve the same constraints, so it exhausts its context more often. Iris-pro can complete more reasoning before context becomes the binding limit.

This provides a useful interpretation of model scale. A larger model may not only know more or reason better. It may navigate toward an answer with fewer costly interactions, reducing its dependence on recovery mechanisms.

The results also vary by benchmark. Context management helps BrowseComp more than Humanity’s Last Exam. AllSpark attributes that pattern to the frequency with which a session actually runs out of context.

BrowseComp tasks demand repeated retrieval, filtering, and evidence integration. Humanity’s Last Exam places more weight on expert knowledge and reasoning, where web retrieval can supplement the answer without dominating every step.

One result suggests that capacity is not always the main bottleneck. Iris-mini with discard-all plus retry and Iris-pro under two managed settings all reach 85.1 on BrowseComp-ZH. The paper notes that this equals 246 correct answers among 289 questions.

That convergence can have several explanations. The remaining questions may contain ambiguity, inaccessible evidence, judge limitations, or search failures that additional model capacity does not resolve. A ceiling observed on one benchmark does not establish a general limit, but it warns against assuming that scale fixes every search error.

This is the central reversal in the AllSpark Iris search agent release. A 256K context window appears enormous, yet a blunt reset can materially improve results. More accumulated context is not always more useful context.

The finding has consequences for product design. An agent interface often presents a single conversation as though continuity were inherently valuable. Behind the interface, a reliable system may need to summarize, prune, branch, or restart that conversation several times.

It also complicates comparisons between open and closed agents. Two products may use the same underlying model but produce different outcomes because one manages context more effectively. Conversely, a weaker model can appear stronger when paired with a more expensive search strategy.

Developers evaluating Iris should therefore treat the context policy as a configurable component. They should measure success rates beside latency, token consumption, search requests, and restart frequency. A higher score may be worth the added cost for occasional investigations but unsuitable for high-volume workflows.

Deliberate forgetting is not evidence that context windows have stopped mattering. A larger window delays the point at which history becomes restrictive. The Iris results instead show that an agent still needs a policy for deciding what deserves to remain inside that window.

What the Iris Scores Do Not Yet Establish

The reported lead is credible enough to test, but it is not independent proof of superior real-world research.

The central numbers come from AllSpark’s own evaluation. The team provides unusually useful details, including unmanaged results and its harness configuration. Independent groups still need to reproduce the scores using the released weights and code.

Baseline comparability is another limitation. Rival projects may use different search engines, page parsers, turn limits, context controls, and judges. A table assembled from separate public reports cannot isolate checkpoint quality as cleanly as a shared evaluation environment.

Even small infrastructure changes can alter search outcomes. Search rankings change over time, websites block automated readers, and extracted pages can omit important content. A model evaluated next month may receive different evidence for the same query.

The judging layer introduces further uncertainty. Iris uses each benchmark’s official evaluation prompt with an LLM-based judge where applicable. Such judges can be sensitive to formatting, verbosity, and answer equivalence. A parseable short answer may score differently from a defensible report containing the same underlying conclusion.

The search benchmarks also capture only part of research quality. A correct final answer does not necessarily show that every cited source was trustworthy. It does not establish resistance to manipulated pages, prompt injection, coordinated misinformation, or stale evidence.

AllSpark reports a single rollout per question for its primary evaluation. Pass@1 is useful because it avoids selecting the best answer from many attempts. However, it leaves open how variable the system is across repeated runs with changing search results.

The retry experiments make the cost question more urgent. A complete restart can consume another sequence of searches and generations. AllSpark explicitly treats combined reset and retry results as an upper-bound exploration rather than its main performance setting because retries impose substantial inference cost.

The project’s openness is also incomplete at launch. Model weights and evaluation code are public, while the broader data and training recipe are expected in stages. The paper explains the process, but full reproducibility depends on the eventual release of concrete datasets, filters, prompts, and training components.

Licensing the checkpoints under Apache 2.0 lowers legal barriers to experimentation. It does not guarantee that every upstream corpus or generated trajectory can be redistributed. Users should inspect the provenance and terms of future data releases before building commercial derivatives.

Iris-pro creates a separate deployment hurdle. Activating 17B parameters is more efficient than activating all 397B, but the full checkpoint still requires substantial memory and infrastructure. Its benchmark advantage may be irrelevant to teams that cannot serve it within acceptable latency and operating limits.

Iris-mini may be a more revealing product candidate. Its smaller active footprint offers a plausible path toward controlled deployment, while its dependence on context management exposes the surrounding costs. Teams should test full-task economics instead of inferring efficiency from active parameters alone.

Real-world evaluations must also include abstention. A useful research agent should recognize when evidence is contradictory, inaccessible, or insufficient. Public benchmarks often reward a final answer, while business workflows sometimes require the agent to stop and report uncertainty.

Security testing is equally important. Search agents consume untrusted text from pages that may contain instructions aimed at the model. Neither strong retrieval scores nor benchmark leakage controls establish resistance to indirect prompt injection.

These caveats do not reduce Iris to a marketing exercise. The project supplies enough artifacts for serious scrutiny and states several limitations in its own paper. That is precisely why independent reproduction now matters.

The correct near-term conclusion is narrow. AllSpark reports leading peer-scale results under documented settings, and its unmanaged scores remain competitive. Whether Iris becomes a dependable research engine will depend on tests that extend beyond benchmark answer accuracy.

Three Signals Will Determine Whether Iris Holds Its Lead

The next stage is about reproducibility, operating cost, and performance beyond the four reported benchmarks.

The first signal is independent reproduction of the headline results. Researchers should run the released checkpoints through the public harness while recording tool calls, context resets, token use, and failures. Close reproduction would strengthen AllSpark’s claim that the release represents a transferable system rather than a private evaluation environment.

A large gap would not immediately invalidate the work. Search results and page availability change, while hardware and serving software can affect long generations. Reproducers should document those differences and test both the managed and unmanaged settings.

The unmanaged numbers deserve particular attention. They offer a cleaner view of behavior learned during post-training because the harness contributes less recovery assistance. If outside evaluators reproduce that advantage, Iris’s training pipeline becomes a stronger competitive reference.

The second signal is the promised release of training data and implementation details. The model weights show the resulting policy, but they cannot reveal every decision used to generate tasks or filter trajectories. Concrete artifacts would let researchers inspect difficulty, diversity, leakage risks, and source coverage.

That release would also permit ablation studies. An ablation removes one component to measure its contribution. Researchers could test whether entity rewriting, closed-book filtering, turn-level filtering, live-search reinforcement learning, or SFT-RL cycling supplies the largest gain.

If those components transfer to other base models, the Iris recipe may matter more than either checkpoint. Competing teams could adopt the same pipeline and narrow the leaderboard gap. Failure to transfer would suggest the results depend more heavily on specific Qwen bases or internal training conditions.

The third signal is evaluation under realistic budgets and adversarial conditions. A useful test should cap search requests, wall-clock time, and generated tokens. It should also measure citations, source quality, abstention, and resistance to malicious page content.

Results under those limits would clarify the practical tradeoff between Iris-mini and Iris-pro. The larger model might solve tasks with fewer turns, while the smaller model might compensate through resets and additional searches. Buyers need total system cost and reliability, not parameter counts alone.

Competitor responses will provide another part of this signal. Open-agent projects can strengthen their own reporting by publishing managed and unmanaged results. Closed providers can offer clearer evidence about citation accuracy, latency, and repeated-run consistency.

For developers, the best immediate action is to treat Iris as a testable research stack. Start with a bounded task set drawn from your own domain. Record whether the agent retrieves the right sources, survives incomplete pages, and admits uncertainty when evidence fails.

Then change one system component at a time. Disable context resets, restrict search calls, replace the search provider, or shorten the context window. Those tests reveal whether the AllSpark Iris search agent is genuinely useful for your workload and where its benchmark advantage comes from.

The release has already delivered one durable lesson. Search-agent performance lives in the interaction between a model and its operating system. The next question is whether independent testing confirms that Iris has improved both parts, or mainly found a better way to keep searching after its context fills.