360 CyberGym Results: Tulongfeng Claims 99.4% Vulnerability Reproduction

360 says its Tulongfeng security system reached approximately 99.4% vulnerability reproduction success on CyberGym, an evaluation built around historical software bugs. The 360 CyberGym results appeared in coverage dated October 30, 2026. They describe a near-ceiling score on a defined task, rather than a general success rate for discovering unknown vulnerabilities.

The distinction matters for security teams comparing specialized systems with agents built around models from OpenAI, Anthropic, and Google. Reproducing an identified bug requires substantial engineering work. Searching an unfamiliar application for an unspecified flaw presents a different problem. A strong result on the former deserves attention without becoming evidence for every claim about the latter.

The central issue is how much operational value a near-perfect reproduction score establishes. Buyers need evidence connecting successful benchmark runs with dependable investigations in their own environments. For 360, the opportunity is to demonstrate that connection through transparent evaluation details and verifiable security outcomes.

What the 360 CyberGym Results Actually Establish

The reported result concerns reproducing known vulnerabilities, and its exact evaluation date remains unclear.

An October 30 vulnerability reproduction report, published by Xinhuanet and attributed to China.org.cn, described Tulongfeng’s performance. It reported success rates of 99.27%, 99.47%, and 99.40% across three evaluations. The stated average was 99.38%, covering all 1,507 tasks.

That gives the story a verified publication date. It does not establish when testing finished or when a leaderboard entry changed. The report described the result as recent, without supplying those dates. Readers should therefore treat October 30 as the coverage date, rather than a confirmed evaluation milestone.

The coverage also called Tulongfeng the leaderboard leader. That ranking claim needs separate treatment from the reported numerical result. The publicly retrieved benchmark page did not expose a Tulongfeng entry sufficient to independently check its placement and configuration. This analysis therefore attributes both the score and leadership claim to the reporting.

A benchmark result has several layers of evidence. A published announcement establishes what someone reported. A detailed submission explains how the evaluation operated. Independent reproduction tests whether another team obtains comparable outcomes. Those layers answer different questions, and publishing the same announcement across several websites does not substitute for them.

The terminology also needs care. Vulnerability reproduction means generating an input that demonstrates a specified software flaw. Discovery means identifying a flaw whose existence or location was not already supplied. Describing a reproduction score as a discovery rate expands the claim beyond what the evaluation directly measures.

For readers encountering the headline through a news feed, this is the essential correction. The announcement concerns an automated security investigation task with a defined starting point. It does not establish that Tulongfeng finds nearly every vulnerability in arbitrary software, nor that deployed customers receive equivalent results.

That narrower interpretation still leaves an important development. If the reported performance holds under documented conditions, Tulongfeng would represent a highly capable system for turning known bug descriptions into executable evidence. Such work occupies a practical place between receiving a report and deciding what engineering action it warrants.

The significance rests on that connection. A security system becomes useful when its output helps someone verify a problem, understand its scope, and make a defensible decision. The score is an entry point for evaluating that usefulness, rather than a complete description of it.

Known Bugs Give Security Agents a Defined Starting Point

CyberGym measures a demanding investigation task, but it supplies information that open-ended discovery must first uncover.

According to the official CyberGym evaluation, its main Level 1 setting provides a vulnerability description and an unpatched codebase. A proof of concept, or PoC, is an executable test that demonstrates the specified flaw. Success requires the test to trigger the vulnerable version while leaving the patched version clean.

This creates a meaningful engineering challenge. An agent, a model-driven system that takes actions through software tools, must connect a written description with actual program behavior. Its output must survive execution-based checking. A plausible explanation alone cannot satisfy the task.

However, the supplied description changes the search problem. The system begins with evidence that a relevant vulnerability exists. It can organize its investigation around that premise. An open-ended review must also decide where to search, which suspicious behavior matters, and whether further investigation deserves additional resources.

Consider the difference from a developer’s perspective. A specific bug report provides a reason to examine particular behavior. A broad request to assess a large application offers fewer constraints. Both require judgment, but they place that judgment at different points in the investigation.

This distinction explains why the reported result pressures security vendors without settling the entire market. Suppliers that sell automated investigation face a concrete question: can their systems produce similarly convincing evidence under comparable conditions? Suppliers promising broad discovery face a wider question about finding useful starting points.

Neither group can answer those questions with fluent explanations alone. A system that describes a vulnerability accurately still needs to show the relevant behavior. Conversely, a system that produces an unexplained failure leaves engineers with additional interpretation work. Useful automation connects the hypothesis, test, observed behavior, and conclusion.

The benchmark’s bounded task also helps buyers separate capability from coverage. Capability asks whether a system can complete a particular investigation. Coverage asks which applications, environments, and vulnerability classes it can assess. A high score supports a capability claim within its evaluation scope; coverage requires additional evidence.

That separation should influence procurement discussions. Buyers should ask vendors to describe the work their systems reliably complete. “Reproduces reported software vulnerabilities” conveys a clearer promise than a broad statement about autonomous security. It also gives a pilot project a task that engineers can evaluate directly.

The operational stakes are therefore specific. If automation can complete reproduction work reliably, analysts can spend less effort establishing whether a report describes an observable problem. Their remaining work includes assessing impact, coordinating ownership, and reviewing the proposed response. Those responsibilities remain important even when the initial technical investigation improves.

360 Puts Specialized Coordination Behind the Score

Tulongfeng’s reported design emphasizes coordinated security work, making the system architecture part of the claim.

The October coverage describes a security-focused language model working with specialized agents. According to that account, different roles divide the investigation into stages, from developing hypotheses through verification. The proposed advantage comes from combining security knowledge, tools, and coordination.

This is a system-level proposition. A model’s ability to reason about code matters, but so does the process surrounding it. The surrounding software determines what evidence the model receives, which actions it can take, and how it handles unsuccessful attempts. Those choices influence the final outcome.

Specialization offers a plausible rationale. A role focused on locating relevant code can produce evidence for another role testing a hypothesis. A verification role can assess whether the observed behavior supports the original claim. In principle, dividing responsibilities makes complex investigations easier to manage.

That explanation remains a rationale, rather than a verified account of why Tulongfeng obtained its reported score. The coverage does not provide enough controlled comparisons to isolate the contribution of each component. It cannot establish whether coordination, model training, tools, or resource allocation supplied the largest improvement.

A useful contrast comes from Tsinghua University’s RedbudAI work. Its published single-model evaluation reports 86.2% success using GLM-5.2, with a separate repeated-attempt analysis reaching 93.6%. The authors distinguish those measurements rather than presenting them as interchangeable outcomes.

RedbudAI also describes local execution, restricted networking, and a designated final test for each task. These details make its result more interpretable. They reveal the operating conditions behind the number, even though they do not establish a controlled comparison with Tulongfeng.

The comparison illustrates a broader analytical problem. A single-model system can still contain substantial orchestration, meaning software that organizes tools and investigation steps. A multi-agent system can vary widely in how independently its roles operate. Labels alone tell buyers little about the actual work performed.

Accordingly, Tulongfeng’s reported lead cannot establish that specialized agent teams are universally superior to other designs. Establishing that would require holding other conditions comparable. Without such controls, the result describes the performance of a complete configuration.

The same restraint applies to comparisons involving OpenAI, Anthropic, or Google. A specialized security system and a general model connected to an agent framework are different evaluation objects. Any comparison should identify the complete configuration before drawing conclusions about the underlying model.

For developers, this shifts attention toward evidence passing between stages. Does the investigator preserve the original hypothesis? Does the verifier inspect relevant execution results? Can another engineer reconstruct the reasoning? These questions connect architecture with practical trust more directly than the number of agents does.

That is where 360’s technical story becomes interesting. Its claim invites examination of how security expertise enters the workflow and survives successive decisions. A detailed release would help distinguish repeatable engineering advantages from an attractive architectural description.

Near-Ceiling Scores Make Evaluation Details More Important

A result close to the maximum increases the importance of scoring rules, resource limits, and reproducible evidence.

CyberGym’s maintainers caution that teams evaluate and submit their own results. They also warn that small differences between strong systems need not represent meaningful capability gaps. Those qualifications directly affect how readers should interpret a headline about first place.

At high performance levels, rank alone offers limited explanation. Buyers need to understand whether systems differ in task success, resource use, reliability, or evaluation conditions. A small numerical advantage can coexist with important operational disadvantages. It can also reflect a real improvement that deserves closer investigation.

The official submission requirements recognize this issue. They require cost reporting alongside success rates and request model usage, task duration, example logs, and final outcomes. These disclosures help reveal what completing the work actually required.

For an enterprise buyer, duration matters because security investigations compete with engineering deadlines. Resource use matters because a configuration must fit available infrastructure. Variability matters because occasional long-running tasks can complicate scheduling. None of these considerations invalidates a high success rate; they determine its practical value.

Scoring rules deserve equal attention. CyberGym’s final-submission rules require the system to designate its final answer. Crediting any successful candidate instead would answer a different question. The distinction becomes important when a system produces several candidates during an investigation.

Selecting the correct output is itself useful capability. Engineers need a result they can act on without searching through an unexplained collection of attempts. A system that eventually produces valid evidence still needs a dependable way to recognize and present that evidence.

The reported three evaluations also leave a methodological question. Repeating an evaluation can measure consistency, but repeated opportunities within a task can change the success metric. The coverage does not fully explain that relationship for Tulongfeng. Detailed documentation would clarify what remained constant across runs.

The word “independent” deserves restraint here. Separate runs do not necessarily mean separate external evaluators. The reviewed coverage does not identify an outside laboratory reproducing the full result. Readers should not infer independent auditing from a description of repeated evaluations alone.

Network access creates another comparison issue. Access to public information can introduce shortcuts when historical vulnerabilities already have published fixes or explanations. That is a general benchmark concern, not evidence of misconduct by 360. Transparent access policies help reviewers assess whether a result reflects the intended task.

Other configuration choices also shape interpretation. A runnable vulnerable application offers different feedback from source inspection alone. Memory accumulated across tasks changes what the system brings into later investigations. Documentation should make such choices visible so that readers can compare like with like.

These unresolved details support a measured conclusion. The 360 CyberGym results warrant technical attention, while a definitive superiority claim requires more evidence. Uncertainty about methodology should guide follow-up reporting, rather than become an unsupported accusation or disappear beneath the headline.

Enterprise Value Depends on What Happens After Reproduction

The commercial question is whether a successful investigation reduces the work required to make and implement a security decision.

A reproduced vulnerability gives an engineering team something concrete to examine. It can support communication between researchers, maintainers, and application owners. However, the organization still needs to determine whether the affected code exists in its environment and what response makes sense.

Consider a hypothetical team assessing a report about a software dependency. A system might reproduce the behavior in a prepared environment. The team must then check whether its deployed version and usage expose the same problem. That assessment connects the technical result with operational relevance.

The next decision concerns remediation, meaning the work that removes or reduces the security problem. A proposed fix requires review and testing. Teams must also account for deployment constraints and unintended behavior. Reproduction is valuable evidence for that process, but it does not complete every step.

This is why benchmark categories matter. The separate end-to-end benchmark called CyberGym-E2E covers vulnerability discovery, proof-of-concept generation, and patch generation. Its broader scope highlights the additional work involved in evaluating a complete defensive lifecycle.

That broader evaluation does not make reproduction unimportant. Instead, it identifies the boundaries between tasks. An organization might benefit from an excellent reproduction assistant without delegating discovery or patching to it. Clear boundaries allow teams to adopt useful capabilities while retaining appropriate review.

The October coverage also makes broader claims about Tulongfeng’s deployments and vulnerability findings. Those statements deserve their own supporting evidence. Customer participation, confirmed findings, and completed remediation represent different kinds of outcomes. They should not be combined into a single impression of effectiveness.

For buyers, the most informative evidence would connect a finding with its resolution. Was the issue confirmed by the responsible maintainer? Did the investigation produce enough context for a fix? Could the receiving team reproduce the result? These questions test whether automation transfers useful work across organizational boundaries.

They also help reveal hidden human effort. A system might perform well after extensive environment preparation or expert intervention. Such assistance can be entirely legitimate. Buyers simply need to understand which responsibilities remain with their own staff.

Knowledge management becomes relevant at this handoff. Engineers need the original report, environment details, reproduction notes, and review decisions to remain connected. A searchable engineering knowledge base addresses that documentation need, separate from the security system’s technical capability.

This connection matters when investigations change owners. A result that only its original operator understands creates additional work. A documented result allows another engineer to inspect the assumptions and continue the response. Traceability therefore belongs in discussions of operational value alongside automated success.

The enterprise implication is specific: security vendors face pressure to demonstrate complete, reviewable units of work. For 360, strong reproduction performance would provide a credible starting point. Evidence of dependable handoffs and resolved findings would establish how far that capability extends into everyday security operations.

Three Signals Will Clarify the Result

The next useful evidence concerns the submitted configuration, its efficiency, and outcomes beyond historical reproduction.

First, watch for a detailed Tulongfeng evaluation release that connects the reported score with a specific benchmark configuration. It should identify models, tool access, scoring rules, and representative investigation records. This would let readers understand exactly what the reported result describes.

A clear release would strengthen confidence in the claim’s scope. It would also make external review more practical. If documentation remains limited to headline percentages, confidence should remain correspondingly narrow. An absent write-up would leave questions unresolved without establishing that the reported result is false.

Second, watch for comparable efficiency disclosures from 360 and other security-agent developers. Task duration, model usage, and resource requirements would reveal how different systems obtain their results. Those measurements would help buyers evaluate operational fit alongside task completion.

An efficient system that consistently produces reviewable evidence would support a stronger adoption case. A configuration requiring extensive resources might still suit specialized investigations. The point is to identify the workload it serves, rather than assume every high-scoring system belongs in the same deployment category.

Third, watch for documented findings and remediation outcomes on software outside the historical evaluation. Maintainer confirmation, reproducible evidence, and completed fixes would provide concrete support for broader discovery claims. Clear reporting should distinguish newly identified problems from reproductions or variants of known issues.

These signals should guide coverage over the coming months. They would strengthen or narrow the interpretation of Tulongfeng’s reported performance without requiring predictions about who will dominate security automation. The immediate news remains the claim itself and the engineering approach behind it.

For developers and security buyers, use the 360 CyberGym results as a prompt to inspect evidence before changing your workflow. Ask what task the system completed, what information it received, and how reviewers checked its final output. Then connect those answers with the investigation your team actually needs to finish. Keep the reported score beside its configuration, limitations, and supporting records. Revisit that assessment when detailed artifacts or confirmed findings become available. If you are comparing vendors, request the same disclosures from each one. Which system leaves your engineers with a result they can independently understand, reproduce, and use to resolve a security problem?