Training AIs to Solve CAPTCHAs: Salman and Bicakci Test a Solver That Learns

Oguzhan Salman and Kemal Bicakci report 85.4% classification accuracy with a CAPTCHA solver designed to improve through use. Their research on Training AIs to Solve CAPTCHAs challenges a familiar defensive assumption: a puzzle that defeats an automated solver will keep defeating it. The proposed system combines specialized image recognition with broader visual reasoning.

The central contest is between fixed visual defenses and an adaptive solver. A CAPTCHA, a test intended to distinguish humans from automated software, presents a recognition problem. Changing that problem can interrupt a specialized model. The harder question is whether the interruption lasts when another model supplies training examples.

Google’s reCAPTCHA provides the practical reference point, but the implications extend to how websites evaluate protection. A successful puzzle defense today does not establish durable resistance. For developers and security buyers, the research shifts attention toward recovery time, complete verification outcomes, and the cost of legitimate access.

Training AIs to Solve CAPTCHAs Moves Beyond Fixed Models

Salman and Bicakci, researchers at Istanbul Technical University, posted their preprint on September 2, 2026. Bruce Schneier highlighted the adaptive CAPTCHA research on October 19. The distinction matters: the recent attention concerns an existing research release, rather than a newly announced commercial product.

The consequential change is the proposed learning loop, rather than the ability to recognize objects in a puzzle.

The system pairs YOLOv8, a specialized visual recognition model, with a vision-language model, which processes images alongside text. A routing mechanism chooses between them according to confidence. This arrangement gives the system a specialized path and a broader fallback within the same architecture.

The authors report 85.4% overall accuracy and 84.2% macro accuracy across 16 classes. Overall accuracy counts predictions across the evaluated samples. Macro accuracy weights each class equally, making weaker categories harder to hide behind common ones. These measurements describe classification performance, rather than a universal website access rate.

That distinction changes how readers should interpret the headline figure. A website does not grant access because a model correctly labels an isolated image. It evaluates an interaction that can include multiple selections, refreshed images, verification decisions, and retries. A classification benchmark measures an important component of that interaction.

The paper also reports a live evaluation on Google’s reCAPTCHA v2 demonstration page. Across 100 sessions, all eventually passed, while 181 of 206 puzzles succeeded on their first attempt. Those results provide deployment evidence beyond static images. They remain observations from the authors’ test conditions, rather than an independent audit across websites.

For a defender, the important distinction is between immediate recognition, eventual completion, and sustained resistance. Each answers a different question. Combining them into a single success percentage would obscure where the system works and where other protections still matter.

Consider a registration flow that presents an image challenge after suspicious activity. Correct image selections establish that the visual task was solved. They do not establish that subsequent account behavior is acceptable. Security analysis needs to preserve that boundary even when the image solver performs well.

Training AIs to Solve CAPTCHAs therefore becomes a question about behavior over repeated encounters. The research asks whether a solver’s future capabilities change after it sees something unfamiliar. That is a more demanding test for defenses than comparing an unchanged model against a fixed image collection.

A Visual Challenge Can Interrupt a Bot Without Stopping Its Learning

A defense that causes errors needs a separate explanation for why those errors should persist.

The pressure falls most directly on security teams treating visual difficulty as a durable barrier. Their immediate measurement might show that a modified puzzle defeats an existing detector. That result is useful, but incomplete. It says little about what happens after the solver receives feedback or updates its model.

Imagine an authorized evaluation of a website’s registration protection. A new image category initially frustrates the test solver. The security team records the failure and considers the change effective. If the evaluation ends there, it never measures whether later exposure removes the advantage.

An adaptive evaluation would continue after that initial failure. It would examine whether the solver improves, what information drives improvement, and whether the protected action remains blocked. The decisive outcome is sustained resistance under defined conditions. An impressive initial error rate cannot answer that question alone.

This creates pressure on procurement language as well. A claim about detecting automated traffic needs a defined scope. Does it concern image recognition, challenge completion, suspicious sessions, or abusive actions? Buyers need those distinctions before comparing products or interpreting a laboratory result.

The same reasoning applies to the operator’s workload. Replacing a visual challenge is worthwhile when it produces meaningful protection. Its value becomes less clear if maintaining that protection requires frequent changes that inconvenience legitimate visitors. The relevant comparison includes both defensive benefit and ongoing maintenance.

There is also a timing issue. A defense can retain practical value even when it eventually loses effectiveness. A temporary barrier might protect a short registration window or interrupt an ongoing campaign. The research does not establish that every temporary defense is useless.

Instead, it suggests a more precise question: how much useful delay does the intervention create? That delay should be measured against the attacker’s objective and the defender’s response time. A short interruption and a lasting obstacle deserve different operational descriptions.

Website owners should also distinguish human verification from authorization. A visitor might complete a challenge and still lack permission for a particular action. Conversely, a legitimate user might struggle with verification despite having permission. Treating the visual result as a complete trust decision collapses these separate judgments.

For AI product developers, the same boundary matters when agents interact with websites. Technical ability to complete a screen does not establish permission to proceed. Product behavior should follow the user’s authority and the destination service’s rules, even as visual capabilities improve.

The immediate implication is therefore specific. Teams relying heavily on a fixed recognition gap need evidence that the gap survives repeated exposure. Teams already evaluating complete interactions should incorporate adaptation into those evaluations. Neither conclusion requires declaring every CAPTCHA obsolete.

The Fast Detector and the Broad Model Have Different Jobs

A standalone specialized detector and a general visual model face different constraints. The detector focuses on patterns represented in its training. The broader model offers another way to interpret unfamiliar content. Combining them changes which component must handle each encounter.

The architectural argument is that general reasoning should help improve specialized execution, rather than repeat every familiar decision indefinitely.

The authors describe a confidence-based router, a component that directs uncertain predictions to the broader model. The operating principle resembles escalation within a support process. Routine cases follow the specialized path. Cases requiring another kind of judgment receive additional attention.

That arrangement makes confidence quality an important issue for evaluation. A confident mistake does not automatically reach the fallback. An overly cautious router can send routine work through the more demanding path. Measuring the combined system therefore requires examining its decisions about escalation.

This is where the comparison between specialized and general AI becomes useful. A broad model can cover cases outside a detector’s established categories. A specialized model can concentrate on recurring patterns. The research treats those strengths as complementary parts of an adaptive CAPTCHA solver.

Earlier work supplies historical context without establishing a directly comparable ranking. Andreas Plesner, Tobias Vontobel, and Roger Wattenhofer reported complete challenge-solving success under their setup in the 2024 reCAPTCHA v2 study. Their work used YOLO-based recognition and also examined browser context.

That earlier result prevents an exaggerated interpretation of the current news. Automated image challenge solving did not begin with this preprint. Different studies also use different protocols, retries, environments, and outcome measures. Their percentages cannot establish a clean performance hierarchy without controlling those differences.

The more useful comparison concerns what each architecture retains after an encounter. A fixed classifier applies existing parameters. A general model can interpret a new input without necessarily changing the surrounding system. An adaptive architecture introduces a route for experience to influence later performance.

This distinction also keeps broader AI claims in perspective. An agent remembering a conversation is different from updating a recognition model. Retrieving a previous example is different from learning a new category. Reports about improvement should identify which mechanism actually changed.

For the visual defense, those implementation differences affect the threat being evaluated. A challenge might defeat a particular classifier while remaining legible to another model. If the system can use that second judgment productively, a component failure need not become a lasting system failure.

The paper’s hybrid solver evaluation provides the underlying experimental account. Its findings should be assessed as the authors’ reported results. The central architectural question remains understandable without accepting a universal claim: does experience reliably improve later execution?

That question applies beyond CAPTCHA research, but the article’s security conclusion stays narrower. A fixed visual barrier faces a different opponent when the solver can adapt. Evaluations designed around an unchanged model risk measuring an advantage that disappears after deployment.

A Screenshot Interface Changes the Test, Not the Meaning of Trust

The researchers describe a system that reads screenshots and sends operating-system input events. It does not rely on browser automation or the DOM, the browser’s structured representation of a page. The browser functions as a visible interface for the solver.

Removing access to page structure strengthens the visual interaction claim, but it does not establish universal invisibility.

This matters because recognizing a puzzle and operating its interface are separate tasks. A model can understand an image while failing to select the correct location. A complete interaction also needs to handle changes on the screen and determine whether the task has finished.

The screenshot approach places those interaction demands inside the evaluated system. It supports a claim about operating through visible content. That is relevant to computer-use agents, which must connect visual interpretation with actions. It should still be described within the tested environment.

A website can evaluate more than how an image was read. Therefore, the absence of a particular automation mechanism does not establish the absence of every detectable signal. Claims about being undetectable would require their own evidence. The reported interface design does not settle that broader question.

The live demonstration results also need careful boundaries. Eventual session completion includes recovery from failed attempts. First-attempt puzzle success describes a different outcome. Both are useful, but they inform different decisions about user experience, reliability, and defensive friction.

A buyer evaluating a protection system should ask which result matches the business objective. Preventing an abusive registration concerns whether the action ultimately occurs. Limiting interaction load concerns how much work precedes that outcome. Measuring visual accuracy alone cannot answer both.

Existing products also show why the market cannot be reduced to image puzzles. Google documents interaction risk scoring for reCAPTCHA v3, which returns a score without presenting an interactive challenge. Its documentation connects that score with actions selected by the website operator.

Cloudflare describes noninteractive browser challenges for Turnstile, using signals from the visitor’s browser environment. These product descriptions illustrate a broader defensive surface. They do not establish that either service defeats the research system.

The neutral comparison is about what gets measured. A visual solver addresses the recognition and interaction tasks it encounters. A risk-based service evaluates additional context according to its own design. Evaluating one does not automatically evaluate the other.

For product teams, this means a familiar checkbox should not define the entire security discussion. The protected action, surrounding controls, and observed abuse all matter. An authorized test should state which of those layers it actually exercised.

Training AIs to Solve CAPTCHAs raises a concrete challenge for visual verification. It does not justify collapsing every verification product into the same category. The architecture deserves attention precisely because its demonstrated scope can be described without turning it into a universal access claim.

The Evidence Needs to Survive Adaptation on Both Sides

An adaptive solver deserves an adaptive evaluation, including defenses that change in response to its behavior.

The strongest skeptical reading asks whether the reported advantage survives outside the chosen conditions. A research system can produce a meaningful result without covering every operating environment. Replication should test whether the same mechanism remains useful when its assumptions become less favorable.

One important question concerns the quality of feedback. A model-generated answer is a prediction, rather than independent ground truth. If later learning relies on those answers, evaluators need to examine what happens when they are wrong. Improvement should be measured alongside persistent mistakes.

Another question concerns preservation of existing performance. Learning something new is less useful if the system loses competence on familiar inputs. A credible evaluation needs to track both. Otherwise, apparent progress on recently encountered cases can conceal regression elsewhere.

Confidence deserves similar scrutiny. A router can only escalate cases according to the information available to it. If difficult inputs produce misplaced certainty, fallback behavior might be weaker than expected. Testing the routing decisions matters alongside testing the models themselves.

These are evaluation questions, rather than claims that the reported system necessarily fails. They identify evidence that would strengthen or narrow the argument. A cautious reading keeps the observed result separate from assumptions about every future input distribution.

Independent reproduction would provide another useful boundary. An artifact release allows outside researchers to inspect and test a method. It does not, by itself, establish that outsiders obtained the same outcomes. Reproduction should include comparable conditions and clearly identified departures from them.

Operational context matters as well. A demonstration page is a useful test environment, but a production service protects a particular workflow. That workflow can introduce additional decisions before or after a challenge. Generalizing across services requires evidence about those complete interactions.

Defensive changes also need to account for legitimate access. W3C’s CAPTCHA accessibility analysis documents barriers associated with human verification tasks. It also explains why an audio alternative does not automatically provide equivalent access.

That perspective complicates the simplest response to stronger image recognition: making puzzles harder. Increased difficulty needs evidence of improved protection alongside acceptable human outcomes. Otherwise, a change can impose friction without establishing durable resistance to automation.

The relevant outcome is the relationship between protection and access. A defense that blocks legitimate users has a cost even if its automated failure rate looks favorable. An adaptive evaluation should preserve that visibility rather than reporting only the solver’s performance.

For teams following the research, a searchable knowledge base can help keep protocols, results, and limitations together. The practical value lies in preserving context. A remembered percentage without its measurement conditions can mislead a later security decision.

Watch Replication, Complete Sessions, and Defensive Response

The next useful evidence should clarify durability, rather than add another isolated headline percentage. Over the next several months, readers can look for concrete developments in independent evaluation and product response. These are observation priorities, not announced milestones.

The decisive question is whether the solver’s advantage remains useful when both the environment and the defender respond.

First, watch for independent reproductions of the adaptive behavior. A persuasive replication would examine whether repeated exposure improves later results under documented conditions. It should distinguish changes in recognition from changes in retries or interaction handling.

Successful reproduction would strengthen the claim that adaptation is a meaningful architectural advantage. Failure to reproduce it would narrow the conclusion toward the original setup. Either outcome would be more informative than another summary repeating the benchmark figures.

Second, watch for broader reporting on complete verification sessions. Evaluations should distinguish image predictions, puzzle attempts, eventual completion, and time spent. They should also identify which protected action, if any, followed verification.

Consistent completion across varied, authorized environments would strengthen the operational case. Strong classification with unreliable session completion would indicate a narrower capability. This distinction gives defenders a way to respond proportionately instead of reacting to an ambiguous success rate.

Third, watch for defensive evaluations that allow the solver to adapt after a change. A useful response would measure whether protection persists, how much delay it creates, and what happens to legitimate visitors. The initial failure of an unchanged model is only the starting observation.

If defensive changes retain effectiveness after repeated exposure, they would weaken the broadest interpretation of the adaptive threat. If the advantage repeatedly disappears, they would strengthen the case for shifting attention beyond visual difficulty. Human completion outcomes should accompany either result.

For developers, these signals provide a concrete reading discipline. Ask what changed between attempts, what counted as success, and who verified the outcome. For buyers, ask how those measurements connect to the action being protected. For AI users, distinguish a capable interface agent from an authorized one.

Training AIs to Solve CAPTCHAs now warrants a discussion about persistence, feedback, and complete interactions. Read the evaluation conditions before carrying a success percentage into a product decision. Preserve the difference between a model recognizing an image and a service resisting abuse. When the next claim arrives, look for independent replication, session-level outcomes, and a defense tested after adaptation. Those observations will show whether this architecture delivers a repeatable advantage or remains dependent on its original conditions. For any workflow you operate, start with the protected action and trace the evidence back from there: what would demonstrate that the barrier still works after the solver has had a chance to learn?