BDH-CQ Challenges Token-by-Token AI Reasoning With Recurrent Latent Memory

BDH-CQ reached 29.5% pass@2 on ARC-AGI-1 with 150 million parameters, despite never verbalizing its intermediate reasoning. Pathway researchers introduced the system on August 10, 2026, through a preprint that combines recurrent memory with iterative latent computation. The result challenges a common assumption behind modern reasoning models: harder problems require longer streams of generated reasoning tokens.

The reported score does not make BDH-CQ the most accurate ARC solver. Its significance comes from how the model learns each test task and how little computation the authors report using. Demonstrations update an internal memory during inference, while fixed model parameters process the query repeatedly inside a continuous workspace.

That approach places BDH-CQ against token-by-token reasoning, not against one company or flagship model. Chain-of-thought systems turn intermediate computation into language before reading those tokens back. BDH-CQ keeps that computation internal, potentially avoiding the latency and decoding overhead attached to long reasoning traces.

The evidence remains narrower than the headline suggests. The evaluation covers visual grid transformations, not open-ended language, coding, or scientific research. Important architectural details also remain proprietary, limiting independent reproduction and scrutiny.

BDH-CQ Learns a Task Without Updating Its Parameters

BDH-CQ turns demonstrations into a temporary recurrent memory, then uses that memory to solve a new query.

The Pathway-led team presented BDH-CQ in an August preprint. Its authors include researchers from Pathway, Bielik AI, and New York University. The paper describes BDH-CQ as an extension of the Dragon Hatchling architectural family.

Each evaluation task provides several input-output examples and a new input requiring an answer. BDH-CQ processes those demonstrations sequentially. Each example changes a recurrent state that carries context into later inputs.

The model then encodes the query together with that accumulated state. It repeatedly transforms a high-dimensional latent workspace, meaning a continuous internal representation used for computation. Only the final candidate answer gets decoded.

No gradient update occurs during this process. The model’s trained parameters remain fixed, while its recurrent state changes with the supplied evidence. That distinction separates inference-time memory from conventional fine-tuning.

The researchers also say evaluation-task identifiers and evaluation demonstration pairs were excluded from training. Their model instead trained on a mixture of ARC-style sources and privately curated examples. Public sources included ARC-AGI-1 training data, RE-ARC, ConceptARC, ARC-Heavy, and ARC-GEN100K.

This setup matters because many compact ARC systems adapt directly to each evaluation puzzle. They optimize model components, use puzzle-specific identities, or vote across augmented versions. BDH-CQ claims to infer the transformation from demonstrations alone.

The testbed was the public ARC-AGI-1 evaluation split. ARC tasks use small colored grids to express transformations involving objects, symmetry, counting, topology, and spatial relations. A solver must infer an unstated rule from a handful of examples.

The public split contains 400 tasks. BDH-CQ solved 97 on its first ranked candidate and 118 when either of two candidates could count. Those results correspond to 24.25% pass@1 and 29.5% pass@2.

Across 419 individual test pairs, it reached 25.78% pass@1 and 31.03% pass@2. Task accuracy is stricter because every required test output must be correct. A single wrong grid makes the entire task unsuccessful.

The researchers calculated inference from measured H200 GPU time. Their default configuration used approximately 0.85 H200 GPU-seconds per task. They argue that this result moves beyond the previously reported ARC-AGI-1 cost-accuracy frontier.

That efficiency claim is the event’s central news value. BDH-CQ does not win by producing the highest aggregate score. It claims a new operating point where meaningful performance needs unusually little inference compute.

The paper also reports a black-box audit by coauthors affiliated with Bielik AI and New York University. They reproduced the 29.5% score without access to model weights. That offers a useful check on the deployed service, although it is not equivalent to unaffiliated replication.

Most importantly, the result joins memory, adaptation, and inference within one recurrent process. The system does not need a separate optimizer for every puzzle. It also avoids turning every intermediate state into a sentence.

Why Token-by-Token Reasoning Is Under Pressure

BDH-CQ targets the computational format of chain-of-thought reasoning, not merely its benchmark accuracy.

Large reasoning models commonly allocate more computation by generating more tokens. Those tokens can document partial deductions, test alternatives, and correct earlier mistakes. They also force internal work through a discrete vocabulary.

Every generated reasoning token requires another autoregressive decoding step. The model must project its hidden state into vocabulary probabilities, select a token, and consume that token again. Longer traces therefore increase computation and latency.

Verbal reasoning remains useful because people can inspect it. It can also support tool calls, collaboration, and explicit verification. However, language becomes a bandwidth restriction when the intermediate state does not need human interpretation.

BDH-CQ tests another allocation mechanism. Instead of extending a visible transcript, it adds recurrent transformations within a continuous state. One internal state can encode several partial possibilities without spelling each one out separately.

That difference resembles parallel versus serial work. A verbal trace normally selects one token at each position. A high-dimensional state can preserve multiple active features and transform them together.

The idea is not unique to BDH-CQ. Meta’s Coconut research feeds continuous hidden states back into a Transformer, allowing reasoning without decoding every intermediate step. Recurrent-depth models similarly reuse computation blocks across multiple internal iterations.

BDH-CQ adds a distinct element: demonstrations continuously modify recurrent task memory. The query then uses that memory during latent iteration. The system therefore connects in-context adaptation with nonverbal reasoning inside one architecture.

This connection pressures two established routes. The first is conventional chain-of-thought, where examples specify the task and tokens supply the workspace. The second is task-specific optimization, where a compact solver adapts separately to each puzzle.

Pathway’s system claims to preserve flexibility from the first route without its mandatory narration. It also claims to preserve compact computation from the second route without puzzle-specific parameter updates.

The contrast becomes sharper on ARC. Hierarchical Reasoning Model and Tiny Recursive Model systems use iterative latent computation, but their ARC procedures include transductive adaptation. Evaluation demonstrations participate in optimization associated with each puzzle.

BDH-CQ’s authors argue that a previously unseen task can enter through context instead. They compare this arrangement with a person learning a rule from examples, then applying it without rewriting their entire reasoning machinery.

That analogy should remain limited. Recurrent state updates are mathematical operations, not evidence of human-like understanding. The paper calls the design brain-inspired but explicitly avoids claiming biological imitation.

The underlying Dragon Hatchling architecture uses sparse positive activations, low-rank communication, and recurrent associative state. Its public architecture repository describes locally interacting artificial neurons and Hebbian-style working memory.

Hebbian memory strengthens associations between co-active features. In BDH-CQ, the practical point is that context can modify a persistent internal state. The state need not grow as an explicit list of every previous key and value.

That property could eventually matter beyond visual puzzles. Long reasoning traces consume memory, decoder time, and communication bandwidth. A recurrent state offers another way to preserve information while controlling those costs.

Yet removing language also removes a convenient inspection surface. A hidden trajectory cannot be trusted simply because it is efficient. Developers need other methods to test whether the internal process is stable, biased, or vulnerable to manipulation.

The pressure on token-based reasoning is therefore conditional. BDH-CQ shows that visible text is not the only possible computational workspace. It does not show that hidden computation can replace language across every task.

Horizon MachineLearning Discussion Points to a Deeper Mechanism

The important mechanism is not silent reasoning alone; it is the separation between contextual memory and the active reasoning workspace.

A widely shared MachineLearning discussion helped surface the paper, but the underlying technical claims come from the research itself. BDH-CQ maintains two internal structures with different jobs.

The recurrent contextual state changes while the model ingests demonstrations. It stores associations relevant to the newly presented task. The latent reasoning workspace begins after those examples have entered memory.

During query solving, the workspace changes across multiple reasoning steps. The contextual state remains available during those transformations. A decoder converts the final workspace into one or two ranked grid candidates.

This division prevents one vector from carrying every responsibility. The memory answers what the demonstrations established. The workspace handles what computation remains necessary for the query.

The authors describe the memory update at a high level. A new state depends on the previous state and the next demonstration. The trained parameters governing that update do not change.

They describe the workspace similarly. An encoder initializes it from the query and accumulated context. A recurrent function repeatedly updates it before a decoder creates the answer.

That description explains the concept, but not the full implementation. The paper says exact dimensions, update rules, and internal system details remain proprietary. It also withholds the complete training recipe.

The evaluated product includes more than the central neural architecture. It uses input transformations, candidate construction, ranking, and an inference pipeline. Those surrounding components can influence both accuracy and compute.

Even with that limitation, controlled experiments offer clues about what the recurrent memory actually retains. One test created a new color mapping inside each task. Demonstrations defined how several colors should change.

BDH-CQ solved all 96 held-out outputs at rank one as simultaneous bindings increased from two to eight. The result suggests that contextual memory can store and apply dense task-specific mappings.

Other tests varied the scale of a demonstrated operation. Boundary propagation remained correct across all 48 held-out outputs over distances from two through eight. Motif copying also remained correct across all 48 outputs as targets increased.

These are favorable examples because the underlying operation stays local and regular. The model can expand a learned action without reconstructing a complicated output order.

Ordering produced a different curve. Performance stayed near saturation through five objects, then fell at longer sequences. At eight objects, BDH-CQ solved only one of 24 held-out outputs under pass@2.

The error pattern was broad rather than local. Only three outputs at that length even had the correct dimensions. This suggests an execution bottleneck in building the ordered result, not only uncertainty about the rule.

Nested containment failed differently. Performance remained strong through depth four, then declined at depth five. Those outputs usually retained the correct dimensions and exceeded 99.9% mean cell accuracy for their best candidates.

A single relational mistake often caused failure. This pattern suggests that the model nearly completed the transformation but lost track of one containment decision. Exact-match scoring appropriately counted those outputs as wrong.

Adding a demonstration at the target complexity changed the results. Depth-five nesting rose from 19 of 24 correct outputs to 24 of 24. Length-eight ordering improved from zero to 13 of 24.

This intervention strengthens the in-context learning claim. Byte-identical test inputs performed differently when demonstrations covered the required complexity. The examples changed the model’s effective behavior without parameter updates.

The same evidence also defines a boundary. BDH-CQ does not reliably extrapolate every rule beyond demonstrated complexity. Its memory can bind a transformation while its workspace still struggles to execute that transformation at greater scale.

Reasoning effort produced another predictable tradeoff. The high setting reached 29.5% pass@2, while medium reached 27%. Low effort reached 21%, with the authors reporting progressively lower computation.

This relationship resembles test-time compute scaling in other reasoning systems. More recurrent steps improve the probability of an exact answer. The difference is that additional effort occurs inside continuous states rather than an expanding transcript.

That mechanism makes BDH-CQ more than a compressed chain-of-thought experiment. It creates a task memory from demonstrations, separates that memory from query computation, and scales effort through recurrent depth.

The Benchmark Wins Expose Clear Failure Modes

BDH-CQ’s strongest evidence also shows that contextual learning does not guarantee consistent rule execution.

The paper evaluates BDH-CQ on ConceptARC, which organizes visual tasks into 16 concept families. The model reached 59.38% strict task pass@2 with semantic identifiers. It reached 60% after identifiers became opaque and concept groups were mixed.

Those similar scores reduce one obvious concern. The model did not appear to depend on meaningful task names or concept-grouped batches. Both versions produced 374 correct results across 480 individual test pairs.

However, aggregate stability did not mean identical behavior. Only 276 of 480 complete ranked candidate lists matched across the two runs. Request context or internal search behavior changed many outputs without moving the overall score.

More importantly, test-pair accuracy reached 77.92%, while strict task accuracy remained 59.38%. The 18.54-point gap shows that a correct answer on one input often failed to transfer across every input.

Fifty-two of 160 ConceptARC tasks had one or two correct test inputs without solving all three. Under a strict rule-induction view, those partial successes do not demonstrate consistent application of the inferred rule.

Performance also varied sharply by concept. The system solved nine of ten tasks involving filled versus unfilled distinctions. It also solved nine of ten tasks in the TopBottom2D family.

Copy and Order each reached only two of ten. Each family contains just ten tasks, so those counts cannot establish a stable ranking. They still reveal where the current system deserves closer pressure testing.

Composition created another major weakness. BDH-CQ solved rotation and relocation separately, then solved their combination across all 72 held-out outputs. Reflection combined with relocation succeeded on 47 of 72.

Color swapping performed much worse. It succeeded on only 26 of 72 outputs as an isolated operation across the pooled motif families. When combined with relocation, it failed on all 72.

The layout mattered heavily. The model learned color swapping more readily when motifs shared a fixed arrangement. Performance nearly disappeared across two shuffled families, even before composition entered the problem.

This result cautions against describing the system as learning abstract operators without qualification. It sometimes binds a reusable transformation. In other cases, success depends on representational regularities within the demonstrations.

ARC itself also limits what can be concluded. Its grids exclude factual recall, natural-language ambiguity, social context, tool use, and long-form planning. Strong visual induction does not establish broad reasoning competence.

The paper’s title emphasizes in-context learning with recurrent latent reasoning. It does not claim that the 150-million-parameter system can replace a general language model. The outlook lists language and mathematical reasoning as future extensions.

Reproducibility remains the larger concern. The complete evaluated system is unavailable, while exact memory updates and latent workspace details remain undisclosed. Researchers cannot yet reproduce the headline result from the public Dragon Hatchling repository.

The reported audit provides some assurance that the service returned the stated score. However, its evaluators are paper coauthors, and they lacked model weights. A stronger test would involve unaffiliated evaluators running a frozen artifact.

Training contamination also requires careful language. The researchers excluded public evaluation demonstrations and task identifiers, according to the paper. Their private data and full checkpoint-selection process remain unavailable for inspection.

The opaque-identifier experiment rules out a narrow request-side shortcut. It does not prove that no related patterns entered training. The paper itself acknowledges that ConceptARC is not a fresh benchmark.

Cost comparisons require caution as well. BDH-CQ’s estimate comes from measured hardware time under an assumed accelerator rate. Other leaderboard entries can reflect provider pricing or separate hardware estimates.

That means plotted values do not always measure the same economic quantity. The model’s short measured runtime remains notable, but the exact size of its advantage depends on comparison methodology.

Silent reasoning adds a governance tradeoff. Users cannot inspect the intermediate path because no linguistic path exists. The final answer can be checked on ARC, where every cell has an exact target.

Many real applications lack that verification. A medical summary, business forecast, or research synthesis rarely has an immediate oracle. Hidden reasoning would need confidence calibration, consistency tests, and external evidence checks.

This problem does not invalidate latent computation. Human-readable chains can also contain plausible explanations generated after the underlying decision. Visible reasoning is not guaranteed to be faithful.

The practical requirement is better evaluation. A latent system needs behavioral probes that test memory, extrapolation, composition, stability, and failure recovery. BDH-CQ’s controlled experiments are useful precisely because they expose these boundaries.

Recurrent Latent Reasoning Changes the Cost Debate

BDH-CQ shifts attention from model size alone toward the structure of inference and adaptation.

A 150-million-parameter model is small beside frontier language systems. Yet parameter count does not fully explain its reported efficiency. The architecture also avoids repeated language decoding and puzzle-specific optimization.

That creates three different computation budgets. Training builds the general ability to process ARC-style tasks. Demonstrations update temporary recurrent memory. Latent iterations allocate query-specific reasoning effort.

Keeping those budgets separate can give developers finer control. A system might process more demonstrations without changing its parameters. It might also run more internal iterations only when a query needs them.

Current language models often combine these functions inside one token stream. Prompts carry demonstrations, attention preserves context, and generated tokens expand reasoning. The resulting interface is simple but computationally serial.

BDH-CQ offers a more structured division. Memory records evidence from context, while recurrent depth performs the active search. The decoder appears only when the system needs an external answer.

This design resembles fast-weight memory, where activations create temporary associations within a fixed network. It also connects with linear attention, which can summarize past information inside a recurrent state.

The deeper competitive question concerns adaptation. Task-specific recursive solvers can perform well after optimizing on each puzzle’s examples. General language models adapt through context but often spend many tokens reasoning.

BDH-CQ aims for contextual adaptation without verbose decoding or backward-pass optimization. That combination is the main opponent to established methods, even if its absolute accuracy remains below stronger systems.

Developers should not read the result as evidence that tokens are obsolete. Language remains an effective interface between models, users, tools, and auditors. It also lets a system communicate uncertainty and request missing information.

A future architecture may combine both modes. Latent recurrence could handle high-bandwidth internal computation. Selective decoding could expose checkpoints when verification, collaboration, or tool use requires language.

The BDH-CQ paper points toward this hybrid. It notes that latent states and language can serve different functions. Intermediate text need not disappear everywhere simply because it is unnecessary at every step.

The approach may also change memory design. Conventional long-context models retain growing token histories or compressed caches. Recurrent memory instead updates a fixed-size state as new evidence arrives.

Fixed-size memory introduces its own tradeoff. Compression can discard information, merge associations, or overwrite earlier evidence. The paper’s color-binding test shows capacity within one controlled range, not unlimited retention.

Long-horizon evaluation should therefore test interference. Researchers need to know what happens when demonstrations conflict, arrive in different orders, or include irrelevant examples. They should also measure how quickly old associations decay.

Security testing matters for the same reason. If inference inputs modify recurrent state, malicious demonstrations might steer later answers. A latent memory attack may be harder to diagnose because no textual scratchpad records the transition.

Enterprise uses would require reset boundaries and memory isolation. A temporary task state should not leak between users or unrelated jobs. Systems also need ways to identify corrupted context before it affects downstream queries.

For knowledge work, the underlying idea remains attractive. People rarely narrate every internal step while reading notes, comparing evidence, and forming a conclusion. They build a working model, revise it, then communicate selected reasoning.

Software that supports knowledge blending already treats context gathering and answer generation as connected processes. BDH-CQ explores a lower-level architectural version of that connection.

The result also raises a product question. Should users pay computation costs for readable chains they never inspect? For exact and verifiable tasks, silent latent iteration may deliver a better balance.

For consequential decisions, the answer changes. Users may need cited evidence, explicit assumptions, or a reviewable derivation. A latent core would need an external reporting layer that communicates those elements faithfully.

The long-term contest is therefore not hidden thoughts versus visible thoughts. It is rigid token serialization versus systems that choose the right representation for each computational and communication step.

BDH-CQ supplies early evidence for that broader design. Its compact ARC result makes the question harder to dismiss. Its limitations prevent any declaration that the contest is settled.

Three Signals Will Determine Whether BDH-CQ Matters

The next phase must test reproducibility, broader domains, and scaling before BDH-CQ can support larger claims.

The first signal is an independently runnable artifact. Pathway has released baseline Dragon Hatchling code, but not the complete BDH-CQ system used for the reported evaluation.

A frozen container, model checkpoint, or hosted evaluation protocol would let unaffiliated researchers reproduce the 400-task result. It would also clarify how candidate construction and ranking contribute to the final score.

Independent evaluation should compare measured hardware time under consistent conditions. It should separate neural inference from preprocessing, augmentation, candidate ranking, and service overhead. Comparable accounting would strengthen or weaken the efficiency claim.

Researchers should also verify exclusion boundaries around evaluation data. Dataset manifests, hashes, and checkpoint-selection procedures would provide more confidence than high-level assurances. Private examples need not be published if their provenance can be audited.

If reproducibility holds, BDH-CQ becomes a credible new point on the ARC efficiency curve. If it fails, the story narrows to an interesting architectural proposal with an unconfirmed system result.

The second signal is performance outside visual grids. The paper identifies language, mathematics, constraint satisfaction, and harder ARC tasks as future directions. Those evaluations will test whether the architecture transfers beyond its current training distribution.

Language reasoning creates different requirements. A model must handle ambiguity, factual knowledge, long outputs, and instructions that change over time. It must also communicate evidence, not merely emit an exact grid.

Mathematics would provide stronger verification while expanding problem structure. Formal proofs, program synthesis, and theorem checking can test latent reasoning without relying entirely on subjective grading.

ARC-AGI-2 would offer another useful challenge. It was designed to resist methods that exploit familiar task patterns and to demand more compositional generalization. A strong result there would support broader claims about adaptation.

Failure on these domains would not erase the current finding. It would show that BDH-CQ is specialized for ARC-style visual transformations. That outcome would still inform efficient solver design.

The third signal is transparent scaling evidence. The paper claims that BDH-CQ can scale and mentions early experiments between one billion and 600 billion parameters. It does not provide enough evidence to evaluate those claims.

The authors also state that the architecture could support much larger configurations. Readers should treat that as an engineering projection until training curves, compute budgets, and downstream results become available.

Scaling may improve difficult ordering and composition. It may also expose memory instability, optimization problems, or diminishing returns. The current 150-million-parameter model cannot answer which outcome dominates.

A particularly useful release would plot model size against task accuracy, inference iterations, and recurrent-memory capacity. That would reveal whether larger models improve abstraction or merely memorize more ARC-style regularities.

Researchers should also test whether reasoning effort scales smoothly. The current low, medium, and high settings show increasing accuracy, but only across three points. More granular curves could identify saturation and instability.

These three signals should arrive in that order: reproducibility, domain expansion, then scaling. A larger proprietary claim offers less scientific value than a smaller result that independent teams can examine.

Developers should watch for one practical detail throughout. Does recurrent memory improve adaptation without creating new context-isolation and security failures? Efficiency alone cannot compensate for unreliable state boundaries.

For now, BDH-CQ establishes a focused result. A compact recurrent system learned visual transformations from demonstrations, reasoned through continuous states, and produced a competitive ARC score with low measured compute.

It also failed predictably on long ordering, inconsistent rule execution, and certain composed operations. Those failures keep the work grounded. They show where richer demonstrations help and where internal execution still breaks.

The right response is neither dismissal nor celebration. Follow the ARC leaderboard, examine any released evaluation artifact, and compare results under consistent accounting.

Then ask a sharper question of every reasoning model: which steps genuinely need language, and which steps only use tokens because today’s architectures provide no better workspace?