Hallucination Detector Claim Finds a Floor, Not a Universal Signal

A researcher on r/MachineLearning says a pre-registered detector cleared its target in 18 of 20 model deployments before the first token appeared. Yet the reported result does not establish a universal hallucination detector. It points instead to a shared performance floor built from model-internal geometry.

The distinction matters. A detector that transfers across many models can become a useful routing signal, even when it cannot identify every false answer. However, the evidence currently comes from a social post describing unpublished or not yet independently reviewed experiments.

The post also reports a notable reversal. Adding the model’s own confidence did not improve deployment coverage beyond the geometry-only detector. The same two deployments failed, suggesting that confidence and geometry captured overlapping information in this test.

That finding pressures two familiar approaches. One treats token probabilities as a convenient factuality score. The other expects a single internal feature to generalize across architectures, datasets, and hallucination types. The reported experiment supports neither expectation in its strongest form.

The Pre-Registered Test Cleared 18 of 20 Deployments

The central result is a qualified success: one geometry-based detector reportedly met its pre-registered coverage bar without becoming universal.

According to the original MachineLearning post, the researcher evaluated two tasks across ten language models. That created 20 model-task deployments in the first run.

The project examined 29 model-internal signals from four broad families. Those families covered attention patterns, residual-stream motion, readout geometry, and confidence. A selector then chose one signal for each model under rules established before the evaluation data existed.

A forward pass is one complete computation through the model for a given input. The proposed detector inspects that computation before generated text becomes available. This separates it from systems that verify completed answers against documents or compare several sampled responses.

The pre-generation restriction makes the reported 18-of-20 result interesting. If reproducible, the detector could assign risk before an application spends time producing a long answer. A product might then request retrieval, route the query to another model, or require human review.

The geometry-only detector needed to pass at least 17 deployments under the pre-registered rule. The post says it passed 18. That supports the narrow claim that a geometry-derived signal can establish broad, though incomplete, coverage across the tested conditions.

The number does not mean the detector caught 90 percent of hallucinations. Deployment coverage and detection accuracy are different measurements. Passing a threshold in 18 deployments says the method met a specified criterion in those settings.

The underlying threshold, class balance, dataset construction, calibration process, and per-deployment uncertainty all affect how readers should interpret the count. Without complete experimental materials, the result cannot be translated into a production error rate.

Pre-registration strengthens the design by limiting opportunities to revise a hypothesis after seeing results. It does not validate the labels, eliminate implementation errors, or guarantee that the selected test represents real applications.

The post says the procedure was pre-registered twice. That detail suggests the researcher attempted to separate exploratory decisions from confirmatory testing. Independent access to both registrations would clarify which features, thresholds, exclusions, and statistical tests were fixed.

The strongest responsible reading is therefore limited. A geometry-only approach reportedly crossed its planned bar across most tested deployments. The remaining failures matter because they define the boundary of the claim.

Why Confidence Failed to Expand Coverage

The sharpest reversal is not that confidence performed badly, but that it reportedly added no coverage beyond geometry.

Model confidence often refers to probabilities assigned to possible next tokens. A high value means the model strongly favors one continuation over alternatives. It does not establish that the favored continuation matches external reality.

The experiment reportedly combined geometry with the model’s confidence. That combined detector also passed 18 of 20 deployments. It missed the same two deployments and rescued none.

The pre-registered bar for the stronger claim was at least 19 deployments. The combined method did not reach it. Under the reported decision rule, the proposition that confidence broadened coverage was falsified.

This result does not show that confidence contains no useful information. It suggests that its useful variation may have overlapped with the chosen geometric signal under these tasks and models. A different dataset or calibration method might reveal complementary information.

Geometry here refers to measurable spatial relationships among internal representations. Those representations are vectors that encode information as the model processes a prompt. Their norms, angles, directions, and movement across layers can correlate with answer correctness.

Confidence emerges from the same underlying network. It is therefore plausible that a geometric measurement near the model’s output already captures much of the information expressed by token probabilities. The post’s reported zero-rescue result is consistent with that explanation.

Recent research also warns that geometric metrics are not interchangeable. A 2026 study of geometric detection metrics found that different statistics responded to different properties, including correctness, relevance, coherence, completeness, and confidence.

That study also reported substantial sensitivity to domain shifts. A signal that separates correct and incorrect history answers might behave differently on mathematics. Its authors introduced normalization and reported gains under multi-domain evaluation, underscoring how calibration can shape results.

The broader lesson is that “hallucination” combines several failure mechanisms. A model can state a common misconception consistently, guess among several answers, ignore supplied evidence, or make a reasoning error. Those failures need not share one internal signature.

Confidence remains useful for abstention policies, especially when calibrated against a known deployment distribution. However, applications should not treat raw confidence as an independent factuality witness simply because it is easy to obtain.

That distinction affects product design. A confidence threshold may identify uncertainty, while a geometry score identifies a related instability. Combining two correlated signals can create the appearance of redundancy without adding a new source of evidence.

A genuinely complementary layer would inspect something different. It might compare claims with retrieved records, test logical consistency, verify citations, or examine whether a tool result supports the answer. Those checks address evidence rather than internal hesitation.

The MachineLearning Claim Challenges Universal Detection

A common floor is operationally valuable, but a universal detector requires stronger evidence across tasks, models, and error definitions.

The reported experiment began with an ambitious question: can one family of internal signals work broadly enough to detect hallucination before generation? Its answer appears to be yes for a floor and no for universality.

That is not a semantic distinction. A universal detector should transfer without relying on favorable task structure, architecture-specific calibration, or a narrow definition of error. Two failures among 20 deployments are enough to reject that absolute claim.

The result still has practical value. Production systems rarely need one score to settle every factual question. They need a gate that reliably identifies enough risky cases to justify a more expensive verification path.

Consider an enterprise assistant answering questions about internal policies. A pre-generation score could route a risky prompt toward retrieval before drafting begins. The final answer would still need support from the retrieved policy documents.

A coding assistant presents a different setting. The relevant failure may involve an invented library method, an incompatible version, or faulty reasoning across several files. A detector calibrated on short factual answers may not transfer to that environment.

Long-form research creates another boundary. One response can contain dozens of claims, with some supported and others false. A single score taken before the first token cannot identify which later sentence will fail.

This limitation explains why deployment coverage should not be confused with claim-level verification. The detector may estimate whether a prompt-response trajectory is risky. It cannot independently establish the truth of every proposition that follows.

Related research supports the possibility of an early signal. The 2026 paper Before the First Token evaluated seven autoregressive transformers using 552 labeled examples from three factual datasets.

Its authors reported chance-level probe performance for models below 400 million parameters. Above roughly one billion parameters, they observed stronger pre-generation signals in some models. The pattern also depended on instruction tuning.

The paper found a significant position-zero effect for Pythia-1.4B and Qwen2.5-7B. Yet Pythia-6.9B showed a flat temporal profile, despite its larger scale. That comparison argues against a simple rule based on parameter count.

The researchers also reported that activation steering along the detected directions did not correct hallucinations. In other words, the signal was correlational rather than causal. Detecting a risky state did not mean that changing that state produced a truthful answer.

That boundary should guide interpretation of the Reddit claim. An internal pattern can predict an outcome without representing a dedicated truth circuit. It may encode task difficulty, prompt familiarity, answerability, or another variable correlated with correctness.

A universal detector would need to separate those alternatives. It should preserve performance under new domains, prompt formats, languages, model families, and forms of post-training. It should also survive intentional attempts to break its assumptions.

The post’s invitation to reproduce or challenge the result is therefore central. The next stage is not a larger headline. It is an adversarial test suite designed around the two misses and around settings the original tasks did not cover.

Existing Detectors Solve Different Parts of the Problem

The field is converging on layered detection because internal geometry, semantic uncertainty, and external verification answer different questions.

One influential approach measures semantic entropy, meaning uncertainty across the meanings of several generated answers. If repeated answers express different claims, the model is more likely to be confabulating.

A peer-reviewed semantic entropy study tested LLaMA, Falcon, and Mistral families from 7B to 70B parameters. It reported stable AUROC values between 0.78 and 0.81 across those evaluated families and scales.

AUROC measures how well a score ranks positive examples above negative ones across thresholds. It does not specify the false-positive rate a particular application will experience. Operators still need a threshold aligned with their costs.

Semantic entropy differs sharply from a pre-generation detector. The Nature method used ten generations for its primary sentence-level experiments. It grouped answers by meaning before measuring their variation.

That procedure can detect arbitrary wrong answers that change across samples. It can miss a model that repeats the same false claim with confidence. The authors explicitly limited their method to confabulations rather than every behavior labeled hallucination.

The cost is also different. Sampling several answers and comparing their meanings requires more computation than reading one internal state. A single-pass detector offers an attractive early filter, especially when most queries are routine.

Black-box systems create another constraint. Commercial application developers often cannot inspect attention maps, residual streams, or hidden vectors. They receive text and perhaps token probabilities through an API.

Internal geometry therefore favors open-weight models, self-hosted deployments, or providers willing to expose suitable telemetry. Semantic consistency methods can work with black-box text outputs, though they demand repeated generation.

Retrieval-based verification takes a third route. It checks whether an answer is grounded in supplied documents. This can catch confident errors when the correct evidence exists, but retrieval introduces its own failure modes.

A retriever might return an outdated policy, omit the relevant passage, or rank a misleading document first. The language model might then cite a source that does not support its sentence. Grounding systems need claim-level checks, not merely document presence.

Human review remains necessary in high-stakes cases. A risk score can prioritize attention, but reviewers need access to original evidence and an audit trail. A searchable knowledge base can help teams preserve that source context.

These methods should be viewed as layers rather than direct substitutes. An internal detector asks whether the model’s computation resembles risky cases. Semantic entropy asks whether sampled meanings remain stable. Retrieval asks whether available evidence supports the claim.

A production pipeline can combine all three without pretending they are independent. The early signal routes queries, semantic checks examine uncertain outputs, and evidence checks validate material claims. Human reviewers handle cases whose consequences justify the cost.

The reported redundancy between confidence and geometry strengthens this layered view. Adding more measurements from the same network does not necessarily add more knowledge. Independent evidence must enter somewhere in the system.

The Verification Gap Is Still Large

The reported pre-registration improves credibility, but the public claim remains incomplete until others can inspect and reproduce the experiment.

The original post supplies a result summary, not a peer-reviewed record. Readers need the registrations, code, datasets, model checkpoints, prompts, labels, and complete deployment-level results to assess the finding.

The phrase “honest selector” also needs a precise definition. Feature selection can leak information when researchers use evaluation data to choose metrics, layers, signs, thresholds, or transformations. A clean split must cover every adaptive decision.

Selecting one signal per model raises a second question. A family of features can transfer broadly while the chosen feature differs across models. That is less universal than one frozen detector operating unchanged everywhere.

Model-specific selection may still be operationally reasonable. Teams routinely calibrate safety thresholds for each deployment. However, the claim should distinguish a universal feature family from a universal detector with fixed parameters.

The two failed deployments deserve more attention than an aggregate count provides. They might share an architecture, task, scale, tuning method, or label distribution. A common cause would identify where the proposed floor breaks.

Uncertainty around the ground truth matters as well. Hallucination labels can depend on whether a task measures factual correctness, faithfulness to context, relevance, or logical consistency. A detector cannot be interpreted without knowing the target definition.

The word “before” can also conceal technical choices. A model has already processed the full prompt before selecting its first output token. Its hidden state can therefore encode prompt difficulty, familiarity, and likely answer content.

That is still pre-generation detection, but it is not prediction without model work. Applications must complete at least the prompt-processing pass. For long contexts, that stage can represent a meaningful portion of inference cost.

The experiment should also report class prevalence and confidence intervals. A pass count alone hides whether one deployment barely crossed its threshold while another exceeded it comfortably. Small test sets can produce unstable rankings.

Adaptive attacks present another challenge. Users can rephrase questions, inject irrelevant context, request a specific answer style, or force long reasoning. A detector calibrated on ordinary prompts might fail when prompt structure changes.

Model updates can shift internal geometry even when product branding stays constant. Quantization, fine-tuning, adapters, system prompts, and inference settings may alter the measured signal. Each change can require fresh validation.

There is also a distinction between research models and hosted frontier systems. Internal-state techniques cannot be assumed to transfer to systems whose architecture, routing, and post-training remain undisclosed.

Recent work on why models hallucinate adds another complication. A 2026 evaluation incentives study argued that next-token training and accuracy-focused evaluations reward guessing over abstention.

A detector can flag risky guessing, but it does not change the incentive that produced it. Deployment teams may need scoring rules that reward appropriate refusal, alongside retrieval, verification, and risk detection.

For those reasons, “18 of 20” should begin the investigation rather than end it. A useful replication would freeze the published procedure, test unseen model families, and disclose every failure without replacing the original threshold.

Three Tests Will Decide Whether the Floor Holds

The claim becomes important if the same floor survives independent replication, distribution shifts, and a production-style routing test.

The first signal to watch is a complete release. That should include both pre-registration records, code, feature definitions, model identifiers, task data, labels, thresholds, and deployment-level scores.

A release would strengthen the claim by making leakage and selection decisions auditable. Missing registrations or incomplete per-model results would weaken it, especially if only successful configurations were available.

The second signal is independent replication on unseen models and tasks. The most informative tests would include open-weight architectures excluded from development, multilingual prompts, long-form answers, tool use, and document-grounded generation.

Replication should preserve the original procedure before researchers tune anything. If performance falls only after domain shift, the detector may still serve as a calibrated local gate. It would not support a universal floor.

The third signal is a routing trial tied to operational outcomes. Researchers should measure whether the early score reduces unsupported claims when it triggers retrieval, abstention, resampling, or human review.

That trial needs more than AUROC. It should report false alarms, missed errors, latency, added computation, review volume, and the accuracy of accepted answers. Those measurements determine whether the detector saves resources or merely moves mistakes.

The most promising outcome is not a perfect truth meter. It is a cheap first gate that catches a stable subset of risky prompts before generation and hands them to a genuinely different verification mechanism.

The reported confidence result makes that design more urgent. If confidence and geometry remain redundant, teams should stop treating their combination as two independent votes. They should pair the internal score with source retrieval or claim verification.

Developers evaluating this work can ask three immediate questions. Does the detector preserve its threshold on our data? Do its misses cluster around a recognizable failure type? Does routing flagged prompts improve final accuracy after all costs are counted?

The broader machine-learning community should try to break the proposed floor with controlled tests, not isolated anecdotes. A failed replication would narrow the claim. A successful one would establish a useful shared signal without pretending it detects every hallucination.

That is the real value of the reported result. It replaces the search for one universal detector with a measurable hypothesis: model geometry may provide a broad early warning, while truth still requires independent evidence.