The Pulse
A New Probe Reads Knowledge LLMs Refuse to Reveal
A preprint by Hiskias Dingeto introduces the Probe of Internal Recognition, a method for distinguishing knowledge that a language model conceals from information it never possessed.
AI.info Team ·
A probe for knowledge behind a wrong response
Hiskias Dingeto submitted a paper to arXiv on September 18, 2026, describing a method designed to address a narrow but consequential question: when a language model gives the wrong answer, does it lack the knowledge or conceal it?
The paper introduces the Probe of Internal Recognition, or PIR. Instead of judging a model only by what it says, PIR examines the model's internal states while it considers a question and a set of candidate answers. The method attempts to identify which candidate the model recognizes as correct, even when the model's output does not reveal that recognition.
Dingeto's approach adapts the Concealed Information Test, a forensic method that presents a suspect with a true detail among plausible decoys and measures a stronger response to the item the suspect recognizes. PIR applies a similar contrast inside a language model, using its internal states to determine which answer it recognizes. The method does not require an honest reference model or a labeled corpus identifying the correct answers.
“PIR is reference-free, needing no honest reference model and no labeled truth corpus.”
Hiskias Dingeto, author of the arXiv preprint
Testing whether models hide known answers
The abstract reports tests across eight models from five model families: Gemma, Qwen, Llama, Mistral and Phi. Across those models, PIR recovers the answer recognized by the model with balanced accuracy ranging from 0.70 to 0.87. The reported unknown-item baseline ranges from 0.28 to 0.40, compared with a chance rate of 0.25.
The method remains readable across several forms of concealment described in the paper. These include prompted deception, trained sandbagging, and external password-locked and circuit-broken checkpoints. In those concealment tests, recognition ranges from 0.85 to 0.93, according to the abstract. The reported result is that when a model hides an answer it knows, its internal recognition signal remains high.
That distinction is central to the paper's proposed use of PIR. A model may fail to provide an answer because it does not possess the relevant knowledge, or it may withhold an answer despite recognizing it internally. PIR is intended to separate those cases by reading the model's internal response rather than relying only on the text it produces.
Unlearning and model audits
Dingeto also reports a different pattern when unlearning removes knowledge. In those tests, recognition drops to the level associated with a question the model never knew. The abstract presents this result as evidence that PIR can distinguish a model that will not answer from one that cannot.
The paper proposes applications in sandbagging audits and unlearning verification. In a sandbagging audit, the method could be used to test whether a model is concealing a capability during an evaluation. In an unlearning evaluation, it could provide an internal signal for whether information has been removed rather than merely hidden from the model's output.
The abstract also states that the signal is causal and adds information beyond black-box behavioral cues. PIR is not limited to multiple-choice questions: Dingeto reports that the method extends to free-form generation as well.
The work is an arXiv preprint by a single author. Its claims describe a research method for probing model representations, not a general guarantee that every concealed capability can be recovered. The abstract frames PIR as a way to investigate whether a model recognizes an answer internally when its outward response says otherwise.