IntroConformal: Conformal Factuality Guarantees for Large Vision-Language Models via Introspective Signals

AI summary

Overview

Research area: Computer vision and multimodal machine learning, specifically factuality control and uncertainty quantification for Large Vision-Language Models (LVLMs), combining conformal prediction with mechanistic interpretability of internal model signals.

Technical level: Advanced. The paper assumes familiarity with conformal risk control, hidden-state representations, and LVLM evaluation protocols, though the core intuition is accessible.

Scope: The paper proposes and empirically evaluates a training-free framework that gives finite-sample, distribution-free guarantees on the rate of non-factual claims in LVLM outputs using signals the model produces about itself, rather than external verifiers or token-level confidence.

What This Paper Is About

LVLMs frequently generate claims that are not grounded in the input image, and users have no reliable way to tell confident-but-wrong outputs from correct ones. Existing statistical factuality-control methods depend on either generation-time token probabilities or external verifiers such as CLIP, which are unreliable for confident hallucinations or add auxiliary dependencies. The paper asks whether factuality control can instead be driven by introspective signals the model already contains, and builds a conformal risk control framework around two such signals.

Key Contributions

1. IntroConformal, a training-free CRC framework for LVLM factuality that derives conformity scores from the model itself, without external verifiers or auxiliary supervision, and provides finite-sample, distribution-free guarantees on response-level non-factual risk.

2. Layer-wise semantic stability ($S_{\text{sem}}$), a conformity score measuring alignment between averaged hidden-state representations in 8 mid layers and the final 4 layers on claim tokens, where higher cosine similarity indicates stable semantic grounding.

3. Verification probability ($S_{\text{prob}}$), a stronger conformity score that prompts the same model with a binary Yes/No factuality question about a claim and image, then reads the Yes-token probability at the first answer position from a single forward pass, normalized against No.

4. Empirical validation across five LVLM architectures and three grounded generation tasks, showing that the proposed signals satisfy the CRC guarantee while reducing abstention and improving claim-level discrimination relative to CLIP-based verification, token-probability baselines, and decoding-based hallucination mitigation methods.

Main Findings

• $S_{\text{prob}}$ separates factual from non-factual claims far better than external or confidence-based signals. On MSCOCO with LLaVA-1.5, $S_{\text{prob}}$ raises the factual/non-factual score gap from +0.0196 (CLIP) and +0.0523 ($T_{\text{prob}}$) to +0.2014, with the highest AUROC of 0.819. On Phi-3.5-Vision the separation is +0.2397 with AUROC 0.763.

• The advantage generalizes across tasks and architectures. On fine-grained captioning, $S_{\text{prob}}$ reaches the highest AUROC across both evaluated models (0.765 and 0.770), with separations of +0.1849 and +0.2468. On document understanding it reaches AUROC values up to 0.728.

• Baseline signals can be anti-correlated with factuality. $T_{\text{prob}}$ on document understanding with LLaVA-1.5 yields a negative gap of -0.0182 (AUROC 0.493), and $S_{\text{sem}}$ on Llama-3.2-Vision MSCOCO reverses similarly (-0.0003, AUROC 0.488), assigning higher scores to non-factual claims.

• $S_{\text{sem}}$ carries only a weak but statistically significant signal. Its factual/non-factual score differences are often below +0.002, yet remain statistically significant in most settings, and it tends to be more conservative, producing higher abstention through stricter filtering.

• The CRC guarantee holds with substantially lower abstention for $S_{\text{prob}}$. On MSCOCO with LLaVA-1.5, $S_{\text{prob}}$ reduces abstention from 57% (CONFLVLM) and 64% ($S_{\text{sem}}$) to 25% while improving F1 from 0.504 to 0.581. On Llama-3.2-Vision it improves F1 from 0.269 to 0.302. It achieves the highest F1 in three of four fine-grained captioning and document understanding settings.

• Risk control remains valid across target risk levels and calibration sizes. In Figure 2(a), empirical risk stays below the desired target for all models across $\alpha \in [0.05, 0.40]$. Calibration improves with size: sweeping calibration from 50 to 400 examples, abstention decreases substantially between 100 and 200 samples before stabilizing, with empirical risk approaching the target from below.

• IntroConformal outperforms decoding- and verification-based hallucination mitigation. Claim filtering efficiency rises from 95.3% (CONFLVLM) to 97.4%, with response accuracy of 91% versus 90%. Woodpecker reaches 59.1% / 41%, CoVe 37.0% / 23%, VCD ($\beta = 0.1$) 35.5% / 20%, and ICD ($\beta = 0.1$, P) 41.1% / 26% on the same metrics.

• The guarantee degrades gracefully under symmetric annotation noise. On LLaVA-1.5 MSCOCO, test risk falls from 0.054 at 0% injected noise to below 0.001 at 15%, while abstention rises from 25.0% to 77.1%. Random label flips inflate apparent calibration risk, so the procedure selects a larger threshold rather than violating the bound. Results average over 20 noise draws.

• The formal operating point. With $n = 400$ calibration prompts, at most 50 claims per response, and $|\Lambda| \leq 20{,}000$ candidate thresholds, testing at $\alpha = \delta = 0.1$ yields concurrent risk control at $\alpha' \leq 0.170$, guaranteeing that at least 83% of retained claims are factual in expectation.

• Annotation reliability was verified. CONFLVLM's GPT-4o labels were validated against human raters with an ICC of 0.85. For the new benchmarks, one human annotator reviewed 372 claims (54.3% factual, 45.7% non-factual) across 50 randomly selected images, achieving 86.0% agreement with GPT-5.4 labels and Cohen's $\kappa$ of 0.71.

Methodology in Plain English

The researchers treat a model's answer as a set of atomic claims and aim to filter out any claim not supported by the image, while guaranteeing that the retained set has a bounded error rate.

The first signal comes from inside the network. For each claim token, they average the hidden-state vectors from the 8 layers just before the final block and from the final 4 layers, then compute cosine similarity between the two averages. If a claim's representation stays similar as it moves through the deeper layers, it is scored as more likely factual; if the representation drifts, the claim is suspect.

The second signal reuses the same model as its own judge. They ask it, in plain language, whether a statement is true given the image, requiring a Yes or No answer, and then read the probability assigned to the Yes token at the first answer position and normalize it by the combined Yes and No probabilities. This is a single forward pass with no sampling, unlike approaches that generate discrete verification answers.

To turn a score into a guarantee, they collect a calibration set of image-prompt pairs split into claims with factuality labels. For any threshold, keeping only claims scoring at or above it gives a response-level non-factual rate. They then use the Learn-Then-Test procedure with a Hoeffding upper confidence bound over candidate thresholds, plus a Bonferroni correction to make the bound valid simultaneously across all thresholds, and pick the smallest threshold that still satisfies the target risk. This makes filtering as aggressive as the guarantee allows.

Evaluation spans three tasks (general scene understanding on MSCOCO, fine-grained captioning built from CUB, Stanford Cars, and Stanford Dogs, and document understanding on SROIE invoices) and five LVLM architectures, with response-level risk and abstention as the metrics tied to the formal guarantee and claim-level TPR, precision, and F1 reported as diagnostics.

Why This Matters

Impact on research. The paper connects conformal prediction with mechanistic interpretability by showing that internal representations and a model's own self-judgment can serve as conformity scores with formal statistical guarantees. It challenges the assumption that reliable factuality control requires external verification, and it documents cases where accepted signals such as token confidence and hidden-state stability are actively misleading.

Real-world applications:

• Medical reporting, where an LVLM describing an image must not assert findings that are not present, and an abstention mechanism is preferable to a confident error.
• Document and invoice processing, where structured financial documents make ungrounded claims costly and where the paper reports its strongest discrimination results after scene understanding.
• Autonomous systems that interpret visual input and must decide when to defer rather than act on a non-grounded claim.
• Content pipelines and accessibility tools that generate image captions, where filtering ungrounded details improves downstream trust.

Industry relevance. Both signals are training-free, require no external model calls, and work with open-weight models or any API that exposes logits, which lowers the barrier to deployment. The framework lets practitioners set an explicit risk target and obtain a provable bound, which is more auditable than heuristic prompting or decoding tricks.

Future Directions

• Overcoming access constraints. $S_{\text{sem}}$ requires white-box access to hidden states, and $S_{\text{prob}}$ requires output logits, so neither applies to APIs that withhold them. Finding introspective signals usable under stricter access limits is an open problem.

• Addressing systematic annotation bias. The guarantee is defined relative to calibration labels, and the noise analysis covers only symmetric noise. The authors flag that systematic bias need not inflate apparent calibration risk and could therefore select a permissive threshold.

• Strengthening human validation. Reliability checking used only a single human annotator, so expanding to multiple annotators and broader label distributions would clarify how well the guarantee tracks human ground truth.

• Reducing inference cost. Both scores require an additional forward pass per claim, comparable to the CLIP scoring in CONFLVLM, leaving room for cheaper introspective signals.

• Extending beyond the tested tasks and architectures. The empirical study covers three task families and five LVLMs; whether the signal rankings and guarantee behavior persist across other modalities and larger models is not established.

Target Audience

Researchers and practitioners working on multimodal model reliability, hallucination detection, and uncertainty quantification, particularly those familiar with conformal prediction. It is also relevant to engineers deploying LVLMs in safety-critical or regulated settings who need calibrated, auditable controls on factual output, and to interpretability researchers interested in whether internal representations carry usable factuality signals.

Authors’ abstract

Large Vision-Language Models (LVLMs) have achieved strong multimodal performance, yet ensuring the factual correctness of generated content remains challenging. Existing methods that provide statistical guarantees on factuality typically rely on external verifiers or generation-time confidence signals, which introduce auxiliary dependencies or often fail for confident but incorrect outputs. We argue that reliable factuality control can instead be achieved through introspective signals derived from the model itself. We introduce IntroConformal, a training-free Conformal Risk Control (CRC) framework that provides finite-sample, distribution-free factuality guarantees. We first instantiate it with layer-wise semantic stability, a conformity score derived from hidden-state representations, and then propose verification probability, a stronger score capturing the model's self-administered judgment on claim factuality. Across multiple LVLM architectures, IntroConformal satisfies the conformal risk guarantee while substantially reducing abstention and achieving competitive or superior claim-level discrimination relative to external verifier-based baselines.