Vroom-Vroom at SHROOM-Visions: A Multi-Judge Committee for Detecting Hallucinated Spans in Vision-Language Outputs

AI summary

Overview

Research area: Vision-language model (VLM) hallucination detection, multilingual span-level annotation, model ensembling, and activation probing.

Technical level: Intermediate — the paper assumes familiarity with VLMs, LoRA fine-tuning, in-context learning, and interpretability probes, but its core idea (a committee of models voting on spans) is conceptually straightforward.

Scope: Describes a shared-task submission that detects and classifies hallucinated character spans in VLM outputs across English, French, Italian, and Chinese by combining fine-tuned VLM judges with activation probes.

What This Paper Is About

Vision-language models frequently generate confident text that the input image does not actually support — invented objects, miscounted items, misread text, or wrong relationships. Flagging where in the output the hallucination occurs, down to individual characters, is much harder than flagging that one exists at all, partly because human annotators themselves disagree about span boundaries.

This paper builds a detector for the SHROOM-Visions shared task that marks hallucinated text spans and assigns each a confidence value, using two parallel strategies: a "committee" of fine-tuned VLMs that vote character by character, and lightweight probes trained on the models' internal activations.

Key Contributions

1. A multi-judge committee. Five VLMs from four backbone families (Gemma-4, Mistral-Small, Qwen3-VL, Qwen3.6) act as independent span annotators; their predictions are projected to character level and aggregated through majority voting with averaged confidences.

2. A progressive probe design. A family of activation probes ("Multi-layer" and "Multi-stage") over Qwen3-VL-4B and 8B hidden states, combining a learned softmax-weighted mixture of all layers, residual-stream differences, and auxiliary heads for span boundaries, sample-level, and token-level hallucination decisions.

3. Empirical evidence linking model disagreement to annotator disagreement. The number of judges flagging a character rises monotonically with the number of human annotators who marked it, supporting the claim that ensemble diversity is a useful proxy for genuine ambiguity.

4. Competitive shared-task results. The committee ranked first in three of four languages and placed on the podium in every language and metric, with the two-vote threshold configuration performing best on correlation and IoU.

Main Findings

• Committee beats every individual judge. The aggregated system achieves the best Cor and IoU across all four languages; the single exception is English Cor_lbl, where a three-shot fine-tuned Qwen3-VL judge scores marginally higher.

• Two votes is the sweet spot. A threshold of at least two judges improves Cor and IoU by recovering recall lost by individual judges while suppressing single-judge false positives. Requiring three votes slightly helps Cor_lbl on the seen split but degrades every language and metric on the unseen challenge set.

• Probes are architecture-insensitive. Multi-layer and multi-stage probe scores are closely correlated with one another, suggesting performance depends mostly on the underlying model's activations rather than the probe's design.

• Probes match individual judges on IoU but trail the committee on correlation metrics. They are useful when inference cost is the binding constraint, since they add only a lightweight head over a single forward pass rather than five full VLM passes.

• Adding probes to the ensemble does not help. Probe errors correlate strongly with the judges' errors, so they add little complementary signal.

• Judge agreement tracks annotator agreement. Mean annotators per character rises from 0.11 (flagged by zero judges) to 2.07 (flagged by all five), with Spearman correlations of 0.39 at character level and 0.51 at record level.

• Annotation is dominated by two error types. Invention and mischaracterization account for 80–88 percent of all error characters, and 80–86 percent of hallucination characters are marked by only one of three annotators.

• The committee reduces probability error. Because a single judge emits only three discrete confidence values, its predictions sit on a coarse grid; averaging over the committee yields a finer range closer to empirical annotator agreement, lowering MAE_p.

• "Other" is the hardest label. Every system performs poorly on it, consistent with it being the rarest and least consistently annotated category.

• English is the weakest language. Longer responses in English lower annotator agreement, capping achievable scores. On the challenge set, the committee's average was 0.5730 Cor, 0.4943 IoU, and 0.4635 Cor_lbl using the two-vote threshold.

Methodology in Plain English

Data preparation. The team used the SHEEP dataset from the SHROOM-Visions task, which contains real VLM responses labeled by three human annotators per item across four languages. Exploratory analysis revealed skewed error distributions and heavy inter-annotator disagreement, which shaped the design. The train/development/test split groups by image ID and stratifies by error type at the span level, not the character level.

Turning annotation into a language the models understand. Gold annotations were rewritten as inline markup — <hall label="..." prob="...">text</hall> — so that a VLM could copy the input format and produce annotated output. Overlapping spans were removed, agreement scores were collapsed into three probability classes (about 0.33, 0.67, and 1.0), and extremely long spans were excluded. Predicted markup was then converted back into character offsets.

Building the judges. Five models were used. Some were fine-tuned with LoRA on distinct data subsets to decorrelate their errors; one Gemma-4 judge worked purely in a few-shot setting, receiving six in-context examples retrieved by weighted cosine similarity over image embeddings (CLIP), response embeddings, and prompt embeddings (BGE-M3), weighted 50:30:20.

Committee aggregation. Each judge's spans were projected onto characters, taking the highest-confidence covering span's label and probability. Characters were then scored by counting how many judges flagged them and averaging their confidences. Spans were rebuilt by merging adjacent runs sharing a label and probability, then filtered by a vote threshold.

Activation probing. Instead of asking a model to produce annotations, probes read the model's internal hidden states and predict hallucination per token. The probes compress the image-token embeddings into mean-pooled tokens (256 was the point of diminishing returns), cross-attend over them, mix information across layers with a learned softmax weighting, and add residual-stream difference signals under the intuition that hallucinations occur when language priors override visual grounding. Auxiliary heads for span boundaries and sample- and token-level hallucination decisions gate the final type prediction. Probes were trained with binary cross-entropy using weighted loss terms.

Evaluation. Three official metrics were used: Cor (correlation between predicted and empirical per-character hallucination probabilities), Cor_lbl (its per-label variant), and character-level IoU. The authors also report MAE_p, the mean absolute difference between predicted and gold character probabilities.

Why This Matters

Impact on research. The paper provides a strong reference point for span-level hallucination detection, arguing that model ensembling is a practical way to absorb the inherent ambiguity that annotator disagreement represents. It also raises a pointed question about interpretability: probes detect hallucinations, but it remains unclear whether they capture genuine mechanistic structure or merely exploit surface linguistic features. The finding that performance is governed by the source model's activations rather than probe architecture sharpens that question.

Real-world applications:

• Verifying automatically generated alt-text for accessibility, where an unsupported claim about an image can mislead screen-reader users.
• Auditing multimodal QA or captioning systems in medical, industrial, or scientific imaging pipelines before output reaches a human decision-maker.
• Validating product descriptions, listings, and catalog metadata generated from photographs in e-commerce.
• Flagging misread or invented text in document-processing and OCR-heavy workflows where characters and numbers matter.

Industry relevance. Any organization deploying VLMs in customer-facing or safety-relevant settings needs span-level rather than document-level error signals, because a user must know which specific claim is unreliable. The cost trade-off the paper documents — five full forward passes for the committee versus one pass plus a lightweight head for probes — gives engineering teams a concrete lever for matching detection quality to inference budget.

Future Directions

• Distinguishing mechanism from surface features in probes. The authors explicitly state it is unclear whether probes reveal how hallucination arises internally or simply learn shallow textual cues. Resolving this requires targeted causal interventions or controlled probes.

• Continuous confidence estimation. Gold agreement is currently collapsed into three discrete classes, which limits how precisely uncertainty can be represented. Learning continuous agreement targets could improve MAE_p and correlation scores.

• Generalizing the vote threshold. The two-vote cutoff was tuned on the labeled test split and applied unchanged to the challenge set. Whether it transfers to other model families, languages, or annotation schemes is untested.

• Handling long fluent hallucinations. Errors that span many words but attract only one model's attention fall below the two-vote threshold and go undetected — a particular problem in the longer English responses. Image preprocessing and aspect-ratio-aware transformation are also unexplored and may affect visual grounding.

Target Audience

Researchers and graduate students working on VLM trustworthiness, hallucination detection, or model ensembling; participants in multilingual annotation and shared-task benchmarks; interpretability researchers studying whether internal-state probes yield mechanistic insight; and ML engineers building production safeguards for multimodal systems who need to weigh detection quality against inference cost.

Authors’ abstract

This paper describes our submission to the SHROOM-Visions shared task on detecting and classifying hallucinated character spans in vision-language model outputs across four languages. We employ several fine-tuned vision-language models as independent annotators and combine their span predictions through character-level majority voting, and additionally explore activation probes. The approach ranks first in three of four languages and places on the podium in every language and metric. Our analysis indicates that disagreement among diverse models tracks disagreement among human annotators.