TRAPSBench: Vision-Language Models Encode but Fail to Express Epistemic Restraint

AI summary

Overview

• Research area: Vision-Language Models (VLMs), video physical reasoning, epistemic uncertainty, selective abstention, and mechanistic interpretability.
• Technical level: Advanced. The paper combines a procedurally generated benchmark with linear probing and causal activation steering inside frozen model weights.
• One-sentence scope: The paper introduces a 1,404-pair video benchmark, a new abstention metric (PECS), and internal-representation experiments across 16 VLMs spanning five families, showing that models encode when they should abstain but fail to say so.

What This Paper Is About

When a video is occluded, chaotic, or the question itself is unanswerable, a reliable model should say "I don't know" rather than guess. The authors build TRAPSBench (Testing Restraint in Ambiguous Physical Scenarios), in which MuJoCo generates matched "control" videos with deterministic outcomes and "void" videos that differ by one targeted change rendering the outcome incomputable. The paper's goal is to measure whether VLMs abstain in these cases and to locate where in the model the failure originates.

Key Contributions

1. TRAPSBench and PECS. A procedurally generated video benchmark of 1,404 matched answerable/unanswerable physics pairs across three uncertainty taxonomies (occlusion, chaotic sensitivity, ill-posed questions), plus the Penalized Epistemic Calibration Score, a conjunction metric that awards zero to both always-abstain and never-abstain strategies.
2. A systematic representation–output gap. VLMs internally encode epistemic uncertainty while their autoregressive outputs suppress it, confirmed three ways: guided prompting unlocks latent abstention (median 1.9× across main-family video-native models), linear probes transfer the internal signal across physics domains (AUROC up to 0.91), and single-layer activation steering causally controls abstention. All three results replicate across three open-weight families with no shared training pipeline (Qwen3-VL, Gemma, LLaVA).
3. Failure-mode asymmetries. Across sixteen VLMs and three prompt regimes, models detect textual impossibility more readily than visual information gaps, and chain-of-thought reasoning can degrade calibration: Qwen3-VL Think overrides its own internal doubt to confabulate more than its non-thinking variant.
4. Causal mechanistic analysis. Cross-dataset steering in Qwen3-VL-8B shows occlusion-family void directions encode a domain-general "evidence is missing" signal that transfers across domains and modalities, while chaotic directions are domain-specific and near-orthogonal, suggesting epistemic transparency governs transferability.

Main Findings

• Spontaneous restraint is poor. Across 16 VLMs in five families, the best standard-regime PECS is 0.292 (Gemini 2.5 Flash: 63.4% accuracy, 49.4% spontaneous abstention recall). Under guided prompting, Gemini 3.1 Pro R-Low reaches 0.568.

• The bottleneck is expression, not perception. Linear probes on frozen Qwen3-VL-8B hidden states (all 37 layers, 2,808 forward passes) decode void vs. control with best-layer AUROC up to 0.91 across physics domains. Cross-dataset AUROC stays largely intact when both training and test sets are restricted to samples the model confidently confabulated on (category means differ by ≤ .03).

• Steering causally controls abstention. A single-layer void direction at layer 20 induces or suppresses abstention in a different dataset. In Qwen3-VL-8B, +α steering reaches 0→1% control abstention under standard inference but 11→54% under guided, while −α works under standard inference (void: 10→4%); the constraint is specifically anti-abstention.

• Occlusion directions generalize; chaotic ones do not. At α = 10, occlusion-family directions reach 75% control abstention across cross-domain paths versus 15% for chaotic. Cosine similarity shows oc, oc_ip, and ch_ip form a correlated cluster (0.19–0.25) while ch is near-orthogonal (≤ 0.08). Same-domain steering confirms both are potent at home (ch→ch: 70%; oc→oc: 46%), so the asymmetry is about generality, not potency.

• Textual impossibility is detected far more readily than visual gaps. Averaged across splits, all sixteen models lie above the identity line. The chaotic splits show a 3–25× gap for main-family video-native models and up to 197× for image models, with a median per-model gap of about 4×.

• Guided prompting mainly shifts discrimination rightward. Guided prompting raises abstention recall 1.4–2.8× (median 1.9×) without hurting accuracy (±2 percentage points for most). The paper reports a median 2.6× increase across the fourteen main-family models (up to 11×). Yet no model reaches the 0.6 iso-PECS contour, which the authors describe as headroom rather than a trade-off.

• Chain-of-thought helps or hurts depending on family. Gemini 2.5 Flash thinking improves abstention recall by 4–13 pp across all datasets; reducing Gemini 3.1 Pro's reasoning budget barely moves calibration (PECS 0.199 vs. 0.197) but raises fabricated premises (96% vs. 87%). Qwen3-VL Think degrades abstention recall versus Instruct (chaotic ill-posed: 71.7% vs. 82.3%) despite the highest doubt rate (24%).

• Confabulation is mostly fabricated observation. 87–99% of confabulations across all six models fabricate visual observations not present in the video, indicating evidence fabrication rather than reasoning error alone.

• Prompting acts as an output gate. Standard-prompt void directions remain nearly as effective under guided inference (54% vs. 60% in Qwen3-VL-8B), suggesting the guided prompt lets an existing signal through. The gate is family-specific: full in Qwen, partial in Gemma (+α: 17–20% standard vs. 39–57% guided), absent in LLaVA (≥50% under both).

• Judging is reliable. The 3-model judge panel (Gemini 3 Flash, Qwen3-VL Instruct, Claude 4.6 Opus) shows unanimous agreement of 88.3–99.5% and Fleiss' κ ≥ 0.84.

Methodology in Plain English

The authors use the MuJoCo physics simulator to generate matched video pairs. A control video contains everything needed to predict the outcome; a void video is identical except for one change that makes the outcome undeterminable — an occluder blocking the answer, a chaotic scenario truncated before resolution, or an ill-posed question. Because the simulator knows the ground-truth state, correct answers are self-labeled, and procedural generation avoids pre-training contamination.

Each model sees a (video, question) pair and returns free-form text. A three-model judge panel scores the text alone — never the video — checking control answers against the MuJoCo ground truth and detecting abstention on voids. The headline metric, PECS, multiplies control accuracy by a clamped Youden's J statistic (abstention recall minus false abstention), so a model that abstains indiscriminately scores zero regardless of how often it abstains.

To find out whether the failure is perceptual or expressive, the authors open the models up. They train simple logistic regression probes on hidden states from frozen open-weight models to see whether void vs. control is linearly separable, and they extract a "void direction" (the normalized difference between mean void and control activations) at one layer, then add or subtract it during generation to test causality. Experiments run under three prompt regimes (Standard, Guided, JSON), and the full pipeline is run three times end-to-end with independent inference.

Why This Matters

Impact on research. The paper argues that abstention benchmarks which only measure behavior miss something the model already knows internally. It extends probing and activation-steering work on truthfulness and refusal to visual epistemic uncertainty, and it introduces a metric that explicitly zeroes out degenerate strategies (the authors verify six such strategies all yield PECS = 0).

Real-world applications:

• Autonomous robots and self-driving systems that must stop or ask for help when a sensor view is blocked or a scene is too chaotic to predict.
• Medical or scientific imaging assistants that should decline rather than guess when the relevant anatomy is not visible.
• Industrial inspection and manipulation systems operating around occluders or highly sensitive dynamics.
• Any deployed agent where a confident wrong answer carries more cost than a refused answer.

Industry relevance. The evaluated models come from three major proprietary families (Gemini, GPT-5, Qwen3-VL) plus open-weight models, and the benchmark is text-only-scored, so it works on black-box APIs. The finding that reasoning-tuned variants can confabulate more than their non-reasoning counterparts is a direct signal for teams training or deploying chain-of-thought models. Code and benchmark are released at https://github.com/facebookresearch/TRAPS-Benchmark under CC BY-NC 4.0.

Future Directions

• Output-stage interventions. The authors conclude that closing the representation–output gap will likely require intervening at the output stage rather than scaling models, but they do not specify which interventions.
• Extending to harder physics. They list soft bodies, fluids, real-world video, void-video fine-tuning, and generative rollouts as future work, noting the procedural setting is deliberately an easier test case.
• Explaining family-specific behavior. The prompt gate and direction geometry differ by family, and the authors scope those claims accordingly; why Gemma's gate is partial and LLaVA's absent remains open. A candidate mechanism offered for CoT's variable effect is reward design that rewards fluent, committed reasoning without penalizing speculation, but this is not tested.
• Probing closed-weight models. Closed-weight models cannot be probed or steered, so the mechanistic results cover three open-weight families; whether the same internal structure holds for API-only models is unaddressed.

Target Audience

Researchers and engineers working on VLM reliability, abstention and selective prediction, mechanistic interpretability, and physical reasoning benchmarks. The paper is also relevant to practitioners deploying vision-language agents in safety-critical settings who need to know how much to trust a model's confidence, and to teams building reasoning-trained models who want evidence that chain-of-thought can worsen calibration. Beginners will find the benchmark framing and PECS metric approachable, but the probing and steering sections assume familiarity with hidden-state analysis.

Authors’ abstract

When visual evidence is occluded or chaotic, models should abstain. In this paper, we show that Vision-Language Models (VLMs) can internally distinguish when abstention is required, but fail to express it anyway. We introduce TRAPSBench, a procedurally generated video benchmark of 1,404 matched physics pairs in which a single targeted change renders the outcome undeterminable from the visual evidence. Furthermore, we introduce Penalized Epistemic Calibration Score (PECS), a new robust metric that requires models to both answer correctly when the outcome is knowable, and abstain when the outcome is not. Across 16 VLMs spanning five families, spontaneous restraint is poor: the best PECS is 0.292. The bottleneck is expression, not perception: linear probes decode answerability from hidden states at up to 0.91 AUROC across physics domains; steering a single-layer void direction causally induces or suppresses abstention. Our results replicate across three open-weight families (Qwen, Gemma, LLaVA). The failure is also more pronounced in visual than textual uncertainty: models detect textual impossibility about 4x more readily than missing visual evidence. Closing this representation--output gap likely requires output-stage interventions.