Where Do Long Captions Fail? Position-Aware Diagnosis and Reinforcement Learning for Detailed Image Captioning

AI summary

Overview

Research area: Computer vision and vision-language models (VLMs), specifically long-form/detailed image captioning, its evaluation, and reinforcement-learning fine-tuning.

Technical level: Intermediate. Readers should be comfortable with autoregressive VLM captioning, rubric/claim-based evaluation, and policy-gradient methods such as GRPO; the paper explains its own positional metrics clearly.

Scope: The paper diagnoses where inside a long caption grounding breaks down, then converts that diagnosis into a position-aware GRPO reward (PAGER) that repairs the specific failures it identifies.

What This Paper Is About

Detailed captioning is usually treated as a question of how much a model says — more visual facts, fewer hallucinations — and evaluated with global scores that collapse a caption into a single number. The authors argue that because captions are generated as ordered sequences, it matters where supported, fabricated, and vague content appears. They build a position-aware representation of captions as ordered atomic claims, show that models fail differently in the middle versus the end of a caption, and train a reward that targets each region separately.

Key Contributions

1. A position-aware rubric representation. Dense human references are decomposed into atomic visual claims; a verifier labels each claim Supported, Contradicted, or Missing and returns an evidence span, which is then mapped to one of four sentence-index quartiles. This yields a per-quartile composition of supported, hallucinated, and generic (unmatched) content without an auxiliary classifier.
2. Two distinct, reliable position-dependent failures. Across six captioners from the Qwen, InternVL, and LLaVA families, hallucination peaks in the middle quartiles, while the final quartile fills with generic, non-evidential language — patterns absent in a same-reference human control.
3. A "substitution" rather than "monotonic decay" account. In three of six models, tail hallucination is lower than head hallucination even though tail genericity is much higher, meaning late captions often avoid concrete claims rather than accumulating more errors. This directly constrains reward design.
4. PAGER (Position-Aware Grounded Evidence Reinforcement). A GRPO objective with four terms — claim coverage, a middle-position reward for supported fine-grained evidence, a tail penalty for generic escape, and a content-aware length budget — each mapped one-to-one onto a diagnostic finding, evaluated for whether it repairs exactly those failures.

Main Findings

• Mid-caption unsupported specificity: Hallucination is not flat across a caption. For all six evaluated models, the mid-caption effect $D_{\text{mid}}$ is positive with a 95% CI excluding zero; the across-model mean is +0.101, ranging from +0.065 (Qwen3-VL-8B) to +0.216 (LLaVA-v1.5-7B). The middle is where models attempt fine attributes, small objects, and relations — the peak error region is where the caption tries to become more informative.
• Tail generic escape: Generic content rises sharply in the final quartile for every model, with mean $D_{\text{tail}}$ of +0.178, ranging from +0.090 (InternVL3.5-8B) to +0.380 (LLaVA-v1.5-7B). Supported content falls correspondingly: mean $\Delta S = S_{Q_4}-S_{Q_1} = $ −0.195.
• Substitution, not accumulation: Three of six models have lower hallucination in Q4 than Q1 (Qwen-Plus −0.027, Qwen3-VL-4B −0.018, LLaVA-v1.5-7B −0.027); the other three rise modestly from +0.034 to +0.094. Yet $D_{\text{tail}}$ is large and positive for every model.
• Human references do not show the pattern: Under the same pipeline, human references have weighted contradiction rates at or below 0.002, $D_{\text{mid}} = $ −0.001 [−0.003, +0.000] and $D_{\text{tail}} = $ −0.013 [−0.028, +0.004]. Tail support stays at $S_{Q_4} = $ 0.945 versus 0.333 averaged over model captions.
• PAGER reduces the diagnosed failures on the same 198-image subset: $D_{\text{mid}}$ drops from +.096 (base Qwen3-VL-4B-Instruct) to +.036; PAGER reaches the highest tail supported rate ($S_{Q_4} = $ .436) and the lowest tail hallucination rate ($H_{Q_4} = $ .235), so tail repair is not a trade of genericity for fabrication.
• Global quality is competitive but not uniformly best: PAGER has the best CAPTURE on IIW-400 (0.642) and DOCCI (0.610), with small margins over CapRL (0.640 / 0.604), but DCScore-F1 favors CapRL (0.552 vs 0.526 on IIW-400; 0.524 vs 0.502 on DOCCI). PAGER's 4-model tie-excluded CapArena win rate pools to 414/(414+16) = 96.28% on IIW-400 and 95.62% on DOCCI.
• Ablations confirm the intended division of labor: Removing $R_{\text{mid}}$ raises $H_{Q_2}$ from .364 to .421 and $D_{\text{mid}}$ from +.036 to +.103; removing $R_{\text{tail}}$ drops $S_{Q_4}$ from .436 to .374 and raises $G_{Q_4}$ from .329 to .378. A "monotone weight" baseline (increasing late-position weight) matches PAGER's mid effect ($D_{\text{mid}} = $ .037 vs .036) but lowers $G_{Q_4}$ to .307 while raising $H_{Q_4}$ to .291 — the highest tail hallucination among ablations, exactly the failure the diagnosis predicts.
• Verifier reliability and transfer: Agreement with human labels on 347 rubric judgments is $\kappa = $ .89/.92 for the two judges (Qwen3-8B supplies training judgments; DeepSeek-V3.2 stays outside the training loop). At a 100-step budget, three-seed 8B/32B training gives mean $D_{\text{mid}} = $ .041/.043 and $H_{Q_4} = $ .216/.214.

Methodology in Plain English

Step 1 — Turn references into checklists. An LLM extractor breaks each dense human caption into atomic, visually checkable claims, each tagged with a dimension (Entity, Visual_Detail, Interaction, Scene_Context) and a weight — default 1.0, 2.0, 1.5, and 1.0 respectively. Compound attributes about the same property of the same entity ("black-and-white dog") are deliberately kept as one claim so the checklist cannot contradict itself.

Step 2 — Check generated captions claim by claim. A verifier reads the whole caption and marks each rubric item Supported, Contradicted, or Missing, quoting the evidence sentence when it is not Missing.

Step 3 — Assign positions. The caption is split into sentences and divided into four ordered quartile bins by sentence index. Each non-missing claim inherits the quartile of its evidence sentence. Sentences that no supported or contradicted claim cites are labeled Generic — a residual category, not a separate classifier's output. This matters because it separates "conflicts with the reference" from "matches nothing in the reference," which can include both filler and correct but unreferenced detail.

Step 4 — Compare composition across positions. The signed effects $D_{\text{mid}}$, $D_{\text{tail}}$, and $\Delta S$ summarize whether hallucination rises in the middle and whether genericity rises at the end. Uncertainty comes from image-level paired bootstrap with B = 2000 replicates.

Step 5 — Turn the diagnosis into a reward. PAGER sums four terms under GRPO: a weighted coverage term (reward supported rubrics, penalize contradicted ones, treat missing as neutral); a middle term rewarding supported Visual_Detail and Interaction claims whose evidence lands in Q2/Q3; a tail term penalizing the fraction of final-quartile sentences a lightweight genericity judge calls generic (it deliberately rewards no non-generic tail sentence, so fabricated detail is still caught by the coverage penalty); and a length term penalizing only over-budget captions against an image-specific budget $L^{\star}(x) = L_0 + c|\mathcal{R}(x)|$. Training uses group-relative advantages with a KL penalty to a reference policy.

Step 6 — Data curriculum and evaluation. Early rubric construction used curated DCI and DOCCI captions; the final GRPO run uses a larger PixMo-derived corpus filtered by image quality, caption length, and a pre-inference-and-judge pass. The base model is Qwen3-VL-4B-Instruct, compared against the unmodified base, a PixMo-Cap SFT baseline (PixMol-4B), and CapRL as an external baseline. At inference the policy sees only an image — no reference or rubric.

Why This Matters

Research impact: The paper shifts detailed-captioning evaluation from "how many facts" to "which facts, where," and shows that hallucination-only metrics miss a real failure mode (vague, non-evidential endings). It also provides a general template — decompose outputs into ordered claims, diagnose position-wise composition, then align training signals to observed failure regions — that is not specific to captioning. The finding that a single monotone late-position weight trades genericity for hallucination is a concrete warning for reward designers.

Real-world applications:

• Accessibility: Alt-text and screen-reader descriptions for images, where a vague trailing sentence wastes the reader's time and a fabricated middle sentence actively misinforms.
• E-commerce and catalog metadata: Product descriptions must stay grounded in the actual image, and errors concentrate in attribute-heavy middle passages.
• Medical and scientific imaging reports: Position-aware grounding checks give auditors a pointer to which part of a generated report to review rather than a single trust score.
• Content moderation and dataset curation: Detecting "generic escape" helps flag machine-generated descriptions that sound plausible but carry no visual evidence.

Industry relevance: The method needs no reference at inference time, runs on a 4B base model, and the diagnostic itself is cheap to apply as a QA gate over existing captioning pipelines. The paper was carried out with Kuaishou Technology, which also provided compute.

Future Directions

• Reference-free or image-grounded verification. The current verifier is reference-supported, so correct details absent from the reference are labeled Generic and incomplete references can create false contradictions. Directly image-grounded checking would remove this ceiling.
• Scaling and reward-overfitting analysis. Transfer results are limited to short-budget settings (100 steps) with per-scale learning rates; the authors explicitly call for controlled scaling studies and broader reward-overfitting evaluation.
• Extending the recipe beyond captioning. The same ordered-claim decomposition could be applied to long-form multimodal reasoning, text generation, and tool-use trajectories, where position-dependent grounding decay may take different forms.
• Stronger human evaluation. Current human studies used unpaid volunteers from the authors' personal circle, and the post-RL residual-label audit is descriptive with intervals spanning zero; larger, independent audits would firm up the label validity and the characterization of unmatched tail sentences.

Target Audience

Researchers and engineers working on vision-language model training, captioning evaluation metrics, or RLHF/GRPO-style post-training who want a diagnostic tool sharper than a global score. It is also useful for practitioners deploying image-description systems where knowing which sentence is unsupported matters more than an aggregate hallucination rate. Readers without RL background can still follow the diagnostic half (Sections 2–3) independently of the training half.

Authors’ abstract

Detailed image captioning is usually framed as an amount problem: a model should mention more visual facts while avoiding hallucination. However, autoregressive VLMs generate captions as ordered sequences, and later generations may be produced under weaker or noisier visual grounding. We therefore argue that long-form captioning should be studied not only by what facts are mentioned, but also by where they appear. We introduce a position-aware rubric representation that decomposes dense human references into atomic visual claims, verifies generated captions against those claims, and induces a quartile-wise composition of supported, hallucinated, and generic content. This lens reveals two qualitatively different failures in long captions: models tend to produce unsupported specificity in the middle, while the final portion often escapes into vague, non-evidential language rather than simply accumulating more hallucination. A same-reference human control does not exhibit these patterns; additional human audits examine the visual validity of the induced labels. Motivated by this diagnosis, we propose PAGER (Position-Aware Grounded Evidence Reinforcement), a GRPO training objective with separate signals for supported fine-grained middle evidence, evidence-bearing tail continuation, claim coverage, and content-aware length control. The result is a closed loop: the same ordered claim representation identifies where long captions fail and supplies the position-specific reward needed to repair those failures. Experiments on detailed-caption benchmarks show that PAGER improves position-wise grounding and pairwise caption preference while preserving competitive global caption quality.