DreamRealm: Exploring and Materializing Hallucination Worlds for Trustworthy Vision-Language Models

AI summary

Overview

Research area: Computer Vision / multimodal large language models (MLLMs); specifically post-training methods for measuring and reducing visual hallucination.

Technical level: Advanced. The paper assumes familiarity with supervised fine-tuning, preference optimization (DPO/GRPO), and contrastive objectives over token log-likelihoods, though the core idea is described in intuitive terms.

Scope (one sentence): The paper proposes DreamRealm, a post-training framework that mines a target model's own hallucinated answers, minimally edits images so those answers become correct, and trains the model to match each hallucinated answer to the image that supports it, reporting gains on six visual perception and hallucination benchmarks with Qwen3.5-9B.

What This Paper Is About

Multimodal models often state things that the image does not support, especially when the correct answer hinges on a small visual detail (an object's size, a count, a color). Existing fixes either correct the model's wrong answers while leaving the picture unchanged, or build contrasting images that are not derived from the specific mistakes the model actually makes. DreamRealm instead starts from the model's own errors: it collects the wrong answers the model is prone to give, edits the original image just enough to make each wrong answer correct, and then teaches the model which image each answer belongs to.

Key Contributions

1. RealmForgePipe — a data pipeline that explores and materializes a model's own hallucinations. It collects observed hallucinations (errors under standard decoding) and mines latent hallucinations (errors that surface under high-temperature sampling and enumeration of key perceptual attributes), ranks them by model likelihood, then derives image-editing targets from them so the wrong answers become valid.

2. RealmSet — the Realized Hallucination Dataset, built by applying RealmForgePipe to a subset of VC-STaR's VisCoR-55K (48,012 image–question inputs) using hallucinations mined from Qwen3.5-9B. It spans 22 perceptual question categories and contains 28,960 human-approved edited examples, with training and held-out splits separated by original-image family.

3. RealmMatch — a training objective that cross-pairs original and edited images with candidate answers to build a likelihood score matrix, then combines supervised fine-tuning on matched pairs with bidirectional contrastive losses (a row-wise image-conditioned response contrast and a column-wise response-conditioned image contrast, plus an SFT term).

4. Empirical validation and ablation analysis — results on six benchmarks plus ablations separating data-construction choices from training-objective choices on Qwen3.5-9B.

Main Findings

• Headline gains on Qwen3.5-9B: Compared with the original Qwen3.5-9B, DreamRealm improves performance across six visual perception and hallucination benchmarks, including +7.26 percentage points in HallusionBench accuracy and +11.34 percentage points in MMVP pair accuracy.

• Best on six of seven reported metrics: Under the authors' unified evaluation protocol, Qwen3.5-9B + DreamRealm obtains the best numerical value in six of the seven reported metrics. The single exception is ObjectHal, where it remains close to the best result from RLAIF-V.

• Model specificity matters: Reusing the Qwen3.5-mined RealmSet to train Qwen2.5-VL-7B, Gemma-4-E4B-it, and InternVL3.5-8B (without re-mining for those models) improves most metrics but by less than the matched Qwen3.5-9B setting; Gemma and InternVL also regress slightly on CV-Bench-2D. The authors conclude hallucination spaces overlap across models but are not fully interchangeable.

• Observed and latent hallucinations are complementary: In the data ablation, original-only supervision degrades HallusionBench, and unguided edits remain below Raw on HallusionBench and CV-Bench-2D. Targeting observed errors improves five of seven metrics over unguided editing, and adding latent hallucinations improves all seven metrics over the "Observed Only" setting.

• RealmMatch outperforms SFT, DPO and GRPO on the same data: With full RealmForgePipe data fixed, SFT improves all metrics and is best on AMBER; DPO is strong on MMVP but degrades HallusionBench; GRPO is more balanced but still behind RealmMatch. RealmMatch improves all seven metrics over Raw and is best on six.

• Construction yield is 30.07% end to end: From 487,812 scored candidate responses, 114,982 (23.57%) were retained for processing; 96,320 entered image editing, 91,986 produced edited images, 43,169 passed the VLM blind check, and 28,960 were human-approved. Per-category pass rates vary widely, from 65.18% (scene_description) and 55.81% (existence) down to 8.16% (chart_read) and 16.67% (multi_image).

• Dataset splits: The 28,960 approved edits split into 27,038 training and 1,922 test examples; training edits span 12,340 original-image families, and no family appears in both splits. The test set also includes 3,078 original examples, for 5,000 test examples in total.

• Costs: Image editing used gpt-image-2 at quality "low" at approximately CNY 0.05 per generated image (excluding VLM verification and human annotation); human review was done by 12 annotators at 500 samples per annotator-hour, charged at CNY 0.0568 per sample, with a second quality check on 10% of the data.

Methodology in Plain English

The method has two halves: making the data, and training on it.

Making the data (RealmForgePipe). Start with an image, a question, and the correct answer. Ask the target model the question many times at a high sampling temperature to see the different ways it goes wrong. Then push further: because perception questions usually hinge on an enumerable attribute (a color, a count, a size), list alternative values of that attribute as extra candidate wrong answers. Remove duplicates that mean the same thing, then score every remaining candidate by how likely the model thinks it is, using the average per-token log-likelihood of the answer. Keep the top ones — these are the "latent hallucinations," the errors the model is most inclined to make.

Making the supporting images. For each selected wrong answer, a verifier model describes what must be preserved and what must change, and proposes three editing instructions in increasing order of how much of the image they alter. An image generation model tries them in order, and the result must clear checks: an edit-success score of at least 4, a visual-plausibility score of at least 3, and a minimal-editing score of at least 4, all on a five-point scale. A blind check then hides the target answer from the verifier, asks the unchanged question of the edited image, and requires the verifier's answer to be equivalent to the intended wrong answer — that is, the answer is invalid on the original image and valid on the edited one. A within-family check requires each image to be matched only by its own answer and by no sibling's answer. Human annotators then review each accepted pair.

Training (RealmMatch). Each question becomes a family: one original image with the true answer, plus several edited images each with its supporting wrong answer. The model scores every image–answer combination, producing a matrix. Three losses act on it: a row-wise loss that prefers the correct answer for each image, a column-wise loss that prefers the supporting image for each answer, and a plain SFT loss on the matched pairs. Because a column fixes the answer text, the column loss directly compares the visual conditions under which the same answer is or is not valid. For examples with no accepted edit, only SFT on the true answer plus row-wise contrast against offline-verified wrong answers is used, since only one image exists.

Evaluation. Six visual perception and hallucination benchmarks, with baselines including VCD, CoCI-CAP, RLAIF-V, SymMPO, mDPO, VC-STaR, MED, IC-VCO, FINER, IRIS, and C²-DPO.

Why This Matters

The paper reframes hallucination mitigation as a matching problem rather than a correction problem: instead of only telling the model "you are wrong here," it teaches the model what the world would have to look like for the wrong answer to be right. That gives supervision on the image-input side, which the authors argue prior response-correction and preference-learning methods largely leave underexplored, and it ties the supervision directly to the target model's own confusions rather than to generic image contrasts from retrieval or unguided editing.

Potential real-world applications (the paper does not report deployed systems; these follow from the problem it targets):

• Medical and scientific imaging, where a reading hinges on a subtle size, count, or positional detail and a plausible-but-wrong description is costly.
• Accessibility tools that describe images for blind and low-vision users, where an unsupported claim about a scene can mislead rather than inform.
• Industrial and retail inspection, where verifying attributes such as color, material, count, or surface pattern is the core task.
• Document and chart understanding, where misread values or trends propagate into downstream decisions.

Industry relevance: The pipeline is built and evaluated on open-weight backbones (Qwen3.5-9B, Qwen2.5-VL-7B, Gemma-4-E4B-it, InternVL3.5-8B) and comes from the Yuvion Team at Alibaba Group, so it is directly applicable to teams shipping multimodal assistants that need fine-grained visual faithfulness. The reported construction costs and per-category yields also give practitioners a realistic sense of the data budget required.

Future Directions

• Per-backbone hallucination re-mining. The cross-model experiment shows that transferring RealmSet to other backbones helps less than training on a model's own hallucinations; running RealmForgePipe separately for each target model is the obvious next step, with the cost of doing so as the open question.
• Reducing construction cost. The authors list data construction cost (image generation plus human review) as a limitation, and the 30.07% overall pass rate — with categories as low as 8.16% for chart_read — suggests headroom in making editing and verification cheaper or more reliable.
• Overcoming image-editing limits. The method is bounded by the underlying editing model; some scenes remain difficult to edit successfully, so better or more controllable editors would broaden which hallucinations can be materialized.
• Extending beyond the current scope. RealmSet covers 22 perceptual question categories from a 48,012-input subset of VisCoR-55K; whether the assignment paradigm transfers to video, multi-image inputs (only 6 synthesis candidates, 1 human-approved in this work), or reasoning-heavy questions is untested here.

Target Audience

Researchers and engineers working on multimodal LLM post-training, hallucination mitigation, and vision-language data construction. It is most useful to readers who already know supervised fine-tuning and contrastive or preference-based objectives and want a concrete recipe for building supervision on the image side; the ablation breakdown of data construction versus training objective makes it valuable for practitioners deciding where to invest effort. Readers seeking a beginner introduction to hallucination evaluation will find the framing accessible but the method details dense.

Authors’ abstract

Multimodal large language models (MLLMs) remain prone to visual hallucinations when correct responses depend on subtle visual details. One line of post-training methods constructs visual contrasts through image retrieval, generation, or editing to improve fine-grained perception. However, these contrasts are not built around the model's own hallucinations, making it difficult to ensure that supervision targets its actual perceptual confusions. Another line suppresses hallucinated outputs through response correction or preference learning under fixed image conditions, but leaves the design of supervision signals on the image-input side underexplored. Inspired by how humans correct misconceptions, we introduce DreamRealm, a post-training framework that explores diverse latent hallucinated responses, minimally edits original images to make them valid, and trains the model to assign each response to its supporting image. Our data construction pipeline, RealmForgePipe, mines observed and latent hallucinations from a target model and materializes them through image editing. Using hallucinations mined from Qwen3.5-9B, we construct the Realized Hallucination Dataset (RealmSet). Our training objective, RealmMatch, constructs a likelihood score matrix across image--response combinations and combines supervised learning with bidirectional contrastive losses to reinforce correct matches and suppress mismatches. Compared with the original Qwen3.5-9B model, DreamRealm improves performance across six visual perception and hallucination benchmarks, including gains of 7.26 percentage points in HallusionBench accuracy and 11.34 percentage points in MMVP pair accuracy. Together with ablation studies on data construction and training methods, these results demonstrate that hallucination assignment is a promising paradigm for improving visual perception and mitigating hallucinations.