The Pulse
Study finds AI safety guards can give unsupported explanations
Researchers found that explanations accompanying multimodal safety decisions often contained unsupported visual facts or incorrect safety interpretations. They propose a training method that rewards visual tool use when it improves grounding.

AI systems that judge the safety of image-and-text content can produce explanations containing unsupported visual details or incorrect interpretations, even when their final classification is right, a preprint found.
Source arxiv.org
The study examined eight rationale-generating safety guard methods across five vision-language backbones and four safety datasets. The authors, who are affiliated with Harbin Institute of Technology, submitted the preprint to arXiv on Nov. 19.
Two kinds of error
Multimodal safety guards classify image-and-text inputs as harmful or benign. Some also produce a rationale explaining the basis for the decision, including where a risk may arise from an image, text or their relationship.
The researchers describe two types of hallucination in those explanations. A visual hallucination introduces facts that the image does not support; a safety attribution hallucination gives observed content an incorrect safety interpretation relative to the dataset's annotations.
The preprint identifies over-flagging, in which a rationale assigns an unsupported harmful attribution to benign content, and under-flagging, in which it omits the harmful attribution from harmful content. It also identifies mis-attribution, where an explanation assigns harmfulness to the wrong target despite correctly identifying the content as harmful.
How the explanations were checked
To assess visual grounding, the researchers used a language model to extract descriptive clauses from each rationale, excluding subjective judgments and hypothetical statements. They then broke the clauses into individual visual, attribute and relation facts. A verifier determined whether each fact was supported by the corresponding image, producing a visual evidence score.
For safety attribution, a judge extracted the attribution from the rationale within the relevant dataset's annotation space. The researchers compared it with the ground-truth attribution. This assessment addresses a different problem from visual grounding: an explanation can describe an image accurately but still give an incorrect account of its safety implications.
The researchers found that neither language-workflow methods nor methods using visual inspection tools consistently eliminated hallucinations. They also found that higher classification accuracy did not necessarily mean the supporting explanations were better grounded.
Visual tools and training
The preprint examines a method known as Think-with-Image, in which models can inspect visual information using operations such as cropping, rotation and optical character recognition before generating a rationale. In the researchers' analysis, safety-critical attributes and relationships were more error-prone than basic visual facts, while the methods used tools infrequently and often made coarse crops.
The authors propose a training approach called Tool-Contrastive Grounding Reinforcement Learning, or TCG-RL. It compares rationales generated with and without visual tools and rewards tool use when it produces a better-grounded explanation.
The authors report that the approach improved visual rationale grounding by approximately 20% over strong baselines while maintaining comparable safety prediction accuracy. They also report that it produced more concise and safety-relevant rationales.
Sources
- Original source · Preprint When Safety Rationales Hallucinate: Diagnosing and Grounding Multimodal Safety Guards arXiv:2611.25793
Related stories
- Google details EmbeddingGemma 2’s multimodal design in report
- TuringSpace test finds spatial action gaps in leading AI models
- Microsoft study finds Kimi K3 probe signals can arrive late
- Alibaba releases Qwen4-Max with a million-token context window
- MIT and Cornell study tests adapting AI text detectors to new writing