Symbal: Detecting Systematic Misalignments in Model-Generated Captions

AI summary

Overview

Research area: Computer Vision and multimodal large language models (MLLMs), with applications in natural-image and medical-image caption auditing. Technical level: Advanced. Scope: This paper introduces the task of systematic misalignment detection for MLLM-generated captions and presents Symbal, a dual-stage detection method, plus SymbalBench, a benchmark for evaluating automated methods on this task.

What This Paper Is About

MLLMs often generate captions with errors, causing images and captions to be misaligned. The paper focuses on systematic misalignments, where a recurring caption error is closely associated with a specific visual feature in the paired image. The goal is to detect such textual errors and their associated visual features from a vision-language dataset, without requiring access to the underlying MLLM.

Key Contributions

1. The paper formalizes the systematic misalignment detection task, where a method receives a vision-language dataset with images and MLLM-generated captions and must output a textual error and an associated visual feature.
2. It presents Symbal

Authors’ abstract

Multimodal large language models (MLLMs) often introduce errors when generating image captions, resulting in misaligned image-text pairs. Our work focuses on a class of captioning errors that we refer to as systematic misalignments, where a recurring error in MLLM-generated captions is closely associated with the presence of a specific visual feature in the paired image. Given a vision-language dataset with MLLM-generated captions, our aim in this work is to detect such errors, a task we refer to as systematic misalignment detection. As our first key contribution, we present Symbal, which utilizes a structured, dual-stage setup with off-the-shelf foundation models to identify systematic misalignments and summarize results in natural language. As our second key contribution, we introduce SymbalBench, a benchmark designed to evaluate automated methods on our proposed task. SymbalBench consists of 1.7 million image-text pairs from two domains (natural and medical images), organized into 420 vision-language datasets with annotated systematic misalignments. Symbal exhibits strong performance on this benchmark, correctly identifying systematic misalignments in 63.8% of datasets, a nearly 4x improvement over the closest baseline. We supplement our evaluations on SymbalBench with real-world evaluations, showing that (1) Symbal can accurately surface systematic misalignments in captions generated by four MLLMs and (2) Symbal is a powerful tool for auditing off-the-shelf image-caption datasets. Ultimately, our novel task, method, and benchmark can aid users with auditing MLLM-generated captions and identifying critical errors, without requiring access to the underlying MLLM. Code is available at https://github.com/Stanford-AIMI/Symbal.