When the Judge Acts: Auditing VLM-Guided Image Selection on Culturally Situated Prompts

AI summary

Overview

Research area: Computer vision and multimodal evaluation, specifically auditing vision-language models (VLMs) that act as judges selecting among pre-generated images, with an emphasis on culturally situated prompts.

Technical level: Intermediate. The core ideas are accessible without deep machine-learning background, but the paper relies on statistical framing (regret, bootstrap intervals, selective prediction, risk-coverage) that assumes some familiarity with evaluation methodology.

Scope: A single-author audit of two small open-weight VLM judges (Qwen3-VL-4B-Instruct and SmolVLM2-2.2B-Instruct) on 300 culturally situated prompts from the CulturalFrames dataset, comparing the images they actually return against held-out human ratings, exact random choice, a CLIP similarity baseline, and an annotation oracle, plus a scale extension with Qwen3-VL-8B-Instruct.

What This Paper Is About

VLMs are increasingly used as judges that pick the best of several generated images, so their choices determine what users actually receive. Existing validation of these judges usually checks whether their scores correlate with human ratings, not whether the image they return is any good. This paper treats the judge as a decision-maker: it measures the human-rated quality of the selected image against the best available candidate and against random choice from the same pool, and asks whether agreement-based filters (consistency across presentation orders, or agreement between two judges) reliably separate good decisions from bad ones.

Key Contributions

1. An audit of judges as decision-makers. Each returned image is scored against held-out human ratings and against exact random choice from the same fixed candidate pool, so every comparison is paired and never uses the judge's own scores.

2. A measurement of how position bias changes what users receive. Both small judges favor early presentation slots, and the same prompt can yield the worst or the best available image depending only on candidate order. Post-hoc ablations show the bias is not caused by the output format or the label letters.

3. An audit of agreement gates across judges. For the 4B judge, keeping only decisions consistent across presentation orders retains choices that beat random, while agreement with a weaker second judge does not. For an 8B judge with little position bias, the same unanimity gate mostly discards good decisions.

4. A released, deterministic artifact. Code and a project page are provided at https://github.com/seochan99/JudgeActs, including an offline browser for every recorded decision, with make analyze recomputing every table, figure, and number from frozen judge logs.

Main Findings

• The 4B judge barely beats random and loses to a simple similarity scorer. Qwen3-VL-4B-Instruct returns a valid choice for all 300 main prompts, with mean alignment 0.770 against 0.730 for random choice and 0.929 for the annotation oracle, a gain of 0.039 (95% CI [0.016, 0.062]). Mean regret is 0.160 [0.136, 0.184]; it returns a best-rated image for 51.0% of prompts and a below-mean image for 36.3% versus 43.1% for random choice. It returns a below-mean image less often than random (paired difference −0.067, 95% CI [−0.118, −0.016]), but still falls short of the CLIP baseline, whose gain of 0.069 is 0.030 higher (95% CI [0.005, 0.055]).

• The 2.2B judge does worse than random. SmolVLM2-2.2B-Instruct acts on 297 prompts (99.0% coverage) with mean alignment 0.659, regret 0.270, gain −0.073 [−0.093, −0.052], and a below-mean rate of 60.3%.

• Position bias is strong in both small judges. Qwen picks slot A in 48.8% of its 900 calls, where uniform choice given the pool sizes would give 27.8% (chi-squared with 3 degrees of freedom = 219, p < 10^-10), and its choice changes across the three orders for 59.7% of prompts. Smol places 93.3% of its choices in slots A or B. Qwen still responds to content: when rotations spread generators across slots, it picks images from the generator with the highest mean human rating in 57.9% of calls, against 27.2% under uniform choice.

• Order unanimity separates good decisions from bad ones for the 4B judge. Unanimity keeps 121 prompts (40.3%), where Qwen's gain over random is 0.141 [0.110, 0.173] and regret is 0.074; on the 179 rejected prompts its gain is −0.030 [−0.057, −0.002], below random. The kept pools are not easier: their mean best-minus-worst rating range is 0.454 versus 0.404 for rejected pools. The two-of-three gate keeps 80.7% of prompts with gain 0.094 [0.071, 0.118], and simple voting over the three orders raises gain across all 300 prompts from 0.039 to 0.065 [0.043, 0.086].

• Agreement with a weaker judge keeps the wrong prompts. Qwen and Smol agree on 59 prompts (19.7%), where Qwen's gain is −0.060 [−0.111, −0.011]; on the 241 prompts where they disagree it is 0.064 [0.039, 0.087].

• The position bias is a property of the judge, not the interface. With a choice-only output format, Qwen still picks the first slot in 44.8% of calls and changes its choice on 54.0% of prompts. With opaque two-character codes instead of A/B/C/D, it picks the first-presented image in 51.8% of calls and changes its choice on 62.7% of prompts. Adding a fourth rotation for four-image pools gives slot shares of A 52.5%, B 20.4%, C 14.4%, D 12.8% against a uniform 25.0%.

• Scale largely removes the bias and inverts the gate's value. Qwen3-VL-8B-Instruct picks the first slot in 27.3% of calls, close to the uniform 27.8% (p = 0.06), though its choice still changes across orders on 43.0% of prompts. Its gain over random is 0.097 [0.077, 0.117], above the CLIP baseline by 0.028 (95% CI [0.003, 0.054]), and its stereotype difference is no longer detectable (0.016 [−0.004, 0.036]). For the 8B judge, prompts that unanimity rejects stay above random (0.068 [0.036, 0.099]), so the gate mostly discards good decisions. On the 126 prompts where the 8B and 4B judges disagree, the 8B choice has gain 0.098 and the 4B choice −0.039.

• Quantization and runtime do not explain the scale difference. The 8-bit 4B control run agrees with the main 4B run on 94.9% of calls and reproduces its first-slot share (48.2%) and gain (0.044).

• An alignment gain does not carry over to every cultural outcome. Qwen lowers missing explicit expectations by 0.056 [−0.084, −0.026] and missing implicit expectations by 0.034 [−0.057, −0.009], both surviving Bonferroni adjustment. It raises the stereotype rating from 0.125 to 0.148 (difference 0.023, pointwise 95% CI [0.002, 0.045]), but this interval includes zero after adjustment ([−0.002, 0.051]). On pools where stereotype ratings differ across candidates, the difference is 0.036 [0.004, 0.068]. The increase is no longer detectable after aggregating across orders: 0.007 [−0.013, 0.027] for voting and 0.010 [−0.019, 0.039] on unanimous prompts, versus 0.032 [0.003, 0.063] on rejected prompts.

• A CLIP baseline also raises stereotype ratings. A ViT-L/14 CLIP prompt–image similarity scorer with no order sensitivity achieves regret 0.130 [0.109, 0.153], gain 0.069 [0.047, 0.090], and a below-mean rate of 28.0%, but raises the stereotype rating relative to random by 0.026 [0.006, 0.046], as much as Qwen does, while sharply reducing missing explicit expectations (−0.113). A smaller ViT-B/32 backbone gives nearly the same gain (0.063 [0.042, 0.085]) but no stereotype increase (0.003 [−0.016, 0.023]).

• Agreement with an independent, position-invariant scorer behaves like order unanimity. Qwen agrees with CLIP on 49.0% of prompts (chance 27.8%); where they agree its gain is 0.099 [0.067, 0.130], and where they disagree it is −0.018 [−0.047, 0.011].

• Country-group results are underpowered. Qwen's point estimate of gain is positive in 7 of 10 country groups, ranging from 0.107 in Canada to −0.018 in India, with the highest regret in Japan (0.275, pointwise 95% CI [0.171, 0.387], where random choice also has the highest regret at 0.266).

• Order checks are cheap. On the authors' hardware, one Qwen decision takes a median of 3.5 s and three orders take 10.6 s.

Methodology in Plain English

The starting point is the public CulturalFrames release, which contains culturally situated prompts, several generated images per prompt, and human ratings of prompt alignment (mean values 0, 0.5, or 1), plus flags for stereotypes and missing explicit or implicit expectations. The release has 3,637 image rows and 3,577 image annotation records comprising 10,412 individual ratings; 60 image rows have no annotation record (1.6%). After filtering, decoding every image, checking identifiers, and requiring at least three valid annotated candidates, 963 eligible sets remain, reduced to 961 unique prompts after removing 2 repeated requests within countries. The main split used is 300 prompts, 30 per country across 10 country contexts (100 three-candidate and 200 four-candidate sets), with 11 sets having identical candidate alignment scores. Splits were determined by a deterministic hash of prompt identifiers using seed 20261002, without any judge outputs or annotation values.

For each prompt, the judge sees only the text prompt and the candidate images, with no generator names, country metadata, or human ratings. Each pool is shown in K = 3 cyclic presentation orders (a canonical order plus two deterministic rotations), with candidates placed in labeled slots A through D. The judge's answer is recorded as the image identity, not the slot label.

The central measure is selection regret: the gap between the highest-rated candidate in the pool and the image the judge returned. The authors also report gain over random (paired against exact random choice computed analytically on the same pools, not simulated), the below-mean rate (BMR, the share of returned images rated below the pool mean), and the share of returns that are best-rated. The annotation oracle, which always picks the highest-rated candidate, is a reference point rather than a deployable selector.

Filters are tested as gates. A gate with threshold tau acts only when the fraction of the three orders agreeing on the same image reaches tau: 1 is unanimity, 2/3 requires two of three, and 1/3 always acts. A cross-model gate acts only when two judges make the same order-1 choice. Crucially, for every gate the authors report outcomes on the prompts it keeps and on the prompts it rejects, compared against random choice on exactly the same prompts, so the filter cannot look good simply by keeping easy pools. Gate thresholds were declared before main inference.

The primary judge is Qwen3-VL-4B-Instruct and the secondary judge is SmolVLM2-2.2B-Instruct, both run locally with greedy decoding, a 64-token output limit, and no requested explanation. Qwen images fit an aspect-preserving 448-pixel bounding box; Smol uses a native 384-pixel view with image tiling disabled, so the two judges receive different visual token budgets and their difference cannot be attributed to capability alone. The scale extension uses Qwen3-VL-8B-Instruct at 8-bit weights, with a matched 8-bit 4B control.

Analyses use 10,000 prompt-level bootstrap resamples with seed 20261002 for means and paired differences, and Wilson intervals for a single policy's BMR. The three order calls for a prompt are never resampled independently. Position, complement-set, content-baseline, ablation, and scale analyses were added after main inference and are labeled exploratory.

Why This Matters

Impact on research. The paper argues that score agreement with human ratings is not evidence that a judge makes good decisions, and demonstrates the gap concretely: a judge can rank plausibly yet return a low-rated image depending only on where that image appeared. It also shows that agreement-based abstention is not a general-purpose safety mechanism. A gate validated for one judge can be useless or harmful for the next, so filters around judges need the same auditing as judges themselves, including on the prompts they reject.

Real-world applications:

• Best-of-N image generation interfaces, where a judge silently determines which of several candidates a user receives.
• Image and media curation pipelines that use a VLM to pick one output from a candidate set for publication or review.
• Culturally sensitive content tools, where the paper shows a selection can reduce missing expectations while slightly raising stereotype ratings, meaning alignment improvement does not guarantee every cultural outcome improves.
• Evaluation and leaderboard design, where the findings argue for measuring returned outcomes under reordered inputs rather than only judging score correlation.

Industry relevance. Order-robustness testing is inexpensive relative to its value: three orders cost roughly three times a single call (a median of 3.5 s versus 10.6 s per decision on the authors' hardware). The finding that position bias largely disappears at 8B parameters while the need for re-auditing does not is directly actionable for teams that swap judge models, and the observation that CLIP-backed similarity scoring raises stereotype ratings in one backbone but not another warns against treating a simple content scorer as a bias-free substitute.

Future Directions

• Audit larger and proprietary judges. The study covers 4B and 2.2B judges plus one 8B extension run at 8-bit weights; whether the position-bias reduction continues at larger scales is untested.
• Determine who handles abstained requests. Abstention leaves a request unresolved, and the paper states it does not assume a human fallback or a zero-risk outcome, leaving the cost and outcome of rejected requests an open practical question.
• Design gates targeted to specific failure modes. The results show agreement helps only when it targets the judge's actual failure mode; developing filters that are diagnosed per judge, and re-auditing them when the judge changes, is the implied next step.
• Separate alignment from cultural outcomes systematically. Because stereotype, missing explicit, and missing implicit expectations move differently, the paper argues against composite scores and for auditing selectors separately on each cultural outcome over broader countries, domains, and generator pools.

Target Audience

Researchers and practitioners working on multimodal evaluation, VLM-as-judge pipelines, best-of-N selection, and culturally situated image generation will benefit most. It is also relevant to teams shipping image generation or curation products who need to decide how much to trust an automated selector, and to methodologists interested in selective prediction and risk-coverage analysis applied to model decisions rather than to scores.

Authors’ abstract

Vision-language models (VLMs) increasingly act as judges that pick the best of several generated images, so their choices decide what users see. Such judges are usually validated by score agreement with human ratings, not by the images they return. We audit VLM judges as decision-makers: on 300 culturally situated prompts, we compare the returned image with human ratings the judge never sees and with random choice from the same candidates, and repeat every decision with the candidates reordered. A 4B-parameter judge barely beats random and falls short of a CLIP similarity baseline. It picks the first image shown in 49% of calls (chance: 28%), and reordering changes its choice on 60% of prompts. For this judge, agreement across orders is informative: decisions that survive reordering are much better than random, whereas agreement with a weaker second judge keeps the wrong ones. An 8B judge shows almost no position bias and outperforms CLIP, yet for it the same filter mostly discards good decisions. Agreement helps only when it targets the judge's failure mode, so filters must be re-audited whenever the judge changes. The 4B judge's slight rise in stereotype ratings is no longer detectable after aggregating across orders or with the larger judge.