ExPO-HM: Learning to Explain-then-Detect for Hateful Meme Detection

AI summary

Overview

Research area: Natural Language Processing / multimodal content moderation — hateful meme detection, explainable AI, and reinforcement learning post-training of large multimodal models.

Technical level: Advanced. The paper assumes familiarity with SFT, DPO, GRPO, policy-gradient objectives, and entropy-based calibration metrics.

Scope: The paper introduces ExPO-HM, a training framework that makes an Explain-then-Detect multimodal model jointly improve binary hateful meme classification, fine-grained classification, and the quality of its written rationales, evaluated on HatefulMemes, MAMI, and PrideMM.

What This Paper Is About

Most hateful meme detectors output only a binary "hateful / benign" label, which is not enough for real content moderation, where moderators need to know the attack type, the targeted group, and the reason for the decision. Recent "Explain-then-Detect" systems that generate a rationale before the label perform worse than simple supervised fine-tuning baselines, and even reinforcement learning methods such as GRPO fail to close that gap. This paper diagnoses why those systems fail and builds a training recipe that makes explanation-driven detection outperform direct detection.

Key Contributions

1. A new paradigm result. The paper presents what it describes as the first Explain-then-Detect hateful meme detection system that outperforms direct detection, producing both accurate labels and interpretable rationales.
2. The ExPO-HM method. A three-part pipeline that mimics how human moderators are trained: SFT warmup on structured policy manuals (SFT-PM), GRPO with curriculum learning (GRPO-CL) that moves from fine-grained to binary classification, and a Conditional Decision Entropy (CDE) reward.
3. Conditional Decision Entropy (CDE). A new metric and reward signal defined as the entropy of the decision conditioned on the generated explanation, used both to score reasoning quality and to penalize confident-but-wrong predictions.
4. A broader evaluation setup. Beyond binary classification, the paper evaluates fine-grained attack-type and target-group classification across three datasets and reasoning quality via LLM-as-a-judge and human evaluation, with extensive baseline comparisons.

Main Findings

• Explain-then-Detect baselines underperform direct detection. On Qwen2.5-VL-7B, the Explain-then-Detect post-training variants (SFT/DPO/GRPO) consistently fall below the Direct-Detection SFT baseline; large agentic systems also lag, e.g. RA-HMD reaches 80.2 F1 on HatefulMemes binary versus 72.4 F1 for U-CoT+ (14B) and 65.6 F1 for LOREHM (34B).
• Naive post-training does not improve reasoning. With Qwen2.5-VL-7B on HatefulMemes, the zero-shot LLM-as-a-judge score is 5.0; DPO drops below that and GRPO only reaches 5.2, even though GRPO improves classification over zero-shot.
• ExPO-HM sets state-of-the-art results. With Qwen2.5-VL-7B it reaches 81.1 F1 binary, 75.6 F1 Attack, 77.2 F1 Target, 6.2 LLM-as-a-judge, and 0.03 CDE on HatefulMemes; 82.3 binary and 73.0 Attack F1 on MAMI; 78.7 binary, 68.4 Stance, and 65.1 Target F1 on PrideMM. With Qwen2.5-VL-3B it reaches 74.7 / 71.5 / 73.7 / 5.1 / 0.16 on HatefulMemes, 80.7 / 70.4 / 0.08 on MAMI, and 75.6 / 66.5 / 62.1 / 0.12 on PrideMM.
• Reported gains over RL baselines. The abstract reports up to 15% and 17% F1 improvement over the GRPO and DPO baselines respectively; on Qwen2.5-VL-7B the paper reports +14.4 F1 on HatefulMemes Attack and +12.7 F1 on Target versus GRPO with equal compute.
• Ablation confirms all three components matter. On HatefulMemes with Qwen2.5-VL-7B: baseline GRPO 74.5 / 61.2 / 64.5 F1 with 5.2 LLM and 0.263 CDE; adding SFT-PM gives 75.8 / 70.8 / 70.2 with 5.6 and 0.092; adding GRPO-CL gives 78.4 / 74.3 / 76.1 with 5.8 and 0.056; adding CDE gives 81.1 / 75.6 / 77.2 with 6.2 and 0.026.
• Curriculum learning lengthens reasoning. Standard GRPO produces short average responses of 28 tokens in binary classification, while GRPO-CL nearly doubles this to 52 tokens.
• CDE correlates with human-judge-style scoring. Across all reported setups with three random seeds each (60 data points), CDE shows a strong negative correlation with LLM-as-a-judge scores (Pearson r = -0.78, Spearman ρ = -0.81, both p < 0.001).
• Better decision calibration. ExPO-HM separates correct from incorrect predictions by CDE (means 0.019 vs 0.048) while GRPO does not (0.278 vs 0.226). For Qwen2.5-VL-3B, ExPO-HM reduces the Brier score from 0.590 to 0.283.
• No entropy collapse. Overall policy entropy remains comparable to the GRPO baseline without CDE, indicating the CDE reward, which acts only on the decision portion of the generation, does not reduce exploration.
• Human evaluation agrees. Coherence was 96% for GRPO versus 100% for ExPO-HM; helpfulness averaged 1.6 for GRPO and 2.2 for ExPO-HM on a 0–4 Likert scale, which normalizes to 4.1 versus 5.5 on the 0–10 scale used by the LLM judge.
• Warmup choice matters more than warmup scores. Binary-label SFT (SFT-B) looks strongest at the SFT stage but performs worse after GRPO-CL, even below no warmup; SFT-PM yields the best post-RL results (81.1 / 75.6 / 77.2 binary/Attack/Target F1 and 6.2 LLM score).

Methodology in Plain English

The authors started from an analogy: human content moderators are not trained on bare yes/no examples. They first study a detailed policy manual, then practice applying it, starting with fine-grained distinctions (what kind of attack, who is targeted) before making overall hateful/benign calls. ExPO-HM copies that sequence in three stages.

First, the multimodal model is fine-tuned on structured "policy manual" prompts built from each dataset's fine-grained labels and annotation guidelines, so it learns to tie decisions to explicit policy items. Notably, the authors do not use human-written gold rationales in this warmup, because they found those off-policy explanations hurt performance.

Second, the model is trained with GRPO, a reinforcement learning method that compares groups of sampled outputs against each other rather than using a separate value model. Curriculum learning is applied: the first 50% of training steps use only fine-grained data so the model explores reasoning about policies, and the remaining 50% use a balanced 50/50 mix of fine-grained and binary data. The authors tested other schedules and found performance similar as long as fine-grained reasoning came before binary classification.

Third, the reward is enriched. Standard GRPO uses a format reward and an accuracy reward, which the authors argue is too weak to shape reasoning — just as a human cannot learn from only yes/no labels. They add Conditional Decision Entropy: they sample many explanations, measure how uncertain the model's final decision is given an explanation, and reward confident correct answers, tolerate uncertainty on wrong answers, and penalize confident errors. CDE also serves as a diagnostic metric for explanation quality. Binary classification is evaluated with macro F1, fine-grained classification with micro F1 (due to class imbalance), and reasoning with an LLM-as-a-judge comparison against human rationales, plus the reported human evaluation.

Why This Matters

Impact on research. The paper challenges the assumption that adding reasoning to a classifier necessarily costs accuracy. It shows that the failure of prior Explain-then-Detect systems is a training problem — missing policy-level cues and a reward signal too coarse to shape reasoning — and offers two reusable ideas, curriculum ordering on fine-grained-to-binary tasks and an entropy-based proxy reward for explanation quality, that could transfer to other multimodal reasoning tasks.

Real-world applications:

• Content moderation queues that surface not just a flag but the attack type, targeted group, and rationale, letting human moderators triage faster.
• Transparency features that show users why a post was actioned, supporting appeals and policy communication.
• Policy auditing, since the model's rationales expose which policy categories it invokes and where it is uncertain.
• Trend monitoring of specific attack types or targeted communities over time, using the fine-grained outputs.

Industry relevance. Social platforms, trust-and-safety teams, and any organization running large-scale moderation pipelines have a direct interest, since the gains are reported on standard benchmarks at 3B and 7B parameter scale rather than requiring closed frontier models. The authors also note they deliberately excluded closed-source reasoning models from the comparison because over 30% of their API requests were blocked due to the harmful nature of the examples — a practical constraint for anyone building in this space.

Future Directions

• Scarce rationale data. Human-written rationales exist publicly only for the Hatred dataset built on HatefulMemes; the authors note other resources such as ArMeme are not yet publicly available, so scaling explanation supervision remains open.
• Multi-class generality of CDE. The paper adopts the full-vocabulary CDE formulation for generalizability to fine-grained multi-class labels, leaving further refinement of the multi-class case as an open question.
• Cultural and linguistic transfer. The ethical statement stresses that moderation judgments are culturally dependent, so adapting policy manuals and evaluating across diverse cultural and linguistic contexts is an unresolved next step.
• Bias and deployment risk. Because the models train on datasets that may reflect societal or annotator biases, the authors call for human oversight and further work on mitigating unfair or disproportionate moderation.

Target Audience

This paper is most useful to researchers and engineers working on multimodal content moderation, explainable AI, and reinforcement learning post-training of large multimodal models, as well as trust-and-safety practitioners who need interpretable detection rather than bare binary flags. It is best read by someone already comfortable with SFT, DPO, and GRPO; readers new to RL fine-tuning will need background reading on those methods, since the paper moves quickly through the policy-gradient and preference-optimization objectives.

Authors’ abstract

Hateful memes have emerged as a particularly challenging form of online abuse, motivating the development of automated detection systems. Most prior approaches rely on direct detection, producing only binary predictions. Such models fail to provide the context and explanations that real-world moderation requires. Recent Explain-then-Detect approaches, using Chain-of-Thought prompting or LMM agents, perform worse than simple SFT baselines, and even advanced post-training methods such as GRPO fail to close the gap. Our analysis identifies two key issues of such systems: important policy-relevant cues such as targets and attack types are not hypothesized by the model as a likely explanation; and the binary reward signal is insufficient to guide reasoning. To address these challenges, we propose ExPO-HM (Explain-then-Detect Policy Optimization for Hateful Memes), inspired by the training and evaluation process of human annotators. ExPO-HM combines SFT warmup, GRPO with curriculum learning, and Conditional Decision Entropy (CDE) as both metric and reward for reasoning quality. Across three hateful meme benchmarks, ExPO-HM achieves state-of-the-art performance on binary detection, fine-grained classification, and reasoning quality, with up to 15\% and 17\% F1 improvement over the GRPO and DPO baselines, respectively. By moving hateful meme detection from simple binary alarms to explanation-driven detection, ExPO-HM provides accurate, interpretable, and actionable moderation support. Code available at https://github.com/JingbiaoMei/ExPO-HM