BLM-Guard: Explainable Multimodal Ad Moderation with Chain-of-Thought and Policy-Aligned Rewards

AI summary

Overview

Research area: Multimodal content moderation / vision-language model alignment, applied to commercial short-video advertising.

Technical level: Advanced. The paper assumes familiarity with vision-language models, supervised fine-tuning, chain-of-thought prompting, and reinforcement learning (specifically GRPO).

Scope: The paper introduces BLM-Guard, a two-stage training framework (rule-guided chain-of-thought SFT followed by self-adaptive GRPO reinforcement learning) for explainable, policy-aligned moderation of short-video ads, along with a companion three-level annotated benchmark.

What This Paper Is About

Short-video platforms now carry huge volumes of multimodal advertising that mixes visuals, speech, and subtitles, and the violations involved are subtler than ordinary community-safety harms: exaggerated claims, misleading cues, or mismatches between what a video shows and what it says. Existing tools, whether static rule filters or general-purpose vision-language models, handle these poorly because they reason weakly across modalities, adapt badly when policies change, and lack commercial-risk-specific reasoning. The goal is a moderation model that both flags violations accurately and produces a readable, policy-grounded reasoning trace for each decision.

Key Contributions

1. BLM-Guard Benchmark: A real-world short-video ad dataset with a hierarchical, policy-aligned annotation scheme spanning a "No Risk" category plus seven core risk scenarios, fine-grained violation types, and three severity levels (high, medium, low), with structured reasoning traces available for a subset.
2. Rule-driven Interleaved-modal Chain-of-Thought (ICoT) synthesis pipeline: A data-generation procedure that selects keyframes and salient patch regions and uses a frozen InternVL-3-78B to produce structured scene descriptions, reasoning chains, and labels, reducing manual annotation cost.
3. Self-adaptive GRPO reinforcement learning with hybrid rewards: A training scheme combining rule correctness, output-format compliance, and a Self-Consistency and Adaptive Reward (SCA-R) supplied by a guide model that dynamically constructs scoring principles, intended to keep the model aligned when moderation policies drift.
4. Multitask modeling of intra-modal and cross-modal violations: A design that jointly models manipulations inside a single modality (e.g., exaggerated imagery) and mismatches across modalities (e.g., subtitle–speech drift) to improve robustness and generalization.

Main Findings

• Strongest scores on the internal benchmark: BLM-Guard reaches severity accuracy of 0.978 (High), 0.940 (Medium), 0.902 (Low) and 0.986 (None), wide accuracy 0.962 and strict accuracy 0.914, with risky-class precision 0.976, recall 0.962 and F1 0.969, and a GPT-4o-scored reasoning consistency of 0.845. The paper states these are the best values in every column of Table 1.
• Baselines trail on strict accuracy and consistency: Among compared models, the strongest strict accuracy outside BLM-Guard is 0.701 (Qwen2.5-VL-7B) and the highest baseline consistency is 0.701 (Kimi-VL-A3B-Thinking).
• Rule-anchored SFT beats single-component tuning: Ans-SFT alone gives strict accuracy 0.648 with consistency 0.412; Think-SFT alone gives 0.612 with consistency 0.585; Rule-SFT (5k) gives 0.783 with consistency 0.776.
• More rule-aligned data helps: Full-SFT on 13k examples reaches strict accuracy 0.7925 and consistency 0.762, versus Part-SFT on 5k at 0.768 and consistency 0.762.
• Reinforcement learning adds further gains: Adding Rule-RL raises strict accuracy to 0.801 and F1 to 0.894; the full setup with the SCA-R reward reaches strict accuracy 0.914, F1 0.969 and consistency 0.845.
• Generalization to public datasets: BLM-Guard transfers to LSPD, XD-Violence, UCF-Crime, FakeSV and FVC, with the paper reporting particular strength on misinformation scenarios (FakeSV/FVC). It notes these datasets do not directly evaluate factual consistency.
• OCR was deliberately excluded: Only ASR transcripts are used, because the authors state OCR in short videos is often noisy from overlays and harms reasoning.

Methodology in Plain English

The framework is trained in two stages.

Stage 1 – cold start with rule-guided reasoning. From each video, 16 frames are sampled uniformly and compared against 7 predefined risk prompts using CLIP-ViT-L/14 embeddings. A hybrid BIN+TOP strategy picks 3 keyframes, balancing temporal coverage with globally salient frames. Each selected frame is passed through InternViT-6B, and the highest-L2-norm patch is kept as the key region. These frames and regions, together with ASR transcripts and a safety-rule schema, are fed to a frozen InternVL-3-78B model that generates reasoning in stages: observation (describe visuals, summarize ASR, judge modality consistency), risk screening and causal analysis, then a final verdict. The resulting reasoning chains supervise fine-tuning with a combined loss: cross-entropy on the <answer> token plus a KL term that pulls the model's <think> reasoning distribution toward a rule-derived prior built from the target violation scene and type.

Stage 2 – reinforcement learning. A curated RL set is built by rejection sampling (4 high-temperature passes, keeping cases the model consistently fails) and safety-aware concatenation of semantically related prompts or risky ASR segments. For each input the model samples a group of responses, each scored by three rewards: a rule reward (1.0 if both violation scene and type match, 0.5 if only the scene matches, 0.0 otherwise), a binary format reward for containing both <think> and <answer> tags, and the SCA-R critique score, where a guide model receives the reasoning trace, ground-truth labels and moderation guidelines, builds weighted scoring principles such as causal clarity and risk attribution, and rates from {0, 0.5, 1}. Optimization uses GRPO with three modifications: token-level rather than sequence-level loss normalization, z-score group advantages, and dynamic sampling that skips batches where all sampled rewards are identical. The clipping factor is annealed over training steps.

Evaluation uses wide accuracy (partial match of scene or type), strict accuracy (both matched), precision/recall/F1 for the risky class, and a 0–1 consistency score assigned by a GPT-4o-based guide model. Baselines are LlavaGuard-7B, QwenGuard-7B, LLaVA-Next-Video-7B, InternVL3-8B, InternVL3-14B, Qwen2.5-VL-7B-Instruct (also the backbone), Qwen2.5-VL-32B, and Kimi-VL-A3B-Thinking, all adapted to video via uniform frame sampling and ASR concatenation.

Why This Matters

Research impact. The paper argues that general harm-detection benchmarks and guardrail models under-serve regulation-sensitive commercial content, and it contributes both a policy-taxonomy dataset and a recipe that combines rule-anchored chain-of-thought supervision with adaptive, critique-based rewards. It positions itself against prior work such as LlamaGuard, LlavaGuard, GuardReasoner and MM-RLHF by using policy-specific, dynamically constructed principles rather than a fixed reward.

Real-world applications:

• Automated pre-publication screening of short-video ads for exaggerated income claims, privacy exposure, or superstition-related content.
• Flagging cross-modal deception, such as videos whose visuals are truthful while the narration overclaims.
• Producing explainable audit records for reviewers and advertisers, since every decision comes with a structured reasoning trace.
• Detecting misleading or manipulated content on public benchmarks covering violence, anomaly, and misinformation (LSPD, XD-Violence, UCF-Crime, FakeSV, FVC).

Industry relevance. The authors are affiliated with Kuaishou Technology, Beijing University of Posts and Telecommunications, and Shandong University, and the data comes from a real short-video platform. Ad moderation is a high-volume, cost-sensitive operation where both accuracy and the ability to explain a rejection to an advertiser matter, which is exactly the trade-off this framework targets.

Future Directions

• Extending the three-level taxonomy and SCA-R principles to policy domains beyond the seven risk scenarios covered, since the paper evaluates unseen policies only by following the SafeWatch protocol.
• Improving use of on-screen text: the authors deliberately dropped OCR because of noisy overlays, leaving subtitle-level visual text handling as an open problem.
• Scaling the benchmark: the paper does not report the total size of the BLM-Guard Benchmark, so broader coverage and published size statistics would help reproducibility.
• Further work on evaluating factual consistency on the external datasets, which the paper notes do not directly measure this property.
• Reducing reliance on a separate guide model for reward scoring, which currently involves a GPT-4o-based critic.

Target Audience

Trust-and-safety and content-moderation researchers; practitioners building ad-review or compliance systems at short-video and e-commerce platforms; researchers working on vision-language model alignment, chain-of-thought supervision, or reinforcement learning from verifiable and critique-based rewards; and dataset builders interested in hierarchical, policy-aligned annotation schemes. Readers need comfort with VLM fine-tuning and policy-gradient RL to follow the methodology sections in detail.

Authors’ abstract

Short-video platforms now host vast multimodal ads whose deceptive visuals, speech and subtitles demand finer-grained, policy-driven moderation than community safety filters. We present BLM-Guard, a content-audit framework for commercial ads that fuses Chain-of-Thought reasoning with rule-based policy principles and a critic-guided reward. A rule-driven ICoT data-synthesis pipeline jump-starts training by generating structured scene descriptions, reasoning chains and labels, cutting annotation costs. Reinforcement learning then refines the model using a composite reward balancing causal coherence with policy adherence. A multitask architecture models intra-modal manipulations (e.g., exaggerated imagery) and cross-modal mismatches (e.g., subtitle-speech drift), boosting robustness. Experiments on real short-video ads show BLM-Guard surpasses strong baselines in accuracy, consistency and generalization.