Towards Autopilot Labeling for Short Video Content Governance

AI summary

Overview

• Research area: Multimodal large language models (MLLMs) applied to content moderation and data annotation for short-video platforms, with an emphasis on LLM-based agents.
• Technical level: Intermediate — the paper assumes familiarity with MLLMs, supervised fine-tuning/LoRA, continued pretraining, and agent tool-use, but the framework itself is described conceptually.
• Scope (one sentence): The paper describes a four-stage "autopilot labeling" roadmap (L0 human → L1 discriminative MLLM → L2 generative MLLM → L3 agentic labeling) for annotating short videos that may violate platform guidelines, and reports accuracy and labor-cost results from a production deployment.

What This Paper Is About

Short-video platforms must label huge volumes of videos for policy violations (sexual suggestiveness, poor quality, clickbait, inappropriate promotion) so that downstream governance models can be trained. Today this work is done entirely by human annotators, which the authors describe as low in accuracy and consistency, slow, expensive (monthly annotation costs reaching tens of millions of dollars), and slow to iterate when guidelines change. The paper's goal is to progressively automate this labeling along a staged roadmap modeled on the levels of autonomous driving, while measuring how much human labor can be removed without degrading label quality.

Key Contributions

1. A staged automation roadmap for annotation: L0 (fully manual), L1 (fine-tuned discriminative MLLM that directly predicts a binary label), L2 (domain-adaptive generative MLLM used zero-shot to produce labels and explanations), and L3 (an LLM agent with tools that handles complex or ambiguous cases).
2. A discriminative MLLM labeler at L1: Built on LLaVA with a classification head, fine-tuned with LoRA (rank-168 matrices inserted into feed-forward transformation matrices of both the visual encoder and the LLM), consuming textual features (title, sticker text, OCR text, ASR transcripts) plus sampled video frames.
3. A guideline-decomposition method plus domain-adaptive continued pretraining at L2: Long annotation guidelines are split into sub-items (for example, borderline sexual content is split into 17 subcategories such as "Adult sexualized behavior" and "Adult body exposure"), each judged separately and aggregated with an "any" rule. Pretraining uses 920k instruction samples across three issues (sexually suggestive content, unoriginal content, clickbait) with three tasks (captioning, VQA, chain-of-thought) and three strategies (caption-only, mixing-all-tasks, two-stage).
4. Three agent tools at L3: referencing precedents (curated examples per sub-item), rethinking annotation guidelines (expanded definitions triggered only when the agent is unconfident), and zooming in on video frames (implemented as grid-cell selection over a 5×5 grid rather than bounding-box cropping).

Main Findings

• Performance rises with automation level: On the Sexually Suggestive Content (SSC) issue, F1 goes from 62.2 (L1) to 62.6 (L2 with guideline decomposition) to 76.5 (L3 with all three tools); on Poor-Quality Content (PQC), F1 goes from 93.4 (L1) to 81.9 (L2 decomposition) to 85.0/86.1 with precedents/rethinking; on Clickbait (CB), F1 goes from 64.1 (L1) to 67.9 (L2 decomposition) to 71.5 (L3 with zoom-in).
• L2 zero-shot alone is weak: Without guideline decomposition, the zero-shot generative labeler reaches only F1 36.0 on SSC (accuracy 60.2, recall 22.1), which decomposition lifts to F1 62.6.
• Combining L1 and L3 is the best configuration: Using the discriminative labeler first and re-labeling only uncertain cases (non-extreme prediction probabilities) with the L3 agent yields SSC F1 81.3 (accuracy 82.4), PQC F1 95.8 (accuracy 96.0), and CB F1 74.6 (accuracy 78.4). The 95% confidence intervals of F1 for this method are SSC 81.3 ± 2.2, PQC 95.8 ± 1.5, CB 74.6 ± 1.7.
• Baselines underperform the framework: LLaVA-OV 7B reaches F1 23.4 (SSC), 61.7 (PQC), 47.0 (CB); Qwen2.5-VL 7B reaches 33.4, 58.4, 46.5; GPT-5 reaches 56.5, 80.5, 60.5.
• Human labor reduction in production: 27% for SSC, 52% for CB, and 85% for Inappropriate Promotional Content (IPC), with the abstract describing reductions of "up to 85%".
• Monetary cost reduction: Assuming one unit of cost per human-labeled video, commercial MLLMs cost 0.34–0.51 units per video and locally deployed MLLMs cost 0.068 units. This translates to 13–18% (SSC), 25–34% (CB), and 42–56% (IPC) cost reduction with commercial MLLMs, and 25%, 48%, and 79% with locally deployed MLLMs.
• More precedents help up to a point: The agent performs poorly with zero or few examples and saturates or declines with too many. The number of precedents used for SSC, PQC, and CB is 96, 12, and 40, respectively, which the authors attribute to differing guideline complexity (17, 4, and 12 sub-issues).
• Grid zoom-in beats bounding-box zoom-in: On SSC, no zoom-in gives F1 76.1, bounding-box generation gives 74.9 (worse than no zoom-in), and visual grid selection gives 76.5.
• The L1 labeler scales with supervised data: Reducing SSC training data from 100% to 20% drops R@P50 from 63.0 to 55.2, showing continued gains from more human labels but also heavy dependence on them.
• Agent cost and latency: At roughly 200k videos per day, L1 requires about 11.1 H100 GPU-hours per day versus about 83.5 for L3 (roughly 5× more queries per video, each with about 1.5× more tokens). Latency is 1.5–1.8s (L1), 5.5–12.2s (L2), 10.5–30.1s (L3), versus 35.1–80.4s for humans.
• Fairness audit: Disaggregating L3 errors on SSC by the primary language of the video's text signals gives F1 from 72.4 to 79.1 across the five largest language groups (76.5 overall), with the weakest group skewed toward false positives (FPR 13.2% vs. 8.6%).
• Failure modes: Continuous fast actions (piston movements, hip-thrust actions in SSC) that fall between sampled frames, and audio-dependent cues (silence in PQC, moaning in SSC).

Methodology in Plain English

The team treats annotation automation as a ladder. At the bottom, humans read guidelines and assign binary labels under either single-review or dual-review workflows. At L1, they take an open MLLM (LLaVA), add a classification head, and fine-tune it on 1 million to 5 million human-labeled videos per issue (about 10% positive) so it outputs a label directly; this offline model is much larger than the small online governance models it feeds. At L2, they stop training a classifier per issue and instead continue-pretraining one generative MLLM on short-video data using captioning, guideline-based visual question answering, and chain-of-thought tasks, then prompt it zero-shot with the guideline. Because guidelines run to tens of thousands of words, they split them into short sub-items (each under about a thousand tokens) judged in parallel, where any positive sub-item makes the video positive. At L3, they wrap this labeler in an agent built with the open-source Agno framework, capped at 10 tool calls (about 5 on average), giving it three tools: pull up similar labeled examples, re-read an expanded definition when unconfident, and crop into a grid cell of a frame to inspect a small region. Evaluation covers three issues (SSC, PQC, CB) on held-out sets of roughly 1k–2k examples with near-balanced classes (for example, SSC has 737 positive and 778 negative; PQC has 809 positive and 809 negative). In production, the model's labels are auto-accepted above an upper threshold or below a lower threshold on positive probability, with everything in between escalated to humans; the specific threshold values are not reported in the available text.

Why This Matters

• Research impact: The paper reframes annotation quality as an automation-level problem and shows that agentic tool use plus a discriminative/generative hybrid can outperform either alone, offering a template that transfers beyond content moderation to any long-guideline, multimodal labeling task.
• Real-world applications:
  • Trust-and-safety teams labeling policy violations at platform scale to train downstream enforcement models.
  • Creator-economy platforms measuring issue prevalence across large video corpora.
  • Reducing annotator exposure to distressing material by absorbing the high-confidence workload automatically.
  • Any moderation pipeline that needs fast iteration when written policies change monthly.
• Industry relevance: The system is described as deployed in production, with concrete operating numbers (about 200k videos per day, 11.1 vs. 83.5 H100 GPU-hours, 27%–85% labor reduction), which makes the cost trade-offs actionable for practitioners rather than hypothetical.

Future Directions

1. Reach L4 (fully autonomous) through stronger base-model reasoning, better-designed domain-adaptive pretraining tasks, and higher-quality pretraining data.
2. Expand the agent toolset with intelligent frame sampling for fast continuous actions and audio understanding for silence and other audio-based violations.
3. Add external information lookup and large-scale precedent retrieval, so the agent queries the internet (for example, to check whether a clip is from a recent film) and selects its own references instead of using a fixed example set per sub-item.
4. Broaden fairness auditing across intersectional slices as a prerequisite for raising the share of auto-accepted labels, and surface agent rationales to affected creators rather than only using them internally for QA.

Target Audience

Trust-and-safety and content-moderation engineers, applied MLLM researchers working on data annotation and agentic workflows, and platform data-operations leads who need to justify automation investments with concrete accuracy-versus-cost numbers. Readers should already be comfortable with MLLM fine-tuning and evaluation metrics, since the paper reports precision, recall, F1, and accuracy without defining them.

Authors’ abstract

The explosive growth of short videos has presented significant challenges to content governance. Traditional manual labeling is inefficient, slow to iterate, and costly. Therefore, we explore and develop a multi-stage autopilot labeling framework aimed at progressively increasing the level of automation in short video content annotation. To move beyond the heavy reliance on human expert labeling at the initial L0 stage, we first introduce a discriminative MLLM labeler at the L1 stage, which is fine-tuned with supervised data to directly predict labels for short videos. Advancing to the L2 stage, we shift to a domain-adaptive MLLM pretrained on short video data and use it for zero-shot labeling, enabling flexible generalization without task-specific supervision. In the L3 stage, we propose an agentic annotation workflow, and equip the agent with several key tools, which empower the agent to handle complex, unfamiliar, or vaguely defined cases, significantly reducing the need for human intervention. We conduct extensive experiments across various content issues, and results demonstrate that each stage of the autopilot labeling framework progressively improves labeling accuracy, and has significantly reduced manual labor costs in real-world applications by up to 85%.