Verus-Insight: Towards Explainable Artifact Detection in AI-Generated Videos

AI summary

Overview

Research area: Computer Vision — AI-generated video detection and explainable artifact analysis, using multimodal large language models (MLLMs).

Technical level: Advanced (assumes familiarity with MLLM fine-tuning, supervised fine-tuning, GRPO reinforcement learning, and detection benchmarks).

Scope: The paper proposes Verus-Insight, a detector trained to explicitly plan what to inspect, verify whether observed content is a genuine generation artifact, and aggregate that evidence into a real/AI-generated verdict, together with Verus-Bench, a benchmark of fine-grained human artifact annotations.

What This Paper Is About

Existing detectors for AI-generated video often produce a real/fake label without faithfully identifying the actual artifacts behind that decision: they may miss subtle spatiotemporal abnormalities, misread normal visual phenomena as artifacts, or land on the correct verdict for the wrong reasons. The authors reframe artifact detection as a structured three-stage process (Inspection Planning, Evidence Verification, Evidence Aggregation) and train a model with human-anchored supervision so that the final source decision is grounded in explicitly verified, human-perceivable artifact evidence. They also build a benchmark with fine-grained human annotations to measure artifact detection beyond video-level accuracy.

Key Contributions

1. Verus-Insight, a detector built on Qwen3-VL-8B-Instruct and trained with human-anchored full-process supervision plus progressive capability training, aimed at both fine-grained artifact understanding and video-level source detection.
2. A capability decomposition of artifact detection into Inspection Planning (what to examine), Evidence Verification (whether inspected content truly exhibits a generation artifact), and Evidence Aggregation (integrating verified evidence into the final verdict), with corresponding capability-level measures: Plan F1, Finding F1, and Balanced Accuracy (BA).
3. Verus-Bench, a benchmark of 192 videos (96 real, 96 AI-generated) containing 326 manually annotated artifact findings across 18 categories, each with an observation, temporal extent, and frame-wise bounding boxes, for evaluating artifact detection beyond video-level source prediction.
4. Design studies and analyses showing that the three capabilities are complementary yet distinct, and that Verus-Insight more reliably converts relevant inspection targets into supported artifact findings without increasing spurious artifact claims.

Main Findings

• Large video-level gains over the backbone: Verus-Insight raises RewardData BA from 48.81 (Qwen3-VL-8B) to 94.79 and Verus-Bench from 66.67 to 89.06, while increasing Artifact-Bench T1 from 50.80 to 66.60.
• Not uniformly best across all benchmarks: On ViF-Bench, Verus-Insight reaches 60.51 BA, below Skyra (63.81) and BusterX++ (61.40). On Artifact-Bench T1, Gemini-3.1-Pro reports the highest value in the table at 69.60, and on T2 Verus-Insight reports 45.60 versus Gemini-3.6-Flash at 62.00.
• Strong fine-grained artifact finding: On Verus-Bench, Plan F1 rises from 17.08 to 24.70 and Finding F1 from 7.56 to 15.97; on RewardData, Plan F1 rises from 18.67 to 31.33 and Finding F1 from 3.38 to 35.42.
• Gains are not from over-reporting artifacts: On RewardData, Finding precision and recall increase jointly from 1.87/18.13 to 33.66/37.36, and Figure 5(b) reports that Verus-Insight achieves the lowest strict artifact false-positive rate on real RewardData videos.
• Stages are distinct capabilities: On RewardData, Verus-Insight has Plan F1 comparable to Gemini 3.6 Flash (31.33 vs. 33.13) but much higher Finding F1 (35.42 vs. 25.54), while VideoVeritas reaches 91.12 BA with only 8.47 Finding F1 (Verus-Insight: 94.79 BA, 35.42 Finding F1).
• Better plan-to-evidence conversion: Conversion P(F=1 | P=1) improves from 45.16% to 54.88% on Verus-Bench and from 30.48% to 75.58% on RewardData, despite lower Plan recall than the backbone.
• Cross-dataset transfer: On the ViF-CoT-4K subset, Verus-Insight reports the highest Plan F1 (35.27) and Finding F1 of 14.02 versus 8.08 for VideoVeritas and 5.34 for BusterX++ (Skyra excluded due to training-data overlap).
• Robustness across artifact conditions: On Verus-Bench, Verus-Insight outperforms compared specialized detectors in Any-Hit rate and artifact coverage for videos with three or more annotated artifacts, and in recall across all temporal- and spatial-extent bins; on RewardData the coverage gains are not uniform across all extent bins.
• Design choices matter: In development studies, combining rubric warm-up with an explicit rubric gives 23.61 Plan F1 / 18.68 Finding F1 / 86.11 BA, versus 19.76 / 18.37 / 78.30 with neither; region tracking plus explicit grounding gives 20.28 / 18.25 / 83.16, the best configuration in that study.
• Ablations: Removing the Stage-II rubric drops Verus-Bench Plan/Finding F1 from 24.70/15.97 to 21.54/13.76 and RewardData from 31.33/35.42 to 26.57/29.35, while slightly improving BA (to 90.63 and 96.07) — evidence that better classification does not imply better artifact understanding. Removing RL gives the largest fine-grained drops (20.99/13.53 on Verus-Bench, 26.00/30.35 on RewardData).

Methodology in Plain English

The authors start from the observation that three different things can go wrong in an artifact detector: it may not look at the right content, it may look but misjudge what it sees, or it may reach the right verdict without recovering the real artifact. They therefore split the task into three explicitly separated steps and supervise each one.

Building supervision. Human annotators mark perceptible artifacts in AI-generated videos (collected from Kling-2.5, Veo 3, HunyuanVideo-1.5, LTX-2.3, and Wan2.2) and in real videos (from NExT-QA, OneThinker-STVG, DiDeMo, and the Internet), recording artifact type, description, temporal interval, and frame-wise boxes. Each artifact annotation is rewritten as an "outcome-neutral" rubric criterion — a statement of what to check that does not reveal whether something is wrong — paired with the original annotation as the verification target. To prevent the model from using "this criterion exists" as a shortcut for "an artifact exists," they add verified-normal criteria that independent verification confirmed are valid checks yielding normal observations. The final training trajectory is a label-neutral video description, the rubric, shuffled criterion-linked verification results, an aggregation summary, and the verdict. This produces 8,411 training instances (6,411 from Verus-Annotation, 2,000 from RewardData's training split), with 36,309 rubric criteria, 11,745 artifact checks, and 24,359 normal checks.

Training in three stages. Stage I warms up region tracking (predicting a box at queried timestamps from an initial box) and rubric planning at a 1:1 ratio. Stage II is full-process supervised fine-tuning on the assembled trajectories, so the model generates every intermediate output itself. Stage III applies GRPO reinforcement learning with a reward gated on response validity and verdict correctness: a correct, valid response earns a base reward of λ = 0.2 (set in the paper), plus (1−λ)·Q, where Q is Finding F1 for AI-generated videos and an indicator that no artifacts were predicted for real videos.

Evaluation. They score inspection coverage (Plan F1), artifact recovery (Finding F1), and video-level detection (Balanced Accuracy), using one-to-one semantic matching; unmatched predicted findings, including on real videos, count as false positives. Design studies are run on a 576-video Analysis-Dev set (288 real, 288 AI-generated, 545 ground-truth findings) held out from training and test data.

Why This Matters

Impact on research. The paper argues that verdict accuracy is an incomplete measure of detector quality, and provides both a decomposition and a benchmark for measuring the intermediate steps. Its ablation showing that removing the rubric raises BA while lowering Plan/Finding F1 is a concrete demonstration that classification metrics can move in the opposite direction from artifact understanding.

Real-world applications:

• Content authenticity verification and provenance checks on user-uploaded or syndicated video.
• Diagnosing what specifically went wrong in a generated video, which is useful for generation-quality evaluation and model debugging.
• Safety and moderation pipelines that need an auditable reason for flagging a video, not just a score.
• Dataset curation and benchmark construction for synthetic-media research.

Industry relevance. Because the approach is built on an 8B open-source backbone and reaches RewardData BA of 94.79 and Verus-Bench BA of 89.06, it is a plausible template for deployable detectors that must justify each decision to a reviewer.

Future Directions

• Closing the remaining artifact-recovery gap: Finding F1 on Verus-Bench is 15.97 in absolute terms, so most human-perceived artifacts are still not recovered; what supervision or reward would move this further is an open question the paper's own numbers raise.
• Extending coverage of artifact conditions: On RewardData the authors report that coverage improvements are not uniform across all temporal- and spatial-extent bins, leaving uneven performance across artifact types to address.
• Reconciling classification and artifact understanding: Since removing the Stage-II rubric improved BA while degrading Plan and Finding F1, understanding when and why video-level accuracy and evidence fidelity diverge is a natural next study.
• Scaling the benchmark and supervision: Verus-Bench contains 192 videos and 326 findings across 18 categories; whether these capability conclusions hold at larger scale and on newer generation models is not established here.

Target Audience

Researchers and engineers working on synthetic-media detection, video forensics, and explainable multimodal models; practitioners building content-authenticity or moderation systems who need artifact-level justification rather than a binary label; and MLLM researchers interested in decomposing a task into supervised sub-capabilities and training them with SFT plus reinforcement learning. Readers without background in MLLM training will find the conceptual framing accessible but the experimental sections dense.

Authors’ abstract

Recent advances in video generation have made synthetic videos increasingly realistic, placing greater demands on reliable AI-generated video detection. Existing artifact-based detectors typically identify generation artifacts as evidence for source prediction, yet they may either fail to recognize subtle artifacts or reach the correct source decision without faithfully identifying the evidence underlying it. In this work, we systematically investigate how to design and train effective artifact detectors and formulate artifact detection as a structured process comprising three complementary capabilities: Inspection Planning, which determines what should be examined; Evidence Verification, which verifies whether the inspected content exhibits genuine generation artifacts; and Evidence Aggregation, which integrates the verified evidence into the final source decision. Building on this formulation, we introduce Verus-Insight, trained with human-anchored full-process supervision and progressive capability training to strengthen both fine-grained artifact detection and video-level detection. We further introduce Verus-Bench, a benchmark with fine-grained human annotations of perceptible artifacts for evaluating artifact detection. Extensive experiments across diverse benchmarks demonstrate that Verus-Insight achieves strong performance in both fine-grained artifact analysis and AI-generated video detection, with substantial gains in recovering human-perceived artifacts. Further analyses reveal that the three capabilities are complementary yet distinct, and that Verus-Insight more reliably converts relevant inspection targets into supported artifact evidence without increasing spurious artifact claims.