MEV: A Multi-Event Video Dataset for Long-Take Generation

AI summary

Overview

• Research area: Computer vision — specifically text-to-video generation, and the data/modeling infrastructure needed for long, single-shot videos that contain several distinct actions in sequence.
• Technical level: Intermediate. The dataset construction and statistics are readable by a general audience, but the experiments assume familiarity with diffusion video models, LoRA fine-tuning, rotary positional encoding, and VBench-style evaluation.
• Scope (one sentence): The paper introduces MEV, a curated 8K-video dataset of long-take, multi-event footage with timestamped event boundaries and hierarchical captions, plus an event-centric evaluation protocol (MEV-Eval) and a fine-tuned Wan2.1 baseline (MEV-Wan) that shows the data improves multi-event generation.

What This Paper Is About

Text-to-video datasets mostly contain short clips that depict one action, captioned loosely, so models get little supervision for keeping a story coherent across several actions inside a single continuous shot. The authors build MEV, a dataset of high-definition long-take videos that are cut into atomic, non-overlapping events with precise start/end timestamps and described at both the whole-video level and the individual-event level. They then show that fine-tuning a video generator on MEV with this temporally grounded conditioning produces smoother event transitions and more coherent long-take narratives than training on a comparable dataset or using existing long-video methods.

Key Contributions

1. The MEV dataset. More than 8k high-definition long-take videos (720p–4K), each containing between 2 and 21 distinct events, totaling over 25k event-level segments. Every event has precise start and end timestamps relative to the original video, plus captions and annotations. The authors describe it as the first public dataset to jointly provide structured, global–local hierarchical captions for long-take video generation.
2. A structured annotation scheme. Eight global structured fields (background, theme, style, shot type, camera movement, lighting, atmosphere, subject profile) plus three caption granularities of roughly 20/80/200 words, alongside event-level camera-aware descriptions (14 motion types plus static, and separate natural-language focus-shift descriptions), non-exclusive semantic attributes (appearance/disappearance, multi-person, animal, child, cooking, sports, travel), and VBench quality scores across six dimensions.
3. A four-stage curation pipeline. Video collection from MixKit and YouTube (227K raw clips), statistical filtering (to 87K clips), model-based purification (to 41K candidates), and rule-guided atomic event segmentation — three annotators labeled 52.9% of videos, and an event-aware Qwen3-VL-32B fine-tuned on that subset handled the remaining 47.1%, followed by manual verification.
4. MEV-Eval and a demonstration model. An event-centric evaluation protocol measuring Event Execution Accuracy (EEA), Event Order Accuracy (EOA), Temporal Grounding Accuracy (TGA) and Boundary Transition Coherence (BTC), together with MEV-Wan, a Wan2.1-1.3B model adapted with MinT-style global and temporal cross-attention plus ReRoPE and fine-tuned with LoRA on MEV.

Main Findings

• MEV-Wan leads on every event-centric metric. On MEV-Eval (Table 2, averaged over the same five random draws of 128 prompts): EEA 0.61, EOA 0.45, TGA 0.60, BTC 0.64 — versus Wan2.1-1.3B (0.42 / 0.20 / 0.33 / 0.41), LongLive (0.47 / 0.39 / 0.46 / 0.58) and Ultra-Wan (0.46 / 0.33 / 0.42 / 0.50).
• Human and VLM judges agree on the ranking. On a fixed subset of 128 test prompts, both human and Gemini 2.5 Pro evaluations rank MEV-Wan first on all four metrics.
• Generic video quality is preserved or improved. On 128 samples from the MEV test set averaged over five random draws (Table 3), MEV-Wan reaches subject consistency 0.960, background consistency 0.959, temporal flickering 0.990, motion smoothness 0.991, aesthetic quality 0.590 and imaging quality 64.924 — the best or joint-best on every column except temporal flickering and motion smoothness, where it ties the top value.
• VBench scores alone do not predict multi-event quality. Ultra-Wan stays competitive on VBench metrics but lags clearly on MEV-Eval, which the authors read as evidence for the value of MEV's curated videos and grounded temporal annotations over caption-inferred, uniformly spaced events.
• Dataset characteristics differ sharply from web-scale corpora. Table 1 lists MEV at 720p–4K with average global caption length 652.4 words and average event caption length 77.5 words, average global video length 16.7 s and average annotated event interval 5.3 s, across 8K videos / 25K clips — far denser supervision than Panda-70M (720p, 13.2 words, 8.5 s, 70.8M clips) or WebVid-10M (360p, 12.0 words, 17.5 s, 10M clips).
• Filtering is aggressive at every stage. The raw pool of 227K clips shrinks to 87K after statistical checks and to 41K candidates after model-based purification, using thresholds such as removing clips with on-screen overlay covering more than 3% of the frame in more than 12% of sampled frames.
• Model choice matters for annotation. Gemini 2.5 Pro reached 22% versus 10% exact-set accuracy against manual labels for camera-motion annotation compared with Qwen3-VL, which frequently predicted "static" for moving-camera clips; Gemini still confused some labels (e.g. truck-left vs. pan-left), so manual verification was retained.
• Qualitative failures in baselines. The paper reports Wan2.1 struggling from the first event onward, LongLive suffering severe content drift (a reflection introduced in the second key frame persisting), and Ultra-Wan misaligning after the third event (a box splitting into two) with video stagnation, while MEV-Wan generates all events with smooth transitions.

Methodology in Plain English

The authors start from a large pool of stock and YouTube footage chosen for real-world activity, long unbroken takes, and decent resolution. They clean it in two passes: fast frame-level checks that throw out clips with burned-in text or graphics, dark letterbox borders, badly exposed frames, or near-grayscale frames, then stronger models — a shot-transition detector to reject edited footage, optical flow to reject clips that are nearly static or jittery, and a vision-language model to reject scenes that are too crowded, too visually busy, or too repetitive.

The heart of the work is deciding what counts as an "event." Rather than using the loose definition common in video understanding (a stretch where the goal and scene stay stable), the authors adopt a process-driven one: an event is an action unit around one micro-intent, operating on one object or object set, and describable by a short procedural phrase. Boundaries can be triggered by a change of intent, a change of object, a move to a functionally different area, completion or failure of a task, or a salient pause or camera move. Three human annotators labeled the boundaries on a portion of the data with a half-second (δ = 0.5 s) agreement tolerance, and a vision-language model fine-tuned on that labeled subset extended the labeling to the rest, with manual verification.

Each video is then described at two levels. Globally, eight structured fields plus short, middle and long captions summarize the scene, lighting, style, shot type and subject. Locally, each event gets its own caption, a camera-motion label from 14 motion types plus static with a free-text description, focus-shift notes, binary semantic tags, and VBench quality scores. Gemini 2.5 Pro does the bulk of this annotation under a JSON schema, with manual auditing.

For the modeling experiment, they take Wan2.1-T2V-1.3B and add a global cross-attention branch for whole-video style and scene, a temporal cross-attention branch that binds each event caption to its time window, and ReRoPE to provide the temporal positional bias so events unfold in order rather than at hard cuts. Training uses contiguous groups of 2–5 events from the same source video, an 80/20 split held at the original-video level, 480×832 resolution, 81 frames, ReRoPE rescale length 8, learning rate 1×10⁻⁵, and LoRA adapters. The comparison model, Ultra-Wan, uses the same backbone and adaptation but is fine-tuned on an equally sized UltraVideo-Long subset with caption-inferred events and uniform boundaries — an ablation of the data quality itself. Evaluation uses the new four-metric MEV-Eval protocol alongside standard VBench dimensions.

Why This Matters

For research, the paper argues that the bottleneck in long-take multi-event video generation is not architecture but training signal: existing datasets lack explicit, reproducible event boundaries, and without them both divide-and-conquer and autoregressive approaches drift or produce illogical transitions. MEV supplies that signal as a publicly released resource and pairs it with an evaluation protocol that measures event structure directly, rather than relying on generic fidelity metrics that the authors show can rank a weaker multi-event model competitively.

Real-world applications:

• Advertising and product demos, where a single continuous shot must walk a viewer through distinct steps (open, use, finish) without cutting.
• Instructional and procedural content, such as cooking, fitness or DIY videos, which the authors explicitly target in their content collection and which map naturally onto the atomic event structure.
• Controllable content creation and editing, where structured per-event captions, camera labels and timestamps allow localized re-generation or re-timing of one segment of a long take.
• Pre-visualization and storyboarding, where a scripted sequence of actions must be rendered as one unbroken camera move.

For industry, the contribution is a bridge between academic-scale data and the large corporate video corpora that typically drive generative video systems: a compact, densely annotated dataset aimed at fine-tuning rather than pretraining, with per-clip quality scores that let practitioners slice subsets by quality profile and semantic attributes for targeted training or retrieval.

Future Directions

• Scaling and diversifying the source pool. The pipeline is explicitly designed to be scalable, and the paper frames MEV as a fine-tuning resource; extending coverage beyond the MixKit/YouTube mix and toward more long-tail activities is a natural next step.
• Replacing or reducing human labeling. 52.9% of videos were manually labeled to bootstrap the event-aware annotator for the rest; improving VLM temporal grounding — the authors note even advanced MLLMs collapse events or misplace boundaries — would remove that dependency and the manual verification burden.
• Better camera-motion and focus annotation. Reported confusion between similar motion labels (truck-left vs. pan-left) and the deliberately simplified 14-label taxonomy leave room for a richer, more reliable camera representation.
• Extending evaluation. MEV-Eval covers execution, order, grounding and transition coherence; whether it generalizes to longer horizons than the events counted here, and how it correlates with long-form human preference over minutes rather than the tens-of-seconds durations reported, remain open.

Target Audience

Researchers and engineers working on text-to-video generation, long-form or multi-event video synthesis, and video–text dataset construction. It is most useful to those who need a fine-tuning dataset for temporally controlled generation or an evaluation protocol for event-structured video, and to practitioners comparing data curation strategies — the MEV-Wan versus Ultra-Wan comparison under an identical backbone and adaptation is the paper's most directly transferable lesson. Readers unfamiliar with diffusion video models will still follow the dataset sections but will find the experimental setup and appendices harder going.

Authors’ abstract

Current datasets for text-to-video generation are largely limited to short clips depicting a single event or coarse-grained captions. As a result, they provide limited supervision for modeling long-take temporal dependencies, often leading to temporally inconsistent generations when describing multiple events. In this paper, we introduce MEV, a multi-event video dataset designed for long-take video generation. MEV provides high-quality real-world videos where subjects can perform a sequence of distinct, non-overlapping events, featuring precise temporal boundaries and fine-grained procedural annotations. This event-centric discretization reduces confounding factors and enables models to support temporally localized conditioning and build long-take videos from atomic event units. Furthermore, we introduce a comprehensive annotation pipeline that segments videos into temporally grounded events, and generates structured captions at both the global level and the event level. In addition, under an event-centric evaluation protocol, we conduct multi-event generation experiments by fine-tuning on MEV with temporally grounded event conditioning, leading to improved long-take event transitions and more coherent content. We believe this work provides a publicly available resource for multi-event fine-tuning and event-centric evaluation, together with practical insights into learning from temporally structured video-text data.