LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preference

AI summary

Overview

Research area: Computer vision / generative AI — specifically evaluating whether video diffusion models (VDMs) implicitly learn physical laws.

Technical level: Intermediate. Readers should be comfortable with diffusion models, denoising objectives, and likelihood/ELBO concepts, though the paper's core idea is intuitive.

Scope: The paper proposes LikePhys, a training-free method to measure intuitive physics understanding in video diffusion models by comparing denoising losses on controlled valid–invalid video pairs, and uses it to benchmark 12 state-of-the-art models.

What This Paper Is About

Video diffusion models generate visually impressive clips but often produce physically impossible motion. The central problem is that existing evaluators struggle to separate physical correctness from visual appearance, rely on biased VLM judgments, or require image-conditioned generation that does not extend cleanly to text-to-video models. LikePhys solves this by asking a simple question: does a diffusion model assign higher likelihood to videos that obey physics laws than to visually matched videos that violate them?

Key Contributions

1. A training-free evaluation method (LikePhys) that uses the diffusion denoising loss as an ELBO-based likelihood surrogate to compare physically valid and invalid video pairs — no fine-tuning, no VLM judge, no pixel alignment required.
2. A curated synthetic benchmark of 12 physics scenarios spanning four domains (Rigid Body Mechanics, Continuum Mechanics, Fluid Mechanics, Optical Effects), where valid and invalid videos share identical appearance except for a single controlled physics violation.
3. The Plausibility Preference Error (PPE) metric — an aggregate score of how often a model wrongly prefers an invalid video over a valid one — validated against human annotators with Kendall's τ = 0.44, outperforming VideoPhy, VideoPhy2, and Qwen 2.5 VL baselines.
4. A systematic study of 12 pre-trained VDMs examining how architecture, model size, training data, frame count, and classifier-free guidance affect intuitive physics understanding, plus a breakdown of which physics laws models handle well or poorly.

Main Findings

• Architecture matters a lot: Early UNet-based models (AnimateDiff, ZeroScope) show PPE above 50%, while newer DiT-based models — Hunyuan T2V (43.6%), Wan2.1-T2V-14B (43.8%), CogVideoX1.5-5B (43.8%) — lead the ranking. Only a few models significantly beat the 50% random-guess threshold.
• PPE aligns with human judgment: LikePhys achieves the strongest overall correlation with human physics-plausibility ratings among automatic evaluators, despite being zero-shot and training-free.
• PPE is independent of visual quality: Pearson correlations with VBench aesthetic quality, subject consistency, and background consistency are near zero. Motion smoothness (r = 0.15) and temporal flickering (r = 0.12) show weak ties, confirming PPE targets physics rather than appearance.
• Scaling helps: Larger models and larger training corpora both reduce PPE, with model size showing a stronger correlation than data size. Longer context windows (more frames) consistently improve physics understanding.
• Guidance strength is largely irrelevant: Varying classifier-free guidance has little impact, suggesting physics understanding is baked into the learned distribution rather than adjustable at inference time.
• Domain-specific weaknesses: Fluid Mechanics is hardest (river flow exceeds 70% PPE; faucet/droplet 30–40%), while Optical Effects are easiest. At the law level, temporal continuity and energy/momentum conservation are poorly handled, whereas geometric invariance and optical consistency are well captured.
• Prompt robustness: Eight prompt variants per scenario produce no significant change in discriminative performance, because the controlled data isolates physics as the only variable.
• Timestep sampling matters: The valid–invalid separation peaks at different timesteps for different models and scenarios, so the authors uniformly sample 10 timesteps for a stable, unbiased estimate.

Methodology in Plain English

The researchers start from the violation-of-expectation idea from developmental psychology: a model that understands physics should be surprised by impossible events. They render pairs of videos in Blender — one physically valid, one with a single deliberate violation (e.g., a ball bouncing with impossible energy, a shadow moving the wrong way) — while keeping camera, lighting, textures, and objects identical.

For each video, they add Gaussian noise at several diffusion timesteps and let the model predict the noise. The average prediction error serves as a proxy for negative log-likelihood: lower loss means the model "expects" that video more. If a model correctly understands physics, the valid video should have lower loss than its invalid twin. PPE simply counts how often the model gets this backwards, averaged across all pairs and variations. A lower PPE means better physics understanding.

Why This Matters

Impact on research: LikePhys offers a model-agnostic, appearance-agnostic evaluation that decouples physics from visual quality — a persistent confound in VDM benchmarking. It provides a reproducible protocol for tracking progress as models scale, and its synthetic benchmark gives controlled, per-law diagnostics that natural video datasets cannot.

Real-world applications:

• World models for robotics: Physically plausible video prediction is a prerequisite for simulators used in planning and control.
• Autonomous driving simulation: Generating realistic traffic and collision dynamics requires faithful physics, not just photorealism.
• Content creation and VFX: Filmmakers and game studios need generators that produce believable dynamics without manual correction.
• Model selection and monitoring: Teams can use PPE during training to select checkpoints and during release to rank candidate models.

Industry relevance: As video generators become commercial products, buyers and regulators will need objective physics-accuracy measures. LikePhys gives an open-source-friendly scoring method that avoids the subjectivity and cost of human or VLM evaluation pipelines.

Future Directions

• Extending the benchmark beyond simulation: The current dataset requires hand-crafted Blender scenarios; scaling to broader physics phenomena (deformables, multi-body chaos, contact-rich scenes) needs new curation pipelines or hybrid simulator/editing approaches.
• Evaluating closed-source models: LikePhys requires access to noise prediction, which is unavailable for API-only systems like Sora or Veo. Developing black-box variants (e.g., using generated-frame likelihoods) is an open problem.
• Physics-aware training objectives: The paper shows models struggle with global conservation and temporal continuity; future work could add explicit conservation or long-range coupling losses during training.
• Longer context and memory: Since frame count improves PPE, investigating multiscale memory and longer temporal windows specifically for chaotic or fluid dynamics is a natural next step.
• Unifying with other benchmarks: Testing LikePhys against IntPhys, Physics-IQ, and Morpheus would clarify how well likelihood preference correlates with alternative physics probes.

Target Audience

Researchers and engineers working on video generation, world models, and diffusion-based generative systems; benchmark designers evaluating physical plausibility; robotics and autonomous driving teams using generative simulators; and graduate students studying intuitive physics in AI. Beginners can follow the core idea — "does the model think impossible videos are less likely?" — but the metric derivation and diffusion machinery assume intermediate familiarity with generative modeling.

Authors’ abstract

Intuitive physics understanding in video diffusion models plays an essential role in building general-purpose physically plausible world simulators, yet accurately evaluating such capacity remains a challenging task due to the difficulty in disentangling physics correctness from visual appearance in generation. To the end, we introduce LikePhys, a training-free method that evaluates intuitive physics in video diffusion models by distinguishing physically valid and impossible videos using the denoising objective as an ELBO-based likelihood surrogate on a curated dataset of valid-invalid pairs. By testing on our constructed benchmark of twelve scenarios spanning over four physics domains, we show that our evaluation metric, Plausibility Preference Error (PPE), demonstrates strong alignment with human preference, outperforming state-of-the-art evaluator baselines. We then systematically benchmark intuitive physics understanding in current video diffusion models. Our study further analyses how model design and inference settings affect intuitive physics understanding and highlights domain-specific capacity variations across physical laws. Empirical results show that, despite current models struggling with complex and chaotic dynamics, there is a clear trend of improvement in physics understanding as model capacity and inference settings scale.