Physics-Driven Spatiotemporal Modeling for AI-Generated Video Detection

AI summary

Overview

Research area: AI-generated video detection (deepfake / synthetic media forensics), combining generative modeling theory (diffusion models, score matching) with physics-inspired conservation laws and non-parametric distribution testing.

Technical level: Advanced. The paper derives a continuity equation for probability flow, defines a ratio statistic from spatial score functions and temporal derivatives, and proves a high-probability upper bound on NSG feature distances under Gaussian assumptions. Familiarity with diffusion models, score functions, MMD, and RKHS kernels is assumed.

Scope (1 sentence): The paper introduces a physics-driven statistic called the Normalized Spatiotemporal Gradient (NSG), estimates it with pre-trained diffusion models, and uses Maximum Mean Discrepancy between NSG features of test and real videos to detect AI-generated video, reporting gains of 16.00% in Recall and 10.75% in F1-score over baselines.

What This Paper Is About

AI-generated videos (for example Sora) now look nearly photorealistic, yet they often violate physical laws in subtle ways — implausible motion, inconsistent spatiotemporal coherence — that artifact-based detectors miss. The authors ask how the intrinsic spatiotemporal dynamics of natural video can be modeled so that these physics violations become detectable. Their answer is to treat video evolution as a probability flow governed by a conservation law, define a measurable ratio of spatial to temporal probability changes, and flag a video as fake when its NSG features deviate strongly from those of real videos.

Key Contributions

1. A physics-driven NSG statistic. The authors formulate video evolution through a probability flow velocity field satisfying a continuity equation, then introduce the Normalized Spatiotemporal Gradient — the ratio of spatial probability gradients to temporal density changes — as a dual field to the intractable velocity field, avoiding the ill-posed inversion while retaining spatiotemporal gradient information.

2. A diffusion-guided NSG estimator with physical priors. Spatial gradients are approximated by a pre-trained diffusion model's learned score function in a single forward pass (no numerical differentiation), and temporal derivatives are approximated using a brightness constancy assumption, avoiding explicit optical flow computation and complex motion decomposition.

3. An NSG-VD detection method with theoretical and empirical justification. NSG features across frames are compared to a reference set of real videos using Maximum Mean Discrepancy with a deep kernel trained via multi-population aware optimization. The authors derive an upper bound on the squared distance between NSG features of real and generated videos showing it grows with the distribution shift term.

Main Findings

• Strong average gains over the strongest baseline. Trained on 10,000 real videos from Kinetics-400 and 10,000 generated videos from Pika, NSG-VD achieves 16.00% higher average Recall and 10.75% higher F1-score than Demamba, and 55.05% higher F1-score than STIL.

• Best average scores across all reported metrics. In the Pika-trained comparison, NSG-VD reaches 88.02% Recall, 91.46% Accuracy, 90.87% F1, and 96.14% AUROC on average, versus Demamba at 72.02%, 84.21%, 80.12%, and 93.88%.

• Large margins on closed-source and emerging generators. On Sora, NSG-VD obtains 78.57% Recall against 48.21% for Demamba. On HotShot, it obtains 92.50% Recall against 40.60% for Demamba.

• Baselines collapse on physically implausible content. TALL drops to 25.00% Recall on Sora; STIL collapses to 1.40% Recall on HotShot and 1.79% on Sora; NPR's Accuracy ranges widely from 57.20% to 98.20% depending on the generator.

• Not all per-generator results favor NSG-VD. On MorphScope, NSG-VD's 68.33% Recall is below Demamba's 87.00%; on Crafter, 80.00% is below Demamba's 98.00%; on WildScrape, 82.50% is below Demamba's 58.20%… (the paper's table shows mixed per-generator Recall, with NSG-VD leading decisively on average).

• Theoretical result. Theorem 1 shows that, with probability at least 1 − δ, the squared NSG feature distance between real and generated videos is bounded by an order term in T / (C⁴ σ(t)²) involving the distribution shift φ = ‖μ‖²/σ(t)², the dimension d, and log(T/δ); the bound grows as φ grows, which implies MMD between real videos is smaller than MMD between real and generated videos.

• A second experimental table is referenced but its numbers are not present in the supplied content. The paper states Table 2 reports comparisons when training with Kinetics-400 and SEINE, but the table body is truncated in the provided text, so its values are not reported here.

Methodology in Plain English

The authors borrow an idea from fluid mechanics. Instead of thinking of a video as pixels, they think of it as a cloud of probability mass that drifts and changes over time. In physics, how much mass moves is described by a continuity equation: the rate at which density changes at a point equals the net flow into or out of it. The same equation holds for any time-evolving probability density, so the authors apply it to video.

Solving for the flow velocity itself is underdetermined — many velocity fields satisfy the equation. So the authors sidestep it. Assuming the divergence term is subdominant (an incompressible-flow-style approximation common in fluid dynamics and quantum mechanics), they normalize both sides and define the Normalized Spatiotemporal Gradient: the spatial gradient of the log probability density divided by the negative temporal derivative of the log density plus a small constant λ that prevents division by zero. Natural video dynamics preserve the product of the velocity field and this ratio at roughly 1; synthetic video, being generated by a shifted distribution, breaks that relationship.

To measure the two ingredients in practice, they use a pre-trained diffusion model. Diffusion models are trained to output the score — the gradient of the log probability density — so the spatial gradient comes from one forward pass of the score network, no numerical differentiation needed. The temporal derivative is approximated by assuming brightness constancy along motion trajectories, which relates the temporal change of log density to the spatial gradient dotted with the inter-frame displacement divided by the time step. This avoids computing optical flow explicitly.

NSG features are then collected for all T frames of each video. A test video is compared against a reference set of real videos using Maximum Mean Discrepancy with a Gaussian kernel, and the video is labeled Fake if the MMD exceeds a threshold τ. To sharpen discrimination, the kernel is a deep kernel combining a learned network over NSG features with a base Gaussian kernel, trained by maximizing a multi-population ratio of the kernel's discriminative power to its variance.

Why This Matters

Impact on research. The paper argues that detecting generated video should not depend on generator-specific artifacts, which age quickly as generators improve. By grounding detection in a conservation principle, it offers a generator-agnostic formulation with a supporting theoretical bound. It also connects score-based diffusion models to a new use — discrimination rather than synthesis — extending prior work that used score models for adversarial and generated-sample detection.

Real-world applications:

• Misinformation and deepfake screening on social platforms, where detection must generalize to generators the platform has never seen.
• Newsroom and broadcaster verification of user-submitted footage before publication.
• Legal and forensic evidence review, where an analyst needs a quantitative deviation score rather than a binary classifier trained on one generator.
• Content provenance and platform moderation pipelines that need a reference-based score compatible with a bank of known-real videos.

Industry relevance. The reported gains are concentrated on closed-source and newly emerging generators (Sora, HotShot), which is exactly the regime where artifact-based commercial detectors degrade. A method that trains on 10,000 real and 10,000 generated videos and still transfers to unseen generators is attractive to platforms that cannot retrain for every new model release. The code is released at the GitHub link given in the abstract.

Future Directions

1. Addressing per-generator weaknesses. NSG-VD trails Demamba on MorphScope and Crafter Recall in the reported table, suggesting the physics-based signal is not uniformly superior; understanding when conservation-based cues add value versus when appearance cues dominate is an open question.

2. Extending the theory beyond Gaussian assumptions. The upper bound in Theorem 1 is derived under Gaussian real and generated distributions with a mean shift; relaxing this to realistic video distributions is left open.

3. Evaluating on the SEINE-trained setting and beyond. Table 2 (Kinetics-400 + SEINE training) is referenced but its numbers are not in the supplied content, and the paper's protocol uses a fixed 10,000 real / 10,000 generated training set — scaling behavior is not reported.

4. Reducing reliance on reference real videos. Detection requires a reference set S_P of real videos and a tuned threshold τ; how performance degrades with small or domain-mismatched reference sets is not reported.

Target Audience

Researchers and graduate students working on AI-generated content detection, media forensics, and diffusion model theory, particularly those interested in physics-inspired inductive biases and non-parametric distribution tests. It is also relevant to practitioners building moderation or verification systems who need detectors that transfer to unseen generators, though the mathematical development makes it a challenging read for beginners.

Authors’ abstract

AI-generated videos have achieved near-perfect visual realism (e.g., Sora), urgently necessitating reliable detection mechanisms. However, detecting such videos faces significant challenges in modeling high-dimensional spatiotemporal dynamics and identifying subtle anomalies that violate physical laws. In this paper, we propose the first physics-driven AI-generated video detection paradigm based on probability flow conservation principles. Specifically, we propose a statistic called Normalized Spatiotemporal Gradient (NSG), which quantifies the ratio of spatial probability gradients to temporal density changes, explicitly capturing deviations from natural video dynamics. Leveraging pre-trained diffusion models, we develop an NSG estimator through spatial gradients approximation and motion-aware temporal modeling without complex motion decomposition while preserving physical constraints. Building on this, we propose an NSG-based video detection method (NSG-VD) that computes the Maximum Mean Discrepancy (MMD) between NSG features of the test and real videos as a detection metric. Last, we derive an upper bound of NSG feature distances between real and generated videos, proving that generated videos exhibit amplified discrepancies due to distributional shifts. Extensive experiments confirm that NSG-VD outperforms state-of-the-art baselines by 16.00% in Recall and 10.75% in F1-Score, validating the superior performance of NSG-VD. The source code is available at https://github.com/ZSHsh98/NSG-VD.