VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree

AI summary

Overview

• Research area: Computer Vision / Video Anomaly Detection (VAD), specifically explainable, training-free anomaly detection using pre-trained vision-language models.
• Technical level: Advanced.
• Scope: The paper proposes VADTree, a training-free framework that builds a Hierarchical Granularity-aware Tree (HGTree) from pre-trained Generic Event Boundary Detection (GEBD) knowledge to adaptively sample video segments of varying temporal spans, then uses VLMs and LLMs for node-wise anomaly scoring, refinement, and multi-granularity score fusion.

What This Paper Is About

Most video anomaly detection systems need large amounts of in-domain training data and cannot explain why something is anomalous. Training-free approaches use pre-trained large models instead, but they typically slice video into fixed-length temporal windows, which do not match anomalies whose durations vary widely. VADTree addresses this by decomposing a video into generic event nodes using a pre-trained GEBD model, organizing them into a coarse-fine tree, and reasoning about anomalies node by node with VLMs and LLMs.

Key Contributions

1. A training-free generic event-centric VAD framework (VADTree) that leverages pre-trained GEBD knowledge to localize anomalous events in temporal position, avoiding the inefficiency and roughness of dense fixed-window sampling while providing multi-granularity perception and reasoning.
2. A Hierarchical Granularity-aware Tree (HGTree) that uses coarse-fine representation of videos based on potential generic event boundaries, built through adaptive coarse-fine hierarchical structuring and redundancy removal.
3. An event-centric anomaly scoring and refining approach that derives generic event anomaly scores from tree nodes by integrating multi-dimensional prior information (event scene, specific characters/objects, actions/behaviors) and multi-granularity scores.
4. Empirical validation showing state-of-the-art performance among training-free, unsupervised, and one-class methods on UCF-Crime and XD-Violence, and even surpassing some weakly supervised methods on the MSAD dataset.

Main Findings

• Sampling quality: Compared with a 10-second fixed-length sliding temporal window, which achieves mIoU = 0.51 on UCF-Crime and mIoU = 0.44 on XD-Violence, the granularity-aware tree achieves mIoU = 0.52 on UCF-Crime and mIoU = 0.64 on XD-Violence, giving higher IoU for events ranging from 3 seconds to 630 seconds.
• UCF-Crime performance: VADTree reaches 84.74 AUC (%) under training-free supervision, exceeding LAVAD by 4.5% and surpassing EventVAD by 2.7%, and outperforming all one-class and unsupervised methods listed.
• XD-Violence performance: VADTree (training-free) reaches 90.44 AUC (%) and 67.82 AP (%), an AUC ROC 5.1% higher than LAVAD and a 2.9% gain over EventVAD. An audio-augmented variant, VADTree* (using Kimi-Audio-7B-Instruct to extract audio captions), reaches 68.85 AP and 90.55 AUC.
• MSAD performance: VADTree achieves 89.32 AUC (%), 67.85 AUC_a (%), 71.41 AP (%), and 75.49 AP_a (%) without any dataset-specific training, outperforming weakly supervised methods such as GS-MoE (87.72 AUC, 69.54 AUC_a, 68.26 AP, 76.68 AP_a) on the overall metrics, while π-VAD (88.68 AUC, 71.25 AUC_a, 71.26 AP, 77.86 AP_a) retains a slight advantage on the anomaly-specific AUC_a and AP_a metrics.
• Component ablation on UCF-Crime: Starting from the HGTree fine cluster baseline at 71.57 AUC (%), adding prior-infused node scoring raises it to 75.67, adding intra-cluster node refinement raises it to 83.05, and adding inter-cluster node correlation yields 84.74.
• Tree construction settings: With only a fine cluster, AUC is 80.89 at γ_min = 0.3, 82.81 at γ_min = 0.4, and 80.85 at γ_min = 0.5. With K-Means coarse + fine clusters, AUC is 83.74 at 0.3, 84.74 at 0.4, and 82.40 at 0.5. Replacing K-Means with K-Medoids at γ_min = 0.4 produces an additional gain, reaching 85.24. Using only leaf nodes collapses HGTree to a single cluster structure and yields a 1.9% lower AUC ROC.
• Model configuration sensitivity: On UCF-Crime, LLaVA-NeXT-Video-7B with DeepSeek-R1-Distill-Qwen-14B gives 84.74 AUC (%), InternVL2_5-8B with DeepSeek-R1-Distill-Qwen-14B gives 83.74, LLaVA-NeXT-Video-7B with t5gemma-9B-2B gives 84.00, and InternVL2_5-8B with t5gemma-9B-2B gives 83.56.
• Efficiency claim: The paper states that VADTree drastically reduces the number of sampled video segments, though specific segment-count or runtime figures are not reported in the provided content.

Methodology in Plain English

The method works in three stages.

Stage 1 — Building the tree. A pre-trained GEBD model looks for "generic event boundaries" (points where one event ends and another begins) in the video. Because GEBD models only handle short clips, the video is split into overlapping windows, each processed separately, and only the central portion of each window's confidence sequence is kept to avoid edge artifacts. The partial sequences are concatenated into one global confidence sequence, and local maxima in that sequence mark candidate boundaries. A recursive TreeInit procedure then splits the video at the most confident boundary, then splits each resulting piece again, and so on, stopping when candidate boundaries run out or confidence falls below a threshold γ_min. The result is a binary tree whose internal nodes represent segmentation decisions and whose leaves correspond to atomic events. The tree is then split into a coarse cluster and a fine cluster using K-means (with 2 clusters) over the boundary confidence values. A RemoveDup step keeps only the finest-grained nodes along each tree path, and a Complete step replicates nodes that cannot be split further so both clusters still cover the entire video.

Stage 2 — Scoring each node. An LLM generates multi-dimensional priors along three axes: event scene, specific characters/objects, and actions/behaviors. The paper deliberately excludes two kinds of cues: micro-expressions (which would require prohibitively high image resolution) and audio-dependent triggers (unavailable in visual-only surveillance). These priors are injected into a VLM prompt, and the VLM captions the frames sampled at each tree node. A second LLM then assigns each node a discrete anomaly score from {0, 0.1, ..., 1}, where 0 is normal and 1 is anomalous.

Stage 3 — Refining and fusing scores. Because a single node's score ignores long-term context and can be tripped up by local changes, the initial score is refined using semantically similar nodes within the same cluster. Similarity is measured by cosine similarity of features from a pre-trained vision encoder (ImageBind), the top-K most similar nodes are selected, and their scores are combined with softmax weights. Finally, a cohesion-driven fusion combines coarse-cluster parent nodes with fine-cluster child nodes: the variance of the child scores within each parent (the intra-cluster cohesion) determines how much weight the parent's score receives versus the children's. A control coefficient β ∈ [−1, 1] shifts the baseline 0.5 fusion weight. Low variance means the children agree and the parent dominates; high variance means conflicting evidence and the finer child semantics are trusted more.

Why This Matters

• Research impact: The paper argues that fixed-length temporal window sampling — the default in training-free explainable VAD — is fundamentally mismatched to anomalies of varying duration. It shows that borrowing pre-trained GEBD structure is a viable replacement, and that hierarchical granularity plus score fusion can beat fixed-window training-free baselines and even some weakly supervised methods without any training.
• Real-world applications:
  • Video surveillance: detecting and temporally locating events such as traffic accidents, explosions, burglaries, and arrests, with textual explanations for each decision.
  • Autonomous driving: the paper cites autonomous driving as a motivating application domain for temporally locating unexpected events.
  • Industrial manufacturing: the paper cites industrial manufacturing as another application domain for anomaly detection in video.
  • Interactive/explainable monitoring systems: because the framework uses LLMs and VLMs, operators can query and receive semantic explanations rather than only scalar anomaly scores.
• Industry relevance: A training-free pipeline lowers the barrier to deployment, since no in-domain labeled video is needed and no parameters are fine-tuned. The paper reports that this generalization advantage shows up most clearly on MSAD, where VADTree beats weakly supervised methods that were trained on extensive data.

Future Directions

1. Extending beyond the two-cluster tree. The paper notes it deliberately uses a two-cluster (coarse + fine) granularity because marginal gains diminish with excessive layers and inference overhead grows, leaving open whether additional layers help.
2. More robust boundary clustering. The K-Medoids result (85.24 AUC at γ_min = 0.4 versus 84.74 for K-Means) suggests that more robust centroid selection around outliers in generic event boundaries is a promising direction.
3. Multimodal extension. The audio-augmented VADTree* variant using Kimi-Audio-7B-Instruct gives 68.85 AP and 90.55 AUC on XD-Violence, indicating that adding modalities beyond vision is a clear line of further work.
4. Verifying robustness of event-aware methods. The paper points out that the generalization of existing event-aware methods such as EventVAD has not been verified and its robustness in identifying complex boundaries is limited, which frames boundary robustness as an open problem.

Target Audience

Researchers and practitioners in computer vision and video understanding who work on anomaly detection, video-language models, or long-video temporal reasoning; engineers building surveillance or monitoring systems who need detection without in-domain training data; and readers interested in explainable, LLM/VLM-driven video analytics rather than conventional supervised detection pipelines.

Authors’ abstract

Video anomaly detection (VAD) focuses on identifying anomalies in videos. Supervised methods demand substantial in-domain training data and fail to deliver clear explanations for anomalies. In contrast, training-free methods leverage the knowledge reserves and language interactivity of large pre-trained models to detect anomalies. However, the current fixed-length temporal window sampling approaches struggle to accurately capture anomalies with varying temporal spans. Therefore, we propose VADTree that utilizes a Hierarchical Granularityaware Tree (HGTree) structure for flexible sampling in VAD. VADTree leverages the knowledge embedded in a pre-trained Generic Event Boundary Detection (GEBD) model to characterize potential anomaly event boundaries. Specifically, VADTree decomposes the video into generic event nodes based on boundary confidence, and performs adaptive coarse-fine hierarchical structuring and redundancy removal to construct the HGTree. Then, the multi-dimensional priors are injected into the visual language models (VLMs) to enhance the node-wise anomaly perception, and anomaly reasoning for generic event nodes is achieved via large language models (LLMs). Finally, an inter-cluster node correlation method is used to integrate the multi-granularity anomaly scores. Extensive experiments on three challenging datasets demonstrate that VADTree achieves state-of-the-art performance in training-free settings while drastically reducing the number of sampled video segments. The code will be available at https://github.com/wenlongli10/VADTree.