TSBOW -- Traffic Surveillance Benchmark for Occluded Vehicles Under Various Weather Conditions

AI summary

Overview

Research area: Computer vision, specifically object detection for intelligent transportation systems and traffic surveillance.

Technical level: Intermediate. The paper is a benchmark/dataset contribution; understanding it requires basic familiarity with object detection metrics (mAP, IoU, precision/recall) but no deep architectural knowledge.

Scope: The paper introduces TSBOW, a large-scale CCTV traffic surveillance dataset and detection benchmark built to stress-test vehicle detection under heavy occlusion, extreme weather, and dense urban traffic.

What This Paper Is About

Existing traffic surveillance datasets cover only mild conditions — light rain, light snow, or haze — and largely ignore the severe weather that climate change is making more frequent. The authors argue that this gap leaves detection models untested exactly where they matter most: when signal quality collapses, roads congest, and accident rates rise. TSBOW is their answer, a year-long, multi-season CCTV dataset from Suwon, South Korea, with explicit labels for weather, scenario, road type, camera scale, and occlusion level, plus a set of detection baselines to measure how badly current models struggle.

Key Contributions

1. A semi-automatic iterative annotation pipeline that combines manual labeling of a curated subset with model-generated annotations (YOLOv12x) for the remaining frames, followed by verification and post-processing — enabling labeling at a scale that pure manual annotation could not reach.

2. The TSBOW dataset itself: 198 CCTV videos, 32.36 hours, over 3.2 million extracted frames across 145 regions of interest, spanning four seasons and four weather categories (normal, haze, rain, snow), with 48,061 manually annotated and verified frames.

3. A challenging, occlusion-focused label set: approximately 71.1 million bounding boxes across eight classes (car, bus, truck, small truck, micromobility, pedestrian, unidentified, others), with occlusion explicitly graded into none, light, and heavy tiers, and with a deliberately balanced class distribution rather than the car-dominated distributions of prior benchmarks.

4. Detection baselines and cross-dataset comparison: YOLOv8x, YOLO11x, YOLOv12x, and RT-DETR-x are trained and evaluated on TSBOW, and a matched comparison set shows models trained on TSBOW outperform those trained on UAVDT and UA-DETRAC at detecting cars in CCTV footage.

Main Findings

• YOLOv12x is the strongest general detector on TSBOW: 0.806 precision, 0.662 recall, 0.744 mAP50, and 0.615 mAP50-95. RT-DETR-x achieves the highest recall (0.740) but noticeably weaker localization (0.552 mAP50-95), producing more false positives.

• Training data source matters more than architecture: on an identical medium-scale comparison set, YOLOv12x trained on TSBOW reached 0.846 mAP50 and 0.869 recall for the car class, versus 0.383 mAP50 for the UAVDT-trained model and 0.558 for the UA-DETRAC-trained model. Aerial drone footage transfers poorly to ground-level CCTV, and UA-DETRAC's model overfits to clear, close vehicles and misses distant ones.

• Disaster scenarios are the hardest condition: the heavily hazed and heavily snowed subset scored the lowest of all scenarios (0.656 mAP50, 0.510 mAP50-95). Snow-covered vehicles blend into white backgrounds, and distant vehicles in heavy haze reduce to indistinct light sources.

• Boulevards and coarse camera scales hurt accuracy: wide, high-density boulevards scored lowest among road types (0.702 mAP50), and coarse-scale views were the weakest scale category (0.733 mAP50), both driven by small, heavily overlapped objects.

• Per-class performance is highly uneven: cars (0.959 mAP50) and buses (0.951) are detected reliably, but the "unidentified" class collapses to 0.317 mAP50, and pedestrians (0.715) and micromobility devices (0.726) lag well behind — reflecting both small object size and the deliberate inclusion of ambiguous, weather-corrupted instances.

• The "unidentified" class reveals a safety-relevant failure mode: in severe snow, models frequently classify degraded vehicle images as background rather than as objects, which can make a heavily trafficked road appear empty to a monitoring operator.

• Occlusion is pervasive and quantified: of the manually labeled instances, 721,684 have no occlusion, 266,420 are lightly occluded, and 143,051 are heavily occluded. Detectors frequently merge two or three overlapping vehicles into a single detection, generating systematic missed detections.

Methodology in Plain English

The authors recorded fixed CCTV routes in Suwon, South Korea, over a full year, deliberately sampling across seasons and weather so that heavy haze and snowfall are represented alongside ordinary sunny and cloudy conditions. Each recording was tagged by scenario (road, intersection, special case, or disaster), by road type (two-lane urban, four-lane standard, or six-plus-lane boulevard), and by camera scale (fine, medium, or coarse, depending on the camera's height and angle).

Annotation proceeded in five stages. First, regions of interest were defined so that only the road area with visible traffic is labeled. Second, trained human annotators drew bounding boxes on a subset of frames using an open-source labeling tool. Third, a YOLOv12x model fine-tuned on Korean vehicle characteristics was used to auto-label the much larger remaining pool of frames. Fourth, annotations were reviewed against strict criteria — tight boxes, occluded objects still labeled as visible, traffic signs and lights overlapping vehicles labeled as "others." Fifth, the results were aggregated and post-processed. License plates and pedestrian faces were blurred for privacy.

For the benchmark, videos were split by time within each clip: the first five minutes for testing, the next two for validation, and the final three for training. Four detectors (YOLOv8x, YOLO11x, YOLOv12x, RT-DETR-x) were pretrained on COCO and fine-tuned at 1280-pixel resolution for 100 epochs, then evaluated at a confidence of 0.5 and an IoU threshold of 0.6. A separate comparison subset of medium-scale scenes — drawing on both Suwon and Seoul footage — was built to test how models trained on TSBOW versus UAVDT versus UA-DETRAC perform on the same data.

Why This Matters

Object detection benchmarks shape what the field optimizes for, and prior traffic datasets effectively told researchers that clear-weather, low-occlusion detection was the problem worth solving. TSBOW reframes the target around the conditions that actually disrupt traffic: extreme snow, heavy haze, congestion, and the partial visibility that comes with dense urban scenes.

Real-world applications:

• Municipal traffic monitoring: cities can evaluate whether off-the-shelf detectors remain reliable on their existing CCTV infrastructure during storms, rather than discovering failures during emergencies.

• Incident and congestion response: the "unidentified" class failure mode — where snowy vehicles vanish from detections — directly threatens a traffic authority's ability to detect a blocked road or a crash under bad conditions.

• Edge-device deployment: the paper is funded by a project on 24/7 real-time traffic surveillance on edge hardware, so the benchmark's precision/recall trade-offs are directly relevant to deciding which detector to deploy on constrained devices.

• Multi-task traffic analytics: beyond detection, the dataset's density and class granularity support crowd counting, speed estimation, and object tracking, which underpin infrastructure planning.

Industry relevance: autonomous driving and ADAS teams benefit from a hard negative set of weather-degraded, heavily occluded scenes that on-vehicle datasets underrepresent. Smart-city vendors and traffic-management system integrators gain a public, reproducible yardstick for comparing detector claims. The dataset is released under CC BY-NC-ND 4.0 at https://github.com/SKKUAutoLab/TSBOW, which supports academic use but restricts commercial reuse.

Future Directions

• Nighttime coverage: the first release is deliberately daytime-only; the authors state that nighttime ground truth will be added in later versions, which will introduce headlight glare, low-light noise, and different occlusion patterns.

• Extended task support: multi-object tracking, semantic segmentation, vehicle counting, and speed estimation are named as planned additions, each requiring different annotation types and temporal consistency checks.

• Closing the manual-to-semi-automatic annotation gap: only 48,061 of over 3.2 million frames are human-verified, so a key open question is how much performance depends on the auto-labeled majority and how annotation noise propagates into model behavior.

• Downstream evaluation on degraded inputs: whether detection performance can be recovered through image restoration, weather-specific augmentation, or occlusion-aware architectures — rather than through more training data alone — remains untested on this benchmark.

Target Audience

Computer vision researchers working on object detection, robust perception, and dataset construction will find the benchmark design and ablation breakdowns most useful. Intelligent transportation and smart-city engineers evaluating detector reliability on real CCTV feeds are the primary applied audience. Researchers studying occlusion handling and adverse-weather robustness will benefit from the graded occlusion labels and the explicit disaster-scenario split. The paper assumes comfort with detection metrics and standard detector families, so readers without that background should treat the tables as the main takeaway rather than the architecture discussion.

Authors’ abstract

Global warming has intensified the frequency and severity of extreme weather events, which degrade CCTV signal and video quality while disrupting traffic flow, thereby increasing traffic accident rates. Existing datasets, often limited to light haze, rain, and snow, fail to capture extreme weather conditions. To address this gap, this study introduces the Traffic Surveillance Benchmark for Occluded vehicles under various Weather conditions (TSBOW), a comprehensive dataset designed to enhance occluded vehicle detection across diverse annual weather scenarios. Comprising over 32 hours of real-world traffic data from densely populated urban areas, TSBOW includes more than 48,000 manually annotated and 3.2 million semi-labeled frames; bounding boxes spanning eight traffic participant classes from large vehicles to micromobility devices and pedestrians. We establish an object detection benchmark for TSBOW, highlighting challenges posed by occlusions and adverse weather. With its varied road types, scales, and viewpoints, TSBOW serves as a critical resource for advancing Intelligent Transportation Systems. Our findings underscore the potential of CCTV-based traffic monitoring, pave the way for new research and applications. The TSBOW dataset is publicly available at: https://github.com/SKKUAutoLab/TSBOW.