Benchmarking Nighttime Traffic Sign Recognition with Illumination-Adaptive Detection and Semantic Attribute Reasoning

AI summary

Overview

Research area: Computer vision, specifically nighttime traffic sign recognition (TSR) — low-light object detection, fine-grained classification, dataset construction, and vision-language modeling.

Technical level: Intermediate. The paper is a benchmark/dataset contribution with a reference baseline model; it assumes familiarity with object detectors (YOLO family, Faster R-CNN), CLIP-style vision-language models, and standard detection/classification metrics, but the core ideas are explained clearly.

Scope: The paper introduces INTSD, a large-scale real-world Indian nighttime traffic sign dataset, benchmarks a wide range of detectors and classifiers on it under a standardized protocol, and proposes LENS-Net, an illumination-aware detect-then-classify baseline.

What This Paper Is About

Traffic sign recognition systems are overwhelmingly trained and tested on daytime imagery, but real driving happens at night, where headlight glare, motion blur, sensor noise, and underexposure degrade images and cause models to fail. The authors argue that existing benchmarks also omit "distractor" categories that TSR models frequently confuse — such as vandalized signs, unidentifiable signs, and large advertisement boards — and that generative daytime-to-nighttime translation produces unrealistic artifacts rather than a genuine night-time test.

To address this, the paper presents INTSD: 11,016 images spanning 41 traffic-sign and distractor classes, collected across six districts in India under varied sensors, weather, and illumination, with 14,044 fully annotated signboard instances. Alongside the dataset, the authors benchmark state-of-the-art models and offer LENS-Net, an illumination-adaptive detector paired with a multimodal classifier, as a competitive reference baseline.

Key Contributions

1. INTSD dataset. The authors describe INTSD as the first large-scale, diverse Indian nighttime traffic sign dataset: 11k+ images and 14k+ fully annotated instances across 41 traffic-sign and distractor classes, captured under varied sensors, weather, and illumination, with attribute-level annotations (image quality, weather, and per-image degradation attributes such as low illumination, blur, and glare). Four additional distractor classes are introduced — tampered, unknown, advertisement, and warning — which the authors state are commonly confused by TSR models and absent from existing datasets.

2. A standardized benchmark. Detection and classification are evaluated across a broad set of contemporary models under a unified protocol on five stratified folds, using mAP@50, mAP@50:95, macro-averaged precision (A.P.), macro-averaged recall (A.R.), accuracy (Acc.), and frames per second (FPS).

3. A cross-domain utility analysis. Using an 8-class shared taxonomy (the intersection of INTSD and the daytime data of Rishabh et al.), the authors compare three training regimes — Day (daytime only), Joint (day plus a size-matched INTSD subset), and INTSD (nighttime only) — to show that daytime-trained models fail at night and that INTSD recovers performance, even when data volume is controlled.

4. LENS-Net reference baseline. A detect-then-classify system combining an end-to-end, per-image adaptive illumination-enhancement module with a classifier that fuses frozen CLIP vision-language representations with soft semantic attribute reasoning over learnable shape and color embeddings. The authors present it explicitly as a baseline rather than the paper's central contribution.

Main Findings

• Daytime-only training collapses at night. On the 8-class shared taxonomy, YOLO-TS trained on daytime data reaches only 5.31% mAP@50 and 1.59% mAP@50:95, and LENS-Net reaches 5.87% and 2.55%. Classification macro-precision in the day-only regime ranges from 54.01% (YOLO-TS) to 71.43% (LENS-Net), consistent with the abstract's "54–71% macro-precision" statement.

• Adding INTSD recovers performance. With the Joint (day + size-matched INTSD) or INTSD-only regimes, scores return to the 90%+ range. YOLO-TS reaches 91.82% mAP@50 (Joint) and 92.06% (INTSD-only), with classification accuracy of 93.92% and 96.85% respectively. This recovery persists even when the injected nighttime data is subsampled to match the daytime set in size, isolating the nighttime signal from data volume.

• LENS-Net leads full-scale detection on INTSD. LENS-Net attains the highest mAP@50 (94.46%) and mAP@50:95 (81.56%) among all detectors in Table 2, ahead of YOLO-TS (90.82 / 77.27), YOLOv8 (90.13 / 77.21), YOLOv10-L (90.00 / 75.81), YOLO-LLTS (90.00 / 73.65), GOLD-YOLO-L (88.70 / 72.11), DAI-Net (87.18 / 62.56), YOLOv12-L (82.84 / 64.93), and Faster R-CNN (62.60 / 43.22). It runs at 78.11 FPS with 11.18M parameters; YOLOv8 is faster (83.71 FPS) with 11.14M parameters.

• LENS-Net leads classification on macro-averaged metrics. LENS-Net achieves the best A.P. (85.09%) and A.R. (79.32%) on INTSD, ahead of EfficientNet-B0 (81.68 / 75.42), YOLO-TS (75.08 / 74.43), Rishabh et al. (74.49 / 69.98), DeIT (65.89 / 59.69), CLIP (65.25 / 68.11), BLIP (49.33 / 47.81), YOLOv12-L (45.70 / 39.60), and YOLOv10-L (37.24 / 26.51). Its accuracy (88.38%) is below YOLO-TS (89.70%) and EfficientNet-B0 (89.12%).

• A trade-off on the 8-class subset. YOLO-TS and Rishabh et al. score higher than LENS-Net on the 8-class subset when trained on nighttime data, but the authors note this advantage does not transfer to the full 41-class INTSD benchmark, where LENS-Net outperforms both. LENS-Net retains comparatively stronger day-only classification (71.43% A.P.), attributed to its semantic attribute reasoning.

• Cross-dataset generalization to CNTSSS. On CNTSSS, LENS-Net reaches the highest classification scores (87.35% A.P., 88.64% A.R., 89.70% Acc.) but lower detection mAP@50 (69.17%) than YOLO-LLTS (76.30%), YOLO-TS (76.47%), and YOLOv10-L (68.84%). LENS-Net records 143.85 FPS for detection and 74.43 FPS for classification on that benchmark.

• INTSD is more discriminative than CNTSSS. The authors state that several methods achieve strong performance on CNTSSS due to its limited class taxonomy, whereas INTSD produces larger variation across models, reflecting increased semantic diversity and intra-class ambiguity.

• Joint training across both datasets helps. Under a unified 44-class label space on CNTSSS and INTSD, the authors report (details in the supplementary material) that incorporating INTSD improves cross-domain robustness without degrading CNTSSS performance.

• Dataset characteristics that create difficulty. INTSD exhibits class imbalance with a long tail — the top five classes are advertisement (4,204 instances), direction (2,911), unknown (1,773), left_curve (855), and speed (851), while airport, blow_horn, and cross_road are severely underrepresented. Image sizes are bimodal around sqrt(area) ≈ 1100 and ≈ 3700, signboard sizes follow a long tail skewed toward small signs, and object density averages 2.32 signboards per image (14044/6004).

• Annotation reliability. Detection verification identified and added 3.12% false negatives; a 0.47% class-labeling error rate after the second verification dropped to 0% after the third and final verification stage.

Methodology in Plain English

Building the dataset. The authors recorded videos at 30 fps using a low-budget (approximately 60 USD) AKASO Ek7000 action camera, supplemented with images from Samsung, iPhone, and Google Pixel smartphones — six sensor types in total, split into high-quality (smartphone) and low-quality (action-camera) subsets. Of the 11,016 images, 10,004 are nighttime, split into 6,004 with signboards and 4,000 without; 1,012 daytime images from the same routes are included to support day-to-night and night-to-day translation tasks. The authors deliberately did not collect a large daytime set because prior Indian datasets already cover that.

Annotation. Frames were subsampled to one per second to remove near-duplicate neighbors, filtered for candidate sign frames, and manually annotated with bounding boxes and class labels using the VGG Image Annotator. Severely glare- or blur-obscured frames where no sign was discernible were discarded, but light-to-moderate blur and glare were intentionally retained. Labels follow the official taxonomy of India's Ministry of Road Transport and Highways, mapped to 41 classes chosen so that no two classes overlap visually. Every box and label was triple-verified. The data is partitioned into five stratified folds of 1188, 1226, 1175, 1189, and 1226 images with 2133, 2134, 2129, 2133, and 2128 instances respectively, with fold definitions publicly released.

LENS-Net detector. The detection backbone is YOLOv8. Because INTSD spans scenes from headlight- and street-lamp-lit to severely underexposed, the authors wrap the backbone in a lightweight PreprocessWrapper whose small parameter head (about 48.93k parameters) predicts per-image enhancement parameters — gamma correction, brightness, and unsharp amount — applied through differentiable transforms before detection. This head is trained end-to-end with the detection loss. To stop it from collapsing to trivial solutions, training injects strong non-differentiable augmentations (photometric jitter, JPEG compression, Gaussian noise, blur, and small rotations) before the wrapper, and adds an L2 regularization term penalizing the predicted parameters for deviating from their no-operation defaults, with per-parameter weights in the range (0, 0.01). Training uses a two-phase zero-one scheduler: head-only optimization with a frozen backbone, then joint end-to-end training.

LENS-Net classifier. Each detected crop is processed by two branches. The first extracts a global image embedding with a frozen CLIP vision encoder and fuses it, via cross-attention, with embeddings of pre-defined textual prompts produced by a frozen CLIP text encoder; a residual connection, layer normalization, and a feedforward block yield a text-conditioned feature. The second branch models shape and color explicitly, predicting soft distributions over learnable shape and color embedding tables (for example, 0.85 circle, 0.1 octagon, 0.05 other) rather than a hard argmax, then concatenating these with the global feature and mixing them through a three-layer MLP. A second cross-attention block lets the text-conditioned feature attend to the attribute features, followed by residual, LayerNorm, and feedforward layers, and an MLP head produces class logits. Training uses class-balanced focal loss with weights proportional to inverse square-root class frequency, plus a rarity-based augmentation. The authors report that a simple MLP mixer matched a graph-based alternative.

Evaluation. All models are trained and evaluated under identical splits and protocols with stratified 5-fold cross-validation. FPS is measured end-to-end from raw image loading through preprocessing, forward pass, post-processing, and CUDA synchronization, averaged over five independent runs.

Why This Matters

Impact on research. The paper argues that low-light degradations — glare, motion blur, sensor noise, and underexposure — are not recoverable from daytime imagery alone, and that INTSD supplies training signal complementary to existing daytime benchmarks. By releasing folds, annotations, and code publicly, it provides a reproducible platform for nighttime TSR and a demonstration that synthetic day-to-night translation is an inadequate substitute for real nighttime data.

Real-world applications:

• Advanced driver assistance systems and autonomous driving that must operate reliably after dark, where the paper shows daytime-trained detectors collapse to roughly 5% mAP@50.
• Navigation and mapping services that need to recognize roadside signage across regions and lighting conditions.
• Smart city and traffic infrastructure monitoring, including detection of tampered, rusted, or vandalized signage.
• Fleet and commercial vehicle safety systems deployed on varied routes and heterogeneous camera hardware, which INTSD's six sensor types and six-district coverage represent.

Industry relevance. Because FPS is reported end-to-end alongside parameters and trainable parameters, the benchmark gives deployment-oriented comparisons, not just accuracy. LENS-Net reaches 94.46% mAP@50 at 78.11 FPS with 11.18M parameters, which is competitive with the fastest detector in the table (YOLOv8 at 83.71 FPS, 11.14M parameters) — relevant for embedded automotive hardware. The paper also shows a practical gap in existing evaluation practice: classification accuracy alone can hide poor performance on rare or ambiguous classes, which macro-averaged precision and recall expose.

Future Directions

• Closing the gap between benchmark rankings and deployment. LENS-Net leads macro-averaged precision and recall on the full 41-class benchmark but trails on accuracy and has lower detection mAP@50 on CNTSSS, and it is beaten by YOLO-TS and Rishabh et al. on the 8-class subset under nighttime training. Which model should actually be deployed, and under what metric, remains open.

• Handling the long tail. The dataset's severe class imbalance — where airport, blow_horn, and cross_road are severely underrepresented while advertisement alone has 4,204 instances — motivates specialized methods for learning robust representations on both head and tail, and for the four distractor classes.

• Rethinking nighttime data synthesis. Since generative translation introduces unrealistic artifacts and prior work shows it does not accurately assess real nighttime performance, a natural next step is determining what combination of real nighttime data and realistic augmentation best approximates true night-time conditions.

• Extending the taxonomy and geography. The authors describe the 41-class taxonomy as inherently flexible, allowing incremental expansion of INTSD, and note their regional focus on India — leaving multi-region nighttime benchmarks and joint label spaces (such as the 44-class CNTSSS + INTSD setting referenced in the supplementary material) as natural extensions.

Target Audience

Researchers and engineers working on traffic sign recognition, nighttime or low-light perception, and autonomous driving datasets; practitioners building ADAS or fleet-safety systems who need to understand how much performance degrades at night and what data can recover it; benchmark and dataset designers interested in attribute-level annotation, distractor classes, and cross-domain utility analysis; and graduate students or applied scientists with intermediate computer vision background looking for a realistic, non-daytime evaluation platform. Portions of the methodology — the adaptive preprocessing head and the vision-language classifier with soft attribute embeddings — assume familiarity with detectors, CLIP, and cross-attention, and full implementation and ablation details are relegated to the supplementary material rather than the main text.

Authors’ abstract

Traffic signboards are vital for road safety and intelligent transportation systems. Yet, recognizing traffic signs at night remains underexplored due to the scarcity of real-world public datasets capturing low-light degradations and distractor classes. Existing benchmarks are predominantly daytime and do not reflect challenges such as headlight glare, motion blur, sensor noise, and vandalized or ambiguous signage. To address these gaps, we introduce INTSD, a large-scale nighttime traffic sign dataset collected across diverse regions of India. INTSD contains street-level images spanning 41 traffic signboard classes, multiple distractor categories, and varied lighting and weather conditions, designed to support both detection and fine-grained classification under nighttime scenarios. To benchmark INTSD, we conduct extensive evaluations using state-of-the-art detection and classification models under standardized protocols. Additionally, we present LENS-Net, a strong baseline that integrates an end-to-end adaptive illumination-aware detector with a multimodal classifier that fuses vision-language representations with soft semantic attribute reasoning over learnable shape and color embeddings. Experiments demonstrate that models trained exclusively on daytime data fail substantially under real nighttime conditions - a gap that is recovered once INTSD is introduced in training, even when controlling for data volume. These results validate INTSD as a complementary nighttime training resource and establish competitive baselines for future research. The code and dataset are publicly available.