Gaze-DETR: Top-Down Guidance Through Priority Maps for Infrared Weak-Small UAV Detection with DETR

AI summary

Overview

Research area: Computer vision — infrared small target detection (ISTD), specifically thermal infrared weak-small UAV detection, using DETR-based (DINO-DETR) object detection with bio-inspired priority-map supervision.

Technical level: Advanced. The paper assumes familiarity with DETR-family detectors, Hungarian matching, multi-scale feature pyramids, Kullback–Leibler divergence, and eye-tracking derived fixation-density maps.

Scope: The paper proposes Gaze-DETR, a bio-inspired detector that learns an internal spatial priority map to provide pre-localization guidance complementary to bounding-box supervision, evaluated on the new TIR-UAV120-Gaze benchmark and on Anti-UAV410 under three priority-supervision schemes.

What This Paper Is About

Infrared small target detection is hard because targets occupy only a few pixels, have low contrast, and are easily buried in clutter, noise, or occlusion. Existing detectors are trained with bounding-box supervision, which specifies where targets are but offers little guidance on which ambiguous candidate regions should be examined or prioritized before localization. The paper asks whether a dense spatial priority signal — inspired by task-driven human visual search and the "priority map" concept from Guided Search 6.0 — can supply that missing pre-localization guidance while keeping the detector image-only at inference.

Key Contributions

1. A new framing of pre-localization prioritization as a complementary supervision problem. The authors cast candidate-region prioritization for box-supervised ISTD as an explicit spatial-priority learning task grounded in task-driven visual search, rather than leaving it to be learned implicitly through final detection losses.

2. Gaze-DETR, a bio-inspired priority-guided detector. The detector predicts an internal normalized priority map (64×64, spatially softmax-normalized) from infrared image features. Two modules consume it: Residual Priority-Guided Feature Modulation (RPFM) for priority-guided multi-scale feature modulation, and Priority-Guided Anchor Query Injection (PAQI) for priority-guided anchor-query injection. All priority-supervision maps are training-only, so inference remains image-only.

3. Three priority-supervision schemes for different annotation conditions. Box-derived Gaussian maps generated directly from training boxes (no extra annotation cost); real-gaze maps built from task-driven fixation-density maps; and transferred pseudo-gaze maps produced by learning a statistical gaze–box relation from paired annotations and applying it to Anti-UAV410 training boxes.

4. The TIR-UAV120-Gaze benchmark. A dataset with paired thermal infrared detection and task-driven eye-tracking annotations, constructed specifically to support real-gaze supervision and the gaze–box relation modeling needed for pseudo-gaze transfer.

Main Findings

• TIR-UAV120-Gaze, box-derived supervision: Gaze-DETR reaches 85.76 mAP50 and 88.77 F1.

• TIR-UAV120-Gaze, real-gaze supervision: Gaze-DETR reaches 86.18 mAP50 and 89.00 F1.

• Anti-UAV410, box-derived supervision: Gaze-DETR reaches 87.06 mAP50 and 90.90 F1.

• Anti-UAV410, transferred pseudo-gaze supervision: Gaze-DETR reaches 87.08 mAP50 and 90.43 F1.

• All three supervision schemes improve over the baseline: The paper reports that box-derived, real-gaze, and transferred pseudo-gaze supervision each improve mAP50 and F1 over the DINO-DETR baseline on their corresponding benchmarks, while preserving image-only inference.

• Cheapest scheme is competitive: On Anti-UAV410, the box-derived scheme (which requires no eye-tracking annotation) records the highest F1 in the paper at 90.90, slightly above the transferred pseudo-gaze scheme's 90.43, while the two are near-identical in mAP50 (87.06 vs. 87.08). The paper frames this as demonstrating that explicit spatial-priority learning provides guidance complementary to bounding-box supervision "across annotation conditions and costs."

• Priority supervision is training-only: The priority-supervision map is used solely to compute the priority loss; RPFM and PAQI operate only on the predicted priority map. Raw gaze samples, gaze coordinates, fixation-density maps, pseudo-gaze descriptors, priority-supervision maps, and test-set bounding boxes are not used at inference.

• Note on scope of reported numbers: The provided paper content reports the headline detection metrics above but does not report dataset sizes for TIR-UAV120-Gaze or Anti-UAV410, the numeric values of the hyperparameters (K peaks, R size priors, λ_p, λ_sim), ablation breakdowns per module, or inference-speed measurements. These are not available in the excerpt analyzed here.

Methodology in Plain English

The authors start from DINO-DETR and add a second, parallel "priority pathway" alongside the standard detection pathway.

Step 1 — Predict a priority map. A small convolutional head takes the highest-resolution feature from the backbone's multi-scale pyramid and produces a single-channel logit map that is resized to 64×64 and passed through a spatial softmax. The result is a normalized map that sums to 1 over all locations, so each cell expresses relative priority rather than a target-presence probability. The head uses a 3×3 convolution (256 input and output channels), GroupNorm with 32 groups, ReLU, then a 1×1 convolution.

Step 2 — Supervise the map densely. During training, the predicted map is compared to a supervision map using KL divergence plus a histogram-intersection similarity term, weighted per sample. The supervision map is built one of three ways: a size-adaptive anisotropic Gaussian centered on the training box (spreads computed from box size relative to image size, with a floor of 1.5); a normalized fixation-density map from real eye tracking; or a pseudo-gaze map transferred from a learned gaze–box relation.

Step 3 — Modulate features (RPFM). The predicted map is resized to each feature level, detached from the detection-gradient path, and used in a residual formula: the original feature plus a learnable per-level scalar times the priority-weighted feature. Each scalar is initialized to zero, so training starts exactly from the behavior of the pretrained detector. Because it is residual, low-priority regions are not masked out — the original multi-scale information is fully preserved.

Step 4 — Inject anchor queries (PAQI). A peak selector finds K spatially separated local maxima from the priority map, suppressing nearby repeats so peaks do not cluster. Each peak gives a center but no size, so centers are paired with R weak-target size priors estimated from the training-set bounding-box distribution, producing K×R priority-guided reference boxes. Query contents come from learnable embeddings. These are concatenated with the original DINO queries, so the decoder sees N_d + N_p queries and all are handled by the same Hungarian matching and detection losses — no extra head or detection rule.

Step 5 — Train and infer. The total loss is the standard DINO detection loss (classification, L1 box regression, GIoU, denoising, and auxiliary encoder/decoder losses) plus a weighted priority loss. PAQI is switched on only after a warm-up period so the priority map stabilizes first. At test time only the infrared image is supplied.

Why This Matters

Impact on research. The paper opens a new supervision channel for ISTD: rather than only improving spatial, temporal, or semantic representations (the focus of prior single-frame, multi-frame, and vision–language methods), it supplies an explicit ranking signal for ambiguous candidate regions before localization. It also contributes a paired detection-plus-eye-tracking thermal infrared benchmark, which enables a line of research connecting human visual search behavior to detector training. The three-scheme design is notable because it separates the architecture from the annotation cost, showing the same architecture can be trained from boxes alone, from real gaze, or from transferred pseudo-gaze — a useful template for domains where eye-tracking data is scarce.

Real-world applications:

• Wide-area surveillance — scanning large infrared scenes for distant UAVs where targets occupy only a few pixels.
• Day-and-night monitoring — thermal infrared operation that does not depend on visible light.
• Anti-UAV warning systems — early detection of intruding UAVs at long range, where low contrast and apparent disappearance are common.
• Airspace and perimeter security — where clutter and target-like background structures can otherwise dominate candidate selection.

Industry relevance. The results are practical for deployment: the detector requires no gaze hardware, no priority maps, and no test-time bounding boxes at inference — only the infrared image — so it fits the inference interface of standard detectors. The box-derived scheme achieves strong numbers with zero additional annotation cost, and the pseudo-gaze scheme transfers a gaze–box relation learned on one dataset to another dataset's training boxes, offering a path to gains where eye-tracking is unavailable.

Future Directions

• Scaling to more datasets and sensors. The pseudo-gaze scheme is demonstrated transferring from TIR-UAV120-Gaze annotations to Anti-UAV410 training boxes. Whether the learned gaze–box relation transfers to other infrared datasets, other platforms, or other target types is not established here.

• Quantifying the contribution of RPFM versus PAQI. The paper reports the full architecture in all experiments and states that only the construction of the supervision map changes. Isolating the individual effect of feature modulation versus query injection, and the effect of the warm-up schedule, are natural follow-ups not reported in the provided content.

• Characterizing the annotation-cost versus accuracy trade-off more finely. Real-gaze supervision gave the best mAP50 and F1 on TIR-UAV120-Gaze, but box-derived supervision gave the best F1 on Anti-UAV410. A systematic study of when the extra cost of eye tracking pays off — and why the two datasets differ — remains open.

• Understanding the priority map itself. The paper positions the predicted map as a spatial ranking, not a target-presence probability. Whether this map can be used to explain or diagnose detector failures, or to detect targets that "disappear" apparition-wise, is not explored in the available content.

Target Audience

Primary: Researchers and graduate students working on infrared small target detection, UAV/anti-UAV perception, attention-inspired or gaze-supervised computer vision, and DETR-family detector design.

Secondary: Practitioners building thermal infrared surveillance and warning systems who need to know whether additional annotation (eye tracking) is worth the cost relative to a box-only scheme; and human-vision researchers interested in how priority-map theory from Guided Search 6.0 can be operationalized as a training loss for a machine detector.

Prerequisite background: Comfort with object detection training objectives, transformer decoders and query-based detection, multi-scale feature pyramids, and distribution-matching losses.

Authors’ abstract

Infrared small target detection (ISTD) remains challenging because tiny, low-contrast targets are easily overwhelmed by clutter, noise, or occlusion. Conventional single-frame and multi-frame detectors rely on bounding-box supervision, which specifies final target locations but offers little explicit guidance for prioritizing candidate regions or preserving weak-target evidence before localization. Task-driven visual search offers such guidance: top-down goals and visual evidence jointly form a spatial priority map that ranks candidate locations. Building on this principle, we propose Gaze-DETR, a bio-inspired detector that learns an internal priority map before localization. First, a priority head predicts a normalized priority map from image features. Second, Residual Priority-Guided Feature Modulation (RPFM) enhances high-priority responses while retaining multi-scale features. Finally, Priority-Guided Anchor Query Injection (PAQI) converts high-priority locations into decoder anchor queries. We train the priority head using three supervision schemes: box-derived Gaussian maps; real-gaze maps constructed from fixation-density maps; and transferred pseudo-gaze maps learned from gaze--box relations in paired annotations and applied to Anti-UAV410 training boxes. To support the latter two schemes, we construct TIR-UAV120-Gaze with paired detection and task-driven eye-tracking annotations. On TIR-UAV120-Gaze, Gaze-DETR achieves 85.76 mAP$_{50}$ and 88.77 F1 with box-derived supervision, and 86.18 mAP$_{50}$ and 89.00 F1 with real-gaze supervision. On Anti-UAV410, it achieves 87.06 mAP$_{50}$ and 90.90 F1 with box-derived supervision, and 87.08 mAP$_{50}$ and 90.43 F1 with transferred pseudo-gaze supervision. These results show that explicit spatial-priority learning provides pre-localization guidance complementary to bounding-box supervision across annotation settings and costs.