DRAFE: Domain-Robust Asymmetric Fusion of Heterogeneous Detection Transformers for Cross-City Fine-Grained Traffic Object Detection

AI summary

Overview

• Research area: Computer vision, specifically cross-city fine-grained traffic object detection within intelligent transportation systems (ITS), evaluated on Track 6 of the 10th AI City Challenge.
• Technical level: Advanced. The paper assumes familiarity with detection transformers (DETR-family architectures), bounding-box fusion strategies such as Weighted Boxes Fusion, domain generalization terminology, and standard detection metrics (mAP, AP50, AP75, scale-stratified AP and AR).
• Scope: The paper presents DRAFE (Domain-Robust Asymmetric Fusion Ensemble), a training-to-inference pipeline that combines two LW-DETR XLarge detectors and one RF-DETR Base detector using asymmetric anchor-support roles and reliability-weighted fusion, evaluated under a privacy-preserving Training-as-a-Service constraint on the Project Hafnia platform.

What This Paper Is About

Traffic object detectors often fail when deployed in cities other than the ones they were trained on, because camera viewpoints, roadway geometry, traffic density, and regional vehicle distributions shift. Track 6 of the 10th AI City Challenge sharpens this problem by requiring fine-grained recognition across ten visually similar classes (Car, Pickup Truck, Single Truck, Combination Truck, Trailer, Heavy-Duty Vehicle, Van, Motorcycle, Bicycle, and Person) on a long-tailed dataset whose imagery participants cannot download. The paper's goal is to build an ensemble that generalizes to unseen target cities using only labeled source-city data, without ever accessing target-domain images during training.

Key Contributions

1. The DRAFE ensemble method, which combines asymmetric detector roles (one anchor, two support models) with class-consistent one-to-one matching, reliability-weighted coordinate fusion, agreement-aware confidence recalibration, and complementary hypothesis recovery.
2. A scalable pseudo-label expansion pipeline that produced a curated traffic-domain corpus of 6,049 images and 203,619 human-reviewed annotations, starting from a meta-model trained on 1,683 manually annotated seed images and yielding 3.6 times greater image coverage.
3. A competitive benchmark result on Track 6 of the 10th AI City Challenge: 0.4022 mAP, ranking sixth among 25 participating teams, an improvement of 0.0553 mAP (15.9% relative) over a preliminary ensemble evaluated under identical conditions and 0.0181 mAP (4.7% relative) over the strongest standalone component detector.

Main Findings

• Ensemble outperforms its components on the hidden benchmark: DRAFE reached 0.4022 mAP, versus 0.3841 mAP for LW-DETR XLarge-A, 0.3822 mAP for LW-DETR XLarge-B, and 0.3836 mAP for RF-DETR Base. The preliminary ensemble scored 0.3469 mAP.
• Scale-stratified gains were broad: DRAFE reported AP50 = 0.5333, AP75 = 0.4259, APS = 0.0571, APM = 0.2335, APL = 0.5460, and AR100 = 0.6981. Relative to LW-DETR XLarge-A, the largest gain was at AP75 (+0.0261), which the authors describe as consistent with improved localization among matched detections.
• Small objects remain the weak point: On the complete hidden benchmark, small objects achieved APS = 0.0571 and ARS = 0.2131, while medium objects reached APM = 0.2335 and ARM = 0.4938, and large objects reached APL = 0.5460 and ARL = 0.8305.
• A fourth detector did not help: On the Hafnia development split, adding a fusion-only fourth model raised AP75 from 0.4312 to 0.4321 and AR100 from 0.6968 to 0.6999, but reduced mAP from 0.4074 to 0.4068. This experiment used a single training seed.
• Reliability weights were tuned on the development split: Adjusting the weights from (0.43, 0.32, 0.25) to (0.42, 0.32, 0.26) — shifting one percentage point from LW-XA to RF-B — increased the development score to 0.4077. Increasing the agreement bonus from λ = 0.03 to λ = 0.04 raised AP50 and AR100 but reduced AP75, medium-object AP, large-object AP, and overall mAP.
• A low-budget detection configuration performed poorly: A thresholding and suppression setup that reduced output to 23.22 detections per image produced a development mAP of only 0.3520, supporting the decision to retain 300 ranked detections per image with no hard preselection threshold.
• Qualitative recovery of objects: On three selected images, DRAFE increased true positives from 7–8 to 10, from 11 to 12, and from 5–8 to 9 compared with standalone detectors, with additional false positives accompanying the recovered objects.
• Leaderboard context: The top-ranked team (SKKU-AL-T1) reported 0.4753 mAP, followed by BIT-ODL (0.4281), Remote Vibecoders from Chisinau (0.4176), BK2TheFuture (0.4169), and S2 Detection (0.4114), with VisionOps/DRAFE sixth at 0.4022.
• Predictions per image differed sharply: LW-DETR XLarge-A and XLarge-B both returned 300.00 boxes per image, whereas RF-DETR Base returned an average of 77.66.

Methodology in Plain English

The team assembled a traffic-domain pretraining corpus from five public sources (MIO-TCD Localization, UA-DETRAC, the Vehicle Detection Image Dataset, a Roboflow vehicle-detection dataset, and Roboflow 100). Because the source taxonomies and box conventions differed, they did not transfer original labels. Instead, human reviewers fully annotated 1,683 seed images in CVAT using the ten-class Track 6 ontology, then trained a meta-model whose only job was to propose initial bounding boxes for a larger pool of images. Every one of the 6,049 images in the final corpus was subsequently reviewed and corrected by humans, so no automatic annotation entered training without verification. The corpus was split into 4,000 training, 1,049 validation, and 1,000 held-out test images. Car and Person account for 84.5% of all annotations, while Trailer and Heavy-Duty Vehicle contain 390 and 391 instances respectively.

Three detectors were then fine-tuned independently inside the Project Hafnia managed environment: two LW-DETR XLarge models (five and two epochs, AdamW, learning rate 5×10⁻⁴, batch size 1) and one RF-DETR Base model (two epochs, AdamW, learning rate 1×10⁻⁴, batch size 4 with gradient accumulation over four steps for an effective batch size of 16, on the Hafnia Lite tier with one NVIDIA T4 GPU and 16 GB of memory). Pretraining happened outside Hafnia; only pretrained initializations and training code were brought in.

At inference, each model processes every image independently. The strongest LW-DETR XLarge model acts as the anchor and supplies the initial hypothesis space, sorted by confidence. Each anchor is matched to at most one detection per support detector, requiring an exact class match and IoU ≥ 0.55; once matched, a support detection is removed from the pool so it cannot serve two anchors. Matched coordinates are averaged using weights that multiply detector reliability (αA = 0.42, αB = 0.32, αR = 0.26) by detection confidence. Confidence is recalibrated by taking the maximum confidence in a matched group and applying a conservative multiplicative agreement bonus (λ = 0.03, giving a 3% increase for two agreeing detectors and 6% for three). Unmatched support detections are retained but scaled by γB = γR = 0.97, so anchor-supported hypotheses keep a modest ranking edge while strong unmatched support detections remain eligible. Everything is pooled, sorted, and truncated to the 300 highest-ranked candidates, with no post-fusion confidence threshold and no post-fusion non-maximum suppression.

Why This Matters

Research impact. The paper shows that detector complementarity and asymmetric inference refinement can be combined productively for single-source domain generalization, a setting where unsupervised domain adaptation is impossible because target data is never available. It also documents a privacy-preserving benchmark where imagery cannot be downloaded and trained artifacts are capped at 2 GB, which constrains what kinds of solutions the community can build. The reported ablations — a fourth detector that hurts, a stricter detection budget that hurts, a larger agreement bonus that helps some metrics and hurts others — provide concrete negative results that are rarely published.

Real-world applications.

• Traffic monitoring and vehicle analytics across municipal camera networks that were not part of the original training deployment.
• Infrastructure management and roadway planning that depends on distinguishing heavy-duty vehicles, combination trucks, and trailers from ordinary cars.
• Incident detection and traffic-state estimation in cities with different vehicle mixes than the development city.
• Privacy-constrained deployments where operators cannot export or centralize raw camera imagery, matching the Training-as-a-Service model used here.

Industry relevance. Transportation agencies, ITS vendors, and companies running managed or federated training pipelines face exactly this constraint: models must be trained on data they cannot hold. The paper demonstrates a workflow compatible with that environment and reports placement in a competitive public benchmark (sixth among 25 teams), which is the kind of evidence procurement and deployment teams look for.

Future Directions

• Improving small-object representation, since APS = 0.0571 and ARS = 0.2131 lag far behind the medium- and large-object results, and the benchmark reports no per-class or error-type decomposition to explain why.
• Characterizing class-specific error patterns and confusion between visually similar categories, which the aggregate scale-stratified metrics cannot isolate.
• Evaluating calibration and reliability of the confidence scores across geographic domains; the paper states explicitly that the fused confidence is designed for candidate ranking rather than probabilistic interpretation.
• Running multi-seed evaluation of both the component detectors and the fusion configuration. The four-model ablation used a single seed, and the authors note that multi-seed evaluation was not conducted within the challenge resource allocation.

Target Audience

Researchers and practitioners working on domain generalization, ensemble detection, and detection transformers; participants in the AI City Challenge and similar ITS vision benchmarks; engineers building privacy-preserving or managed-training detection pipelines; and readers interested in data-centric annotation workflows that combine pseudo-labeling with complete human review. Some background in object detection metrics and DETR-family architectures will make the methodology sections considerably easier to follow.

Authors’ abstract

Deep learning-based object detectors are fundamental to intelligent transportation systems, enabling traffic monitoring, vehicle analytics, and infrastructure management. However, achieving both fine-grained vehicle recognition and robust cross-city domain generalization remains challenging. We present the Domain-Robust Asymmetric Fusion Ensemble (DRAFE), which combines independently trained LW-DETR and RF-DETR detectors for cross-city fine-grained traffic object detection. DRAFE employs a two-stage training strategy that first pretrains complementary detectors on diverse public traffic datasets using pseudo-label expansion and human-in-the-loop annotation refinement, producing a curated corpus of 6,049 images and 203,619 annotations, before challenge-compliant fine-tuning on the Project Hafnia Track 6 dataset. At inference, DRAFE applies anchor-conditioned class-consistent matching, reliability-weighted coordinate fusion, agreement-aware confidence recalibration, and complementary hypothesis recovery. On AI City Challenge 2026 Track 6, DRAFE achieves 0.4022 mAP, ranks sixth among 25 participating teams, and improves by 0.0553 mAP over a preliminary ensemble evaluated under identical benchmark conditions.