Contrastive Hypergraph Source-free Domain Adaptive Object Detection in Adverse Weather Conditions--Extended Version

AI summary

Overview

• Research area: Computer vision — source-free domain adaptive object detection (SFDA-OD), with a focus on robust perception under adverse weather (fog, rain, snow, sandstorm, low illumination).
• Technical level: Advanced. The paper assumes familiarity with mean-teacher self-training, contrastive learning (InfoNCE), graph/hypergraph neural networks, and pseudo-labeling.
• Scope (one sentence): The paper proposes REUNION, a hypergraph-guided source-free adaptation framework that organizes target-domain object proposals into multi-relation hyperedges and optimizes them with a group-wise Hyper-InfoNCE objective, reporting state-of-the-art results on Cityscapes-Adverse and Sim10K→Cityscapes-Adverse benchmarks.

What This Paper Is About

Object detectors trained on clear-weather images degrade sharply when deployed in fog, rain, snow, sandstorms, or low light, because these conditions cause structured visibility loss (contrast attenuation, color shifts, partial occlusion). When the original source training data cannot be kept — due to privacy, licensing, or transmission/storage costs — adaptation must proceed source-free, using only a source-trained detector and unlabeled target images. REUNION's goal is to make that adaptation stable by replacing pairwise contrastive alignment (which the authors argue collapses under severe corruption) with group-level, hypergraph-structured relations among object proposals.

Key Contributions

1. REUNION framework: A hypergraph-guided source-free domain adaptation framework for object detection in adverse weather, built on a mean-teacher backbone with pseudo-label supervision.
2. Multi-relation contrastive hypergraph: A hypergraph over target-domain proposals whose hyperedges encode intra-image context, weather-aware grouping, prototype-based semantic anchors, uncertainty-aware connections between reliable and low-confidence instances, and feature-nearest-neighbor links — optimized by a group-wise Hyper-InfoNCE objective that propagates semantics at the hyperedge level and is claimed to mitigate "agreement collapse."
3. Stabilizers for noisy pseudo labels: Prototype-based semantic regularization (class- and weather-conditioned prototypes updated by EMA) plus uncertainty-aware propagation, where teacher uncertainty is measured by normalized entropy and low-confidence proposals are linked to their K nearest high-confidence neighbors (default K = ten) sharing the same pseudo class and weather label.
4. Layout-preserving weather augmentation: Cross-condition views synthesized with a frozen diffusion model, FLUX.1 KONTEXT, used as augmentation without training a generator or accessing labeled source data.
5. Empirical validation: Experiments across Cityscapes→Cityscapes-Adverse and Sim10K→Cityscapes-Adverse, with ablations over components, hyperedge types, object sizes, and class-wise performance.

Main Findings

• Cityscapes→Cityscapes-Adverse (ResNet-50, AP): REUNION reaches 39.3 mAP, versus 32.3 for the strongest SFDA baseline IRG and 37.7 for the strongest UDA baseline ALDI. The paper reports an improvement of approximately 7.0% AP over IRG. For reference, Source-Only scores 11.0 and the Oracle (trained with labeled target data) scores 42.7.
• Per-condition strength: REUNION records 43.7 on D-Rainy, 38.2 on D-Snowy, 53.9 on D-Foggy, 45.7 on D-Sandstorm, 36.1 on N-Clear, 26.6 on N-Rainy, and 30.8 on N-Foggy. The paper highlights rainy, sandstorm, and night-foggy as particularly challenging conditions where it performs well.
• Sim2Real (Sim10K→Cityscapes-Adverse, AP Car): REUNION achieves 42.6, improving 1.8% over the strongest SFDA baseline IRG (40.8) and trailing the source-using UDA method ALDI (42.4 is reported slightly below 42.6, so REUNION is competitive with UDA). Source-Only is 19.5.
• Object-size breakdown: REUNION obtains 34.6 AP_L and 8.4 AP_S, the best among the listed SFDA methods on both, but 11.1 AP_M — lower than LPLD (14.3) and MT (15.6) in the same table. The paper notes small-object gains "remain limited due to extremely low visibility" and does not analyze the medium-object result.
• Component ablation (Cityscapes-Adverse, AP): Baseline 30.0 → with pairwise augmentation 34.1 → with proposal-level and semantic-level hyperedges but no Hyper-InfoNCE 38.7 → full model 39.3 (AP_L 34.6, AP_S 8.4). The authors attribute the gain without contrastive loss to forward message passing, where the HGNN acts as a relational regularizer before the detection head.
• Hyperedge ablation: Starting from feature-based hyperedges (37.2 AP), adding intra-image and weather-aware hyperedges gives 37.9 (a 0.7 AP gain as reported), and adding prototype and uncertainty-aware hyperedges gives the best 39.3 AP. Adding the prototype hyperedge alone moves AP to 38.8 but AP_M drops from 11.6 to 9.3 in that row.
• Qualitative behavior: The paper reports that IRG and SF-UT often produce false positives in dark regions or miss objects under severe illumination change, while REUNION produces more accurate detections across illumination and weather conditions; class-wise AP comparisons (Fig. 3) show REUNION ahead in most categories, with larger gains in rain, snow, sandstorm, and night-foggy.

Methodology in Plain English

The starting point is a detector trained on labeled clear-weather (or synthetic) images, which is then adapted using only unlabeled target images that carry a coarse weather label (from metadata or a frozen pretrained weather classifier).

1. Teacher-student self-training. Two identical detectors are kept: a student trained by gradient descent on pseudo labels produced by a teacher, and a teacher updated as an exponential moving average of the student. Pseudo labels are the teacher's argmax class and its box regression; the student is trained with a detection loss on those labels plus a KL-divergence distillation loss between teacher and student class logits.
2. Build a hypergraph over proposals. All region proposals in a mini-batch become vertices with L2-normalized embeddings. Hyperedges (groups that can contain more than two proposals) are formed in several ways: each proposal with its K nearest neighbors in embedding space; all proposals from the same image; proposals from images with the same weather label; proposals close to a class-and-weather-specific prototype above a cosine threshold; and low-confidence proposals linked to their K nearest reliable same-class, same-weather neighbors.
3. Pass messages, then contrast. A graph convolution over the hypergraph adjacency matrix lets weak or isolated proposals absorb context and prototype semantics from confident neighbors before the features go to the detection head.
4. Hyper-InfoNCE. For each anchor proposal, positives are other proposals that are hypergraph-adjacent and share the same pseudo class. Each positive pair is weighted by normalized adjacency, so the loss pulls the student toward structurally and semantically consistent teacher embeddings while pushing away all other proposals in the mini-batch. Averaging over anchors with non-empty positive sets gives the final hypergraph contrastive loss.
5. Weather augmentation. A frozen FLUX.1 KONTEXT model generates layout-preserving variants of target images under different weather conditions to strengthen cross-condition consistency, without training a generator or touching source data.

Implementation details reported: R-CNN with pre-trained ResNet-50, Detectron2 and PyTorch, learning rate 0.04, teacher EMA 0.9996, λ = 1.0, four Nvidia A40 GPUs with batch size 16 per GPU. Datasets: Cityscapes (2,950 images, 8 classes) for source training; Cityscapes-Adverse (23,600 images, 8 categories, seven adverse weather and lighting conditions) with 20,650 adaptation and 3,500 evaluation images; Sim10K (9,975 images, 1 class) with 19,768 adaptation and 3,346 evaluation images.

Why This Matters

Research impact. The paper reframes source-free adaptation as group-level representation learning rather than pairwise alignment, and offers a concrete argument (backed by ablations) that relational regularization via hypergraph message passing stabilizes training when pseudo labels are noisy. It also shows a frozen text-to-image diffusion model can be used as a layout-preserving augmentation source in a source-free setting, which is a reusable trick beyond this benchmark.

Real-world applications:

• Autonomous driving and advanced driver-assistance systems that must keep working when a vehicle enters fog, rain, snow, or night-time conditions after deployment.
• Fleet-scale perception updates where the original training data cannot be shipped back or retained for privacy or licensing reasons.
• Robotics and delivery/drone perception operating across changing weather with no labeled data from the deployment site.
• Surveillance and traffic monitoring cameras whose imaging conditions drift seasonally or with illumination changes.

Industry relevance. Because the method requires only a source-trained checkpoint and unlabeled target images, it fits practical constraints of model maintenance: no data-sharing agreements, no re-annotation, and adaptation on the deployment side. The reported gain over the strongest SFDA baseline (32.3 → 39.3 mAP) is large enough to matter for a safety-critical component, and the pipeline (Detectron2, ResNet-50, four A40 GPUs) reflects realistic engineering rather than an exotic setup. The competitive standing against UDA methods that do use source data is the practically interesting claim.

Future Directions

• Broader and mixed conditions: The authors name mixed weather and low-light scenes as extensions beyond the seven conditions in Cityscapes-Adverse.
• Better pseudo-label calibration: The paper explicitly flags improving calibration under severe corruption as open work, which connects to the observed medium-object and small-object weakness.
• Efficiency of hypergraph message passing: Lighter hypergraph schemes are proposed as future work, since the current pipeline needs proposal-level hypergraph construction per mini-batch.
• Unresolved questions the results raise: Why AP_M (11.1) falls below several SFDA baselines while AP_L (34.6) is best, and why adding the prototype hyperedge alone drops AP_M to 9.3, are not analyzed and invite follow-up study. The reliance on a frozen diffusion model for weather variants also raises untested questions about generator-induced artifacts.

Target Audience

Researchers and graduate students working on domain adaptation, source-free learning, or robust object detection, plus applied engineers in autonomous driving and intelligent transportation who need to maintain detectors without access to source data. Readers should be comfortable with contrastive objectives and graph neural networks; the paper is not introductory. Those specifically interested in hypergraph formulations for vision, or in diffusion models as data augmentation for adaptation, will find the most directly reusable ideas.

Note: the supplied paper content is truncated inside Appendix A, so the remaining supplementary ablations and the class-distribution figure are not covered here.

Authors’ abstract

Adverse weather conditions such as fog, rain, snow, and low illumination introduce structured visibility degradations that severely impair object detectors trained on clear-weather data. When source data are inaccessible due to privacy or trans- mission constraints, source-free domain adaptation (SFDA) becomes necessary. However, existing SFDA methods based on pairwise contrastive learning struggle to capture high-order semantics and often suffer from agreement collapse under severe corruption. We propose REUNION, a hypergraph-guided SFDA framework for object detection in adverse weather conditions. REUNION constructs a contrastive hypergraph over target-domain object proposals, encoding high-order relations through intra-image context, weather-aware grouping, prototype-based semantic anchors, and uncertainty-aware connections between reliable and low-confidence instances. To effectively exploit these structured relations, we introduce a group-wise Hyper-InfoNCE objective that optimizes representations at the hyperedge level, enabling semantic information to propagate from confident proposals to corrupted or low-contrast instances. Experiments with diverse benchmarks demonstrate that REUNION effectively mitigates domain shifts and achieves state-of-the-art performance in SFDA settings.