HNDiff: Haze-Noise Diffusion for Image Dehazing

AI summary

Overview

Research area: Computer vision, specifically single-image dehazing (removing haze/fog from photographs) using diffusion models grounded in atmospheric physics.

Technical level: Intermediate. The paper builds on familiar concepts (convolutional and Transformer dehazing backbones, denoising diffusion) but introduces derived equations from the Atmospheric Scattering Model, so readers benefit from some background in diffusion and low-level vision.

Scope: The paper proposes Haze-Noise Diffusion (HNDiff), a diffusion framework that embeds the Atmospheric Scattering Model (ASM) into both its forward and reverse processes, plus a latent variant that plugs into existing dehazing networks and improves four state-of-the-art backbones across seven benchmark datasets.

What This Paper Is About

Hazy images lose contrast and detail because atmospheric scattering attenuates scene radiance and mixes it with global atmospheric light. Dehazing is hard because the degradation depends on unknown quantities such as scattering coefficients, atmospheric light, and scene depth. Existing diffusion-based dehazers start from pure Gaussian noise and ignore how haze actually forms, which limits fidelity and wastes generative capacity on regions that are not heavily degraded. HNDiff instead makes haze formation part of the diffusion process itself, so the model corrupts and restores images along a physically meaningful trajectory.

Key Contributions

1. HNDiff, an ASM-grounded diffusion framework. The proposed framework incorporates the Atmospheric Scattering Model as an inductive bias, so the diffusion process follows the physics of haze formation rather than treating haze as generic noise.

2. A joint haze-noise forward process with a matching reverse process. The forward process adds both haze and Gaussian noise to a clean image; the reverse "dehazing-denoising" process removes both, using two dedicated estimators — one for noise and one for haze.

3. A haze-aware noise scheduler (HANS). The scheduler sets a pixel-wise noise coefficient β_t(x) = 1 − e^(−α_t σ(x) d(x)), so hazier regions receive stronger noise injection (encouraging generation of lost content) and clearer regions receive weaker noise (preserving detail fidelity).

4. Latent HNDiff with a Feature Gating Module (FGM). HNDiff runs in a latent space as a prior generation network, and the resulting clean latent priors are injected into an existing dehazing backbone through the FGM, making the method plug-and-play rather than a standalone architecture.

Main Findings

• Consistent gains across four backbones. HNDiff improves FocalNet by an average of +0.48 dB, ConvIR by +0.59 dB, and SGDN by +0.63 dB in PSNR, and it is also applied to RIDCP.

• Per-dataset average PSNR improvements. Averaged over baselines, HNDiff yields +0.54 dB on NH-HAZE, +0.87 dB on O-HAZE, +0.41 dB on Dense-HAZE, +0.40 dB on RW2AH, +0.47 dB on SOTS-Indoor, and +0.72 dB on SOTS-Outdoor. The paper reports an overall average gain of +0.57 dB PSNR and +0.009 SSIM across all datasets and baselines.

• Best result on the real-world RTTS benchmark. RIDCP+HNDiff achieves 0.417 FADE, 5.08 NIMA, and 16.09 BRISQUE, which the authors describe as the best among compared methods on that benchmark (RIDCP alone: 0.944 FADE, 4.43 NIMA, 18.78 BRISQUE).

• Both components help, and HANS adds more. In the ablation on NH-HAZE with FocalNet as baseline (20.36 PSNR), noise diffusion alone reaches 20.46, haze diffusion alone reaches 20.61, combining both without HANS reaches 20.68, and the full HNDiff reaches 20.89.

• The physical design beats generic diffusion priors. Comparing prior generators, a plain U-Net reaches 20.41 PSNR, DDPM reaches 20.46, RDDM reaches 20.43, and HNDiff reaches 20.89 (SSIM 0.697 versus 0.696 for the baseline FocalNet).

• Latent space is better than image space. For the image-space variant, PSNR is 21.37, SSIM 0.6166, and FLOPs 65.59 G; for the latent variant, PSNR is 21.52, SSIM 0.6254, and FLOPs 36.38 G; baseline FocalNet is 21.18 PSNR, 0.5970 SSIM, and 30.53 G FLOPs.

• Scaling the network is not a substitute. Enlarged FocalNet variants (FocalNet+ at 8.40 M parameters / 68.54 G FLOPs reaching 20.37 dB, and FocalNet* at 8.28 M / 64.05 G reaching 20.51 dB) perform below HNDiff, which reaches 20.89 dB with 7.82 M parameters and 36.38 G FLOPs.

• Small diffusion budgets suffice. With FocalNet and SGDN, performance peaks at T = 4 steps; ConvIR peaks at T = 6; T = 0 gives limited performance in all backbones and larger T brings no additional gains.

• Modest overhead. HNDiff adds parameters, FLOPs, and training VRAM: FocalNet goes from 3.74 M / 30.53 G / 2854 M to 7.82 M / 36.38 G / 3932 M; ConvIR from 5.51 M / 41.96 G / 4506 M to 9.59 M / 45.39 G / 5550 M; SGDN from 11.09 M / 52.95 G / 22980 M to 15.30 M / 56.38 G / 23914 M.

Methodology in Plain English

The researchers start from the standard equation that describes how haze forms: a hazy image equals the clean scene attenuated by transmission, plus atmospheric light scaled by how much transmission is lost. Their key move is to treat this equation as the mean of a diffusion step rather than assuming zero mean, as ordinary diffusion does.

In the forward direction, they repeatedly apply that haze equation while also adding Gaussian noise. The noise is not uniform across the image: a scheduler ties the noise strength at each pixel to how hazy that pixel is (β_t(x) = 1 − e^(−α_t σ(x) d(x))). Heavy-haze regions get strong noise so the model can generate plausible missing detail; clear regions get little noise so existing detail is preserved by regression.

Because dehazing datasets only contain hazy-clean image pairs, they do not provide the atmospheric light, scattering coefficients, and depth needed to compute transmission. The authors handle this with a reparameterization trick and a continuous accumulation formulation, so the model can sample the fully hazy-noisy image in one step, I_T(x) = I_H(x) + β̄_T(x) ε(x), and they introduce a learnable haze estimator to approximate the transmission term implicitly.

The reverse process starts from that hazy-noisy sample instead of pure noise and iterates a sampling equation, I_{t−1}(x) = (I_t(x) − N_t(x)(1 − e^(−α_t σ(x) d(x)))) e^(α_t σ(x) d(x)), where N_t(x) = A + ε_t(x) is called the atmospheric noise. Two networks do the work: a noise estimator approximating N_t, and a haze estimator approximating the residual transmission term. Both are simplified U-Nets sharing the same architecture.

To avoid the cost and fidelity problems of running this in RGB space, Latent HNDiff performs the diffusion on an encoder representation. The image encoder uses six residual blocks and four CNN layers, and the recovered clean latent prior is fused into the backbone's multi-scale features via a Feature Gating Module built from a pooling operation and a lightweight MLP. Training proceeds in stages: pretrain the backbone with the encoder and FGM using paired data, train HNDiff to predict clean priors with an L1 prior loss, then jointly fine-tune everything. The diffusion step is set to T = 4, and each backbone keeps its own default hyperparameters and training protocol for fair comparison.

Why This Matters

Impact on research. The paper argues that degradation-aware diffusion should respect the physics of the degradation, not just the statistics of noise. It joins a small line of work on degradation-aware diffusion (including blur diffusion and residual diffusion) and shows that embedding transmission and haze density, rather than only haze accumulation, produces measurably better results. Its latent-prior design also makes diffusion a reusable module for existing dehazing architectures rather than a replacement for them.

Real-world applications:

• Autonomous driving and advanced driver-assistance systems, where haze degrades visibility and downstream perception.
• Surveillance and outdoor camera monitoring, where image clarity affects detection and identification.
• Object detection, semantic segmentation, and face recognition pipelines, which the paper explicitly notes are impaired by hazy inputs.
• Aerial and remote sensing, where atmospheric scattering obscures ground detail.

Industry relevance. Dehazing is a practical preprocessing step for camera-based products and vision systems. HNDiff's appeal for deployment is that it runs at only a few diffusion steps (T = 4) and adds modest FLOPs (for example, 36.38 G versus 30.53 G for FocalNet) while delivering measurable PSNR gains, so it can be bolted onto an existing dehazing model rather than requiring a new one.

Future Directions

• Extending beyond haze. The limitations section states that HNDiff is tailored to the Atmospheric Scattering Model and cannot be directly applied to motion blur, raindrops, or low-light conditions; doing so requires integrating degradation-specific priors such as object motion, rain masks, or exposure time, which the authors leave as future work.
• Better fidelity in heavily degraded regions. The paper notes that direct image-space diffusion may suffer fidelity issues in severely degraded regions because of the stochastic nature of diffusion; further work could push on fidelity in those areas.
• Adaptive step counts. Since backbones peak at different numbers of diffusion steps (T = 4 for FocalNet and SGDN, T = 6 for ConvIR), determining the step count per backbone or per dataset remains an open practical question.
• Broader backbone and dataset coverage. The evaluation covers four backbones and seven benchmarks; generalizing the plug-in prior to other restoration architectures and degradation combinations is an open question.

Target Audience

Researchers and graduate students in computer vision and image restoration, particularly those working on diffusion models or low-level vision; engineers building dehazing or enhancement modules into camera-based or autonomous systems; and readers interested in how physical models can be used as inductive biases inside generative models. A working knowledge of diffusion processes and image formation models makes the method sections considerably easier to follow.

Authors’ abstract

Existing diffusion-based methods have recently made significant progress in image dehazing. However, they typically neglect the physics of haze formation and reconstruct clean images from pure Gaussian noise, thereby limiting their restoration potential. To address this issue, we propose Haze-Noise Diffusion (HNDiff), a novel diffusion framework that embeds the atmospheric scattering model as an inductive bias. By grounding diffusion in physical principles, HNDiff ensures that the restoration aligns more closely with underlying mechanisms of haze formation. In its forward process, we introduce joint haze-noise diffusion with a haze-aware noise scheduler, which progressively adds both haze and noise to an image. Essentially, the scheduler adapts noise levels according to haze density, meaning that regions with heavier haze receive stronger noise injection to encourage content generation, while clearer regions receive lighter noise to better preserve details, which directly links the forward degradation process with the physics of haze. In the reverse process, we then derive a physically consistent dehazing-denoising process that simultaneously removes haze and noise to restore a clean image in a manner aligned with the forward degradation process. To further enhance practicality, we propose Latent HNDiff, which compiles clean latent priors that can be seamlessly integrated into existing dehazing networks to boost performance. Extensive experiments show that our work significantly improves leading dehazing backbones and achieves state-of-the-art results on benchmark datasets. The project page is available at https://jin-ting-he.github.io/HNDiff .