ClearAIR: A Human-Visual-Perception-Inspired All-in-One Image Restoration

AI summary

Overview

Research area: Computer Vision, specifically low-level vision and All-in-One Image Restoration (AiOIR), combining multimodal large language models, prompt-based restoration, and cognitive-science-inspired design.

Technical level: Advanced. The paper assumes familiarity with transformer restoration backbones (Restormer), cross-attention, prompt learning, vision-language models, and standard restoration benchmarks (PSNR/SSIM/MUSIQ/NIQE).

Scope in one sentence: ClearAIR proposes a single unified restoration network that mimics the coarse-to-fine order of human visual perception — global quality judgment, then regional localization, then local degradation identification, then fine-detail refinement — to handle multiple degradations (noise, haze, rain, blur, low-light, snow, raindrops, and their composites) within one model of 31M parameters.

What This Paper Is About

Most existing All-in-One Image Restoration methods process an entire image with one uniform strategy, ignoring the fact that degradation is usually not spread evenly across the image and that some regions (flat textures) are far easier to restore than others (complex textures). ClearAIR's goal is to build a single model that restores images hierarchically, the way humans reportedly perceive scenes: first grasping the whole, then locating problem regions, then identifying what is wrong locally, and finally recovering fine detail. The claim is that this perceptual ordering produces fewer oversmoothing artifacts and better preservation of texture than degradation-specific representations alone.

Key Contributions

1. ClearAIR, an HVP-inspired AiOIR framework. A hierarchical, coarse-to-fine restoration pipeline that progressively improves structural integrity and perceptual quality, rather than applying one uniform operation across the whole image.

2. A perception-ordered pipeline combining global and local cues. An MLLM-based Image Quality Assessment model (DeQA) produces an overall quality score embedding that drives a Quality Guidance Module, while a Semantic Guidance Unit (SGU) built on a pre-trained Segment Anything Model and a Task Identifier built on DA-CLIP supply region-level semantic masks and degradation prompts.

3. The Internal Clue Reuse Mechanism (ICRM). A self-supervised module that applies weak then strong augmentation to the restored output and penalizes the L2 distance between the two results, forcing the network to exploit the image's own intrinsic structure to refine fine details.

4. A perceptual ordering ablation. The authors explicitly test three perception orders (How-Where-What, Where-What-How, What-How-Where) and show the human-inspired How-Where-What order performs best.

Main Findings

• Three Degradations task (denoising, dehazing, deraining): ClearAIR reaches the best average performance at 32.74 dB PSNR / 0.919 SSIM (31M parameters), compared with AdaIR at 32.69 / 0.918 and VLU-Net at 32.70 / 0.919. It achieves 31.08 dB / 0.981 SSIM on SOTS dehazing versus VLU-Net's 30.71 dB, and 38.61 dB / 0.984 on Rain100L.

• Five Degradations task (adding blur and low-light): ClearAIR achieves the highest average PSNR of 30.45 dB and SSIM of 0.916, with particularly strong deblurring on GoPro (29.67 dB / 0.887). It is described as slightly behind specialized methods on low-light enhancement and denoising; on LOLv1 it reports 22.83 dB / 0.846.

• All-Weather task (snow, rain/haze, raindrop): ClearAIR reaches an average of 33.90 dB PSNR / 0.946 SSIM, an average gain of 0.22 dB over Histoformer, with 37.79 dB / 0.967 on Snow100K-S, 32.53 dB / 0.932 on Snow100K-L, 32.45 dB / 0.941 on Outdoor-Rain, and 32.82 dB / 0.942 on RainDrop.

• Composited Degradation task (CDD-11): ClearAIR reports 29.34 dB PSNR / 0.886 SSIM, an average gain of 0.62 dB over OneRestore (28.72 / 0.882).

• Real-world generalization: On RainDS, ClearAIR reports 23.24 PSNR / 0.675 SSIM / 54.38 MUSIQ; on Snow100K-real, 61.57 MUSIQ and 2.852 NIQE. The paper states this is the best performance on both benchmarks among the compared methods.

• Single-task setting: ClearAIR outperforms ADFNet and FSNet by 0.18/0.30 dB and 0.34/0.96 dB in PSNR respectively at noise level 15 on BSD68 and Urban100. In dehazing and deraining it surpasses PromptIR by 0.60 dB and 1.48 dB, and beats DehazeFormer by 0.13 dB and DRSformer by 0.38 dB, reporting 31.91/0.978 on SOTS and 38.52/0.985 on Rain100L.

• Perception order matters: On Rain100L with 100K training iterations, Where-What-How yields 37.89 dB / 0.982, What-How-Where yields 38.04 / 0.983, and the paper's How-Where-What order yields 38.21 / 0.986.

• Component ablation: Removing individual components degrades performance — without the MLLM-IQA the result is 37.57 / 0.980, without the SGU 37.43 / 0.978, without the Task Identifier 37.52 / 0.980, and without ICRM 38.03 / 0.985, all below the full model's 38.21 / 0.986.

• Hyperparameter choices reported: total loss weight α = 0.25, ICRM loss weight γ initialized to 0.05, loss weights λ1 = 0.1 and λ2 = 0.05, AdamW with β1 = 0.9 and β2 = 0.999, learning rate 2×10⁻⁴, batch size 4, 300K training iterations, 256×256 random crops.

Methodology in Plain English

The authors build a single restoration network (Restormer as the backbone) and surround it with four modules arranged in a deliberate order that mirrors how they claim human vision works.

Step one — judge the whole image. A multimodal large language model based image quality assessment model called DeQA looks at the degraded image plus a text description and produces a quality representation. Instead of using the model's final output, the authors take the internal state just before the "quality level" token and convert it into a score embedding. An affine transformation then modulates the restoration features, so the network is conditioned on how bad the image looks overall.

Step two — find the regions that matter. A pre-trained Segment Anything Model produces a set of N binary masks, each highlighting a distinct region. These masks are combined with shallow image features through Mask Average Pooling: average the features inside each mask, then broadcast that average back across the region. This yields semantic-aware structural priors. To stay robust when masks are unreliable (which happens under severe degradation), a random subset of masks is dropped during training and merged into the background. The resulting semantic features talk to the backbone through semantic cross-attention.

Step three — figure out what is wrong locally. DA-CLIP acts as a task identifier, producing content embeddings and degradation embeddings (each 1×512). The degradation embedding is turned into a degradation prompt via a small MLP over learnable prompts and a softmax. A Degradation-Aware Module then uses cross-attention between the content embedding and the normalized backbone features, and the degradation prompt generates a degradation mask that modulates those features. This is how the model adapts restoration region by region instead of applying a one-size-fits-all treatment.

Step four — recover fine detail. The Internal Clue Reuse Mechanism takes the restored output image, applies a weak augmentation, then a strong augmentation, and computes the L2 distance between the weakly and strongly augmented versions as an extra training loss. The idea is that the model should learn to use the image's own internal statistics — its intrinsic structure — to sharpen detail, without any external supervision.

The whole thing is trained with a combined objective: L_total = L1 + α · L_inter, with α = 0.25.

Why This Matters

Impact on research. The paper pushes back on a common assumption in AiOIR — that one uniform processing strategy suffices. It argues that spatially non-uniform degradation and varying texture complexity mean restoration difficulty varies across an image, and it introduces a concrete way to test perceptual ordering rather than just asserting it. It also demonstrates that a multimodal LLM's internal quality representation can serve as a practical conditioning signal for a low-level restoration network, and that a no-reference metric style signal can substitute for physical degradation priors (the authors explicitly note that on SOTS, reaching 31.08 dB against VLU-Net's 30.71 dB shows "perceptual enhancement offsets the lack of physical priors").

Real-world applications:

• Smartphone and consumer camera pipelines — a single 31M-parameter model that handles noise, blur, low-light, haze and rain is attractive for on-device processing where running several separate networks is impractical.
• Autonomous driving and robotics — the All-Weather and real-world RainDS/Snow100K-real evaluations target exactly the conditions where perception systems degrade. One author affiliation is Horizon Robotics, an autonomous-driving company.
• Surveillance and outdoor camera networks — restoring footage captured in rain, snow, haze or at night, where ground-truth clean images are never available and no-reference metrics (MUSIQ, NIQE) are the only practical evaluation.
• Photo and video post-production / archival restoration — recovering detail from degraded or composite-degradation captures where the damage type is unknown or mixed.

Industry relevance. The model is compact (31M parameters, smaller than Perceive-IR's 42M and WeatherDiff's 83M) and requires no separate model per degradation type, which matters for deployment cost. The reliance on public pre-trained components (Restormer, DeQA, DA-CLIP, SAM) rather than a fully proprietary stack makes the approach reproducible, and the results on the CDD-11 composite benchmark — where real-world images often suffer several degradations at once — speak directly to production conditions.

Future Directions

• Replace the fixed pipeline with a perceptually adaptive one. The authors state their primary weakness is the inflexibility of the fixed perception order, evidenced by the sensitivity to ordering in the ablation. They propose integrating a Just Noticeable Difference (JND) mechanism (Chou and Li 1995) with a dynamic controller that routes the restoration based on perceptual thresholds and JND-weighted mechanisms that concentrate computation on perceptually critical regions.

• Improve region masking under extreme low visibility. The paper acknowledges the semantic guidance module "struggles with inaccurate region masking under extreme low-visibility conditions," an open problem for the SGU component.

• Make the detail-refinement augmentation perceptually informed. The authors describe ICRM's augmentation strategy as "perceptually uninformed," which they say can lead to suboptimal recovery of fine details — a clear invitation to design augmentation driven by perceptual principles rather than generic transforms.

• Robustness when degradation type and intensity vary spatially within one image. The limitation section frames this as the principal gap between the current design and real-world scenarios, and it is the natural test for whether the region-awareness machinery generalizes beyond the benchmark settings used here.

Target Audience

Researchers and graduate students working on image restoration, low-level vision, and multimodal vision-language models, particularly those interested in all-in-one or universal restoration frameworks. It is also relevant to engineers building camera or autonomous-driving image pipelines who need one compact model to handle many degradation types, and to readers interested in how cognitive-science concepts such as Human Visual Perception are translated into concrete network architectures and evaluated through ablation studies. Prerequisite knowledge of transformer restoration backbones, cross-attention, and standard restoration metrics is assumed.

Authors’ abstract

All-in-One Image Restoration (AiOIR) has advanced significantly, offering promising solutions for complex real-world degradations. However, most existing approaches rely heavily on degradation-specific representations, often resulting in oversmoothing and artifacts. To address this, we propose ClearAIR, a novel AiOIR framework inspired by Human Visual Perception (HVP) and designed with a hierarchical, coarse-to-fine restoration strategy. First, leveraging the global priority of early HVP, we employ a Multimodal Large Language Model (MLLM)-based Image Quality Assessment (IQA) model for overall evaluation. Unlike conventional IQA, our method integrates cross-modal understanding to more accurately characterize complex, composite degradations. Building upon this overall assessment, we then introduce a region awareness and task recognition pipeline. A semantic cross-attention, leveraging semantic guidance unit, first produces coarse semantic prompts. Guided by this regional context, a degradation-aware module implicitly captures region-specific degradation characteristics, enabling more precise local restoration. Finally, to recover fine details, we propose an internal clue reuse mechanism. It operates in a self-supervised manner to mine and leverage the intrinsic information of the image itself, substantially enhancing detail restoration. Experimental results show that ClearAIR achieves superior performance across diverse synthetic and real-world datasets.