Degradation-Aware Metric Prompting for Hyperspectral Image Restoration

AI summary

Authors: Binfeng Wang, Di Wang, Haonan Guo, Ying Fu, Jing Zhang (Beijing Institute of Technology; Wuhan University; Zhongguancun Academy) — arXiv:2512.20251v3 [cs.CV], published 2025-12-23. Code: https://github.com/MiliLab/DAMP

Overview

• Research area: Computer vision / hyperspectral image (HSI) restoration, specifically unified restoration across multiple degradation types.
• Technical level: Advanced. The paper assumes familiarity with attention mechanisms, mixture-of-experts routing, Fourier-domain descriptors, and PSNR/SSIM/SAM evaluation.
• One-sentence scope: The paper proposes DAMP, a unified HSI restoration framework that describes image degradations with six interpretable spatial-spectral metrics and uses those metrics as a router to select specialized experts inside a mixture-of-experts network.

What This Paper Is About

Hyperspectral images capture hundreds of contiguous spectral bands, but real acquisition suffers from noise, blur, missing bands, and mixtures of such degradations. Existing unified restoration models either require explicit degradation labels (e.g., predefined degradation types or text prompts) that are impractical to obtain, or they rely on opaque black-box embeddings that overfit the training distribution and generalize poorly to unseen corruptions. The goal of this work is to restore many degradation types within one model while keeping the degradation description interpretable and transferable to unseen scenarios.

Key Contributions

1. DAMP framework. A unified HSI restoration framework that removes the need for explicit degradation priors while still providing an interpretable characterization of degradation, handling diverse, mixed, and unseen corruptions in a single model.
2. Degradation Prompts (DP). A compact multi-dimensional representation that continuously quantifies degradation severity using spatial-spectral metrics, capturing both degradation severity and shared characteristics across restoration tasks.
3. Degradation-Adaptive Mixture-of-Experts (DAMoE). An architecture in which the DP vector acts as a degradation-aware gating router and Spatial-Spectral Adaptive Modules (SSAMs) serve as specialized experts, enabling adaptive restoration under complex and unknown degradations.
4. Empirical validation across domains. Experiments on both natural-scene and remote sensing HSI datasets showing state-of-the-art results plus zero-shot generalization to unseen tasks, with a public code release.

Main Findings

• Metric feasibility (pilot study). On a pilot dataset of 1,000 degraded HSIs, three representative metrics — High-Frequency Energy Ratio (HFER), Spatial Texture Uniformity (STU), and Spectral Curvature Mean (SCM) — formed distinct clusters per degradation type in t-SNE visualization and were strongly discriminative under a Random Forest classifier. Distribution analysis also showed that distinct degradation types (e.g., regions missing and band missing) can overlap on specific metrics (e.g., SCM), indicating shared underlying characteristics.
• Final DP composition. From a pool of 25 candidate metrics spanning entropy, gradient, frequency, and correlation statistics, a three-stage filter (interpretability, spatial-spectral coverage, discriminability via Random Forest feature importance) produced six metrics: High-Frequency Energy Ratio, Spatial Texture Uniformity, Spectral Curvature Mean, Spectral Curvature Standard Deviation, Gradient Standard Deviation, and Spatial Correlation Coefficient.
• Known-task performance. On Gaussian deblurring (radius 9, 15), DAMP reached 33.84 PSNR / 0.929 SSIM / 4.246 SAM on PaviaU, 52.84 / 0.998 / 0.508 on ARAD, and 46.39 / 0.984 / 2.112 on HyRank. On super-resolution (scale 2, 4): 46.33 / 0.980 / 0.882 on ICVL, 44.01 / 0.981 / 0.866 on ARAD, and 49.77 / 0.987 / 2.165 on HyRank. On inpainting (mask rate 0.7, 0.8, 0.9): 29.41 / 0.797 / 13.163 on PaviaC, 33.62 / 0.711 / 10.982 on Xiong'an, and 38.91 / 0.933 / 6.788 on Chikusei. On Gaussian denoising (sigma 30, 50, 70): 42.86 / 0.974 / 2.229 on ICVL, 41.47 / 0.967 / 2.668 on ARAD, and 26.11 / 0.634 / 28.165 on PaviaC. On completion (rate 0.1, 0.2, 0.3): 50.85 / 0.901 / 12.862 on PaviaU, 63.49 / 1.000 / 0.422 on ARAD, and 56.61 / 0.932 / 10.281 on Xiong'an. Average results were 51.97 / 0.990 / 0.952 (ICVL), 51.43 / 0.989 / 0.936 (ARAD), and 39.42 / 0.851 / 10.113 (remote sensing data).
• Zero-shot generalization. Without any fine-tuning and without knowledge of the blur kernel or noise parameter, DAMP achieved 31.05 dB PSNR / 0.899 SSIM for motion deblurring (0.52 dB above the strongest baseline, PromptIR) and 24.08 dB PSNR / 0.538 SSIM for Poisson denoising (a 2.10 dB improvement over the previous best) on the CAVE dataset.
• Component ablation. Starting from a baseline without DP and SSAM at 45.82 dB PSNR / 0.976 SSIM, adding DP raised average PSNR by 4.20 dB and SSIM by 0.010 (to 50.02 / 0.986); further adding SSAM gave an additional 1.41 dB PSNR and 0.003 SSIM, reaching 51.43 dB / 0.989.
• Routing strategy ablation. DP-based routing (51.43 dB / 0.989) outperformed frequency-based routing (47.72 / 0.983) by 3.71 dB / 0.006, degradation-type routing (46.27 / 0.982) by 5.16 dB / 0.007, and implicit prompt routing (46.81 / 0.982) by 4.62 dB / 0.007.
• Efficiency. DAMP requires 313.80 G FLOPs, 14.91 M parameters, and 133.18 ms, versus PromptIR (573.47 G, 26.13 M, 207.10 ms) and MoCE-IR (365.57 G, 23.67 M, 191.81 ms). MP-HSIR is reported at 894.61 G, 13.88 M, 653.15 ms. The overhead of metric computation and the DP router is stated as under 0.2 ms.
• Metric-severity correlation. Seven of the nine analyzed (degradation type, metric) pairs achieved a Pearson correlation coefficient above 0.4, supporting the assumed monotonic link between the proposed metrics and degradation severity.
• Spectral fidelity. Normalized digital number error across spectral bands indicates that SSAM balances spatial and spectral information and better preserves intrinsic spectral characteristics of HSIs.

Methodology in Plain English

The pipeline starts by computing six numbers directly from the degraded input image. These numbers describe spatial quality (how much high-frequency energy remains, how uniform texture is, how variable gradients are, how spatially correlated pixels are) and spectral quality (how curved and variable spectral curves are). Together they form the Degradation Prompt, a vector that continuously describes how bad the degradation is rather than naming its type.

The restoration network is a hierarchical U-shaped encoder–decoder. The DP vector is embedded and injected into every level of the decoder. There, a mixture-of-experts layer computes gating scores from both a global average pooling of the features and the DP vector, adds Gaussian noise during training to encourage exploration and load balancing, keeps only the top-k experts, and combines their outputs in a weighted sum. Each expert is a Spatial-Spectral Adaptive Module with two parallel branches: one using window-based multi-head self-attention for spatial structure (ℰs) and one using 1D convolutions for inter-spectral correlation (ℰc), mixed by expert-specific learnable coefficients λs and λc that sum to 1. Because these coefficients are fixed per expert rather than predicted per input, experts specialize during training into "Spatial Experts" and "Spectral Experts," and the router assembles the right combination for each degradation.

A residual block fuses the original input with the decoder output to produce the restored image.

Training used PyTorch on a single NVIDIA GeForce RTX 4090 GPU, AdamW with β1 = 0.9 and β2 = 0.999, an initial learning rate of 1×10⁻⁴, an L1 objective, and a batch size of 4, for 3,000 epochs on natural-scene HSIs and 1,500 epochs on remote sensing HSIs. Eight datasets were used: ARAD, ICVL, and CAVE for natural scenes, and Xiong'an, Chikusei, PaviaC, PaviaU, and HyRank for remote sensing. Following Wu et al. (2025b), 1,000 images were cropped into 128×128×31 patches for training with 100 non-overlapping images held out for testing; for remote sensing, 80% of each image was cropped into 128×128×100 patches for fine-tuning and 20% reserved for testing. Separate models were trained per domain because of the domain gap. Evaluation used PSNR and SSIM for spatial fidelity and SAM for spectral accuracy, comparing against 5 unified restoration methods (PromptIR, InstructIR, DFPIR, MoCE-IR, MP-HSIR) and 12 task-specific methods (Stripformer, DeepRFT, Loformer, ESSA, SRDNet, VolFormer, SSDL, Restormer, PFGIN, SERT, SST, HCANet).

Why This Matters

The paper reframes "what degradation is this?" as "how degraded is this, along measurable physical axes?" That shift means the model does not need external degradation labels at deployment, and its conditioning signal is not tied to the training distribution of degradation types — which is the mechanism behind the reported zero-shot gains on motion blur and Poisson noise.

Real-world applications:

• Environmental monitoring, where degraded spectral signatures reduce the reliability of land, water, and atmosphere observation.
• Precision agriculture, where spectral discriminability supports crop and material identification.
• Remote sensing pipelines, including the five remote sensing HSI datasets (Xiong'an, Chikusei, PaviaC, PaviaU, HyRank) used in the study.
• Downstream analysis tasks such as HSI classification and change detection, which depend on clean spectral features.

Industry relevance: the efficiency comparison shows DAMP at 313.80 G FLOPs and 133.18 ms with 14.91 M parameters, lower than PromptIR, DFPIR, MoCE-IR, and MP-HSIR in this comparison, and the paper frames this as practical for resource-constrained HSI restoration. The public code release supports reuse. The authors also state an intended positive societal impact in environmental monitoring, precision agriculture, and remote sensing, with no specific negative consequences highlighted.

Future Directions

• Broader degradation coverage. The authors note that performance on more diverse real-world HSI corruptions remains unvalidated, and that extremely rare or physically distinct degradations may require additional complementary metrics. They plan to expand the degradation benchmark.
• Dynamic metric weighting. Rather than a fixed six-dimensional DP vector, the authors propose dynamically weighting metrics to handle more diverse corruptions.
• Cross-domain unified model. Optimal performance currently requires separate training for natural and remote sensing domains; a single cross-domain restoration model without retraining remains an open challenge, to be addressed with domain-adaptive techniques.
• Understanding the limits of interpretable conditioning. How much of the zero-shot advantage comes from the interpretability of the metrics themselves versus the routing architecture is not isolated in the reported ablations.

Target Audience

Researchers and practitioners in hyperspectral imaging, image restoration, and remote sensing who work on multi-task or "all-in-one" restoration models. It is also relevant to readers interested in interpretable conditioning signals and mixture-of-experts routing in vision transformers, and to engineers building restoration front-ends for spectral classification, change detection, or environmental monitoring pipelines. Because the method section uses formal notation and assumes background in attention mechanisms and expert routing, the paper is best suited to readers with an intermediate-to-advanced machine learning background.

Authors’ abstract

Unified hyperspectral image (HSI) restoration aims to recover diverse degradations within a single model. However, current methods often rely on impractical explicit priors or opaque black-box representations that overfit to training distributions, hampering generalization to unseen scenarios. To bridge this gap, we propose Degradation-Aware Metric Prompting (DAMP), a novel framework that characterizes multi-dimensional degradations through interpretable spatial-spectral metrics. These metrics serve as Degradation Prompts (DP), enabling the model to capture shared characteristics across tasks and adapt to unknown corruptions. Central to our framework is the Degradation-Adaptive Mixture-of-Experts (DAMoE), where Spatial-Spectral Adaptive Modules (SSAMs) serve as experts that utilize learnable fusion coefficients to specialize in distinct degradation degrees. By using DP as a gating router, DAMoE dynamically activates specialized experts tailored to the specific degradation profile. Extensive experiments on natural and remote sensing HSI datasets demonstrate that DAMP achieves state-of-the-art performance and exhibits exceptional zero-shot generalization on unseen restoration tasks. Code is publicly available at \href{DAMP}{https://github.com/MiliLab/DAMP}.