Efficient Perceptual Image Super Resolution: AIM 2025 Study and Benchmark

AI summary

Overview

Research area: Computer Vision — efficient perceptual single-image super-resolution (SR), presented as an AIM 2025 challenge study and benchmark.

Technical level: Intermediate. The concepts (PSNR vs. perceptual quality, GAN-based SR, FLOP and parameter budgets, no-reference image quality metrics) are accessible to readers familiar with deep learning basics, while the metric aggregation and per-class analysis assume some background in image restoration evaluation.

Scope (one sentence): The paper defines an efficiency-constrained benchmark for perceptual 4× super-resolution (≤5M parameters and ≤2000 GFLOPs at 960×540 input), evaluates submitted methods on a new 500-image 4K dataset (PSR4K) plus five existing perceptual SR benchmarks, and shows that a compact model can beat Real-ESRGAN on all evaluated benchmarks.

What This Paper Is About

Efficient super-resolution research has largely optimized for distortion metrics such as PSNR, while the methods that look best to humans (GAN- and diffusion-based perceptual SR) are computationally heavy. This paper sets up a study and benchmark that asks: can you match or exceed Real-ESRGAN's perceptual quality while staying under strict compute and size limits? To answer it, the organizers created PSR4K, a 500-image 4K test set with private degradations and no ground-truth high-resolution images, mirroring realistic deployment conditions.

Key Contributions

1. A new benchmark dataset, PSR4K: 500 low-resolution test images at 960×540, organized into ten semantic categories (animals, architecture, art, food, nature, objects, portraits, sports, text, urban scenes), with five degradation types per category applied through a pipeline kept private. At ×4 scaling the outputs are 3840×2160 (UHD).
2. Fixed efficiency constraints and a relative scoring scheme: a maximum of 5M parameters (approximately 30% of Real-ESRGAN) and 2000 GFLOPs (approximately 22% of Real-ESRGAN) at 960×540 input, with a single score aggregating PI, CLIPIQA and MANIQA relative to the Real-ESRGAN baseline.
3. A comparative evaluation of three submitted solutions (VPEG, MiAlgo, IPIU) plus reference methods (Real-ESRGAN, BSRGAN, and the NTIRE 2024 ESR entries SPAN and R2NET) on PSR4K and five external perceptual SR benchmarks.
4. A fine-grained per-class and stability analysis of the PSR4K categories, including mean, median and standard deviation per metric, plus a qualitative study showing that current perceptual metrics fail to penalize artifacts and hallucinations.

Main Findings

• VPEG ranked first overall. On the PSR4K test set it achieved PI 3.1205, CLIPIQA 0.6544 and MANIQA 0.3919 with 3.1684M parameters and 1631.0842 GFLOPs, yielding a Score of 2.2015 versus 2.7182 for the Real-ESRGAN baseline (16.6980M parameters, 9293.9416 GFLOPs, PI 4.1442, CLIPIQA 0.5302, MANIQA 0.3283).
• VPEG's gains over Real-ESRGAN: PI reduced by 24.7%, CLIPIQA increased by 23.4%, and MANIQA increased by 19.4%, while using roughly 19.0% of the parameters and roughly 17.6% of the FLOPs.
• MiAlgo ranked second with PI 3.7420, CLIPIQA 0.5999, MANIQA 0.3662 and Score 2.4512, at 3.5214M parameters and 1987.3922 GFLOPs — PI reduced by 9.7%, CLIPIQA up 13.2%, MANIQA up 11.5% over baseline, using about 21.1% of parameters and 21.4% of FLOPs.
• IPIU ranked third and was extremely lightweight (0.2762M parameters, 132.1431 GFLOPs — roughly 1.65% of baseline parameters and 1.42% of FLOPs), but its distortion-oriented EFDN design led to weaker perceptual scores (PI 6.0676, Score 3.9536).
• PSNR-oriented efficient models scored poorly perceptually. SPAN (0.1507M, 77.7870 GFLOPs, Score 3.9571) and R2NET (0.2148M, 103.2455 GFLOPs, Score 4.3401) were efficient but had low perceptual scores, with R2NET's PI of 6.6837 being the worst in the table.
• BSRGAN slightly beat Real-ESRGAN on PSR4K in PI (4.2112 vs 4.1442 for Real-ESRGAN — note BSRGAN's value is higher, i.e. worse; the paper reports BSRGAN slightly outperforming in PI and CLIPIQA 0.5779 vs 0.5302) at identical reported efficiency (16.6980M, 9293.9416 GFLOPs).
• Cross-dataset generalization: VPEG achieved the best PI values on all tested datasets, reducing PI by approximately 26.5% and 30% on PIPAL and RealSR respectively. MiAlgo led in CLIPIQA and MANIQA (except on RealSRSet), with improvements of roughly 34% and 28% on PIPAL.
• Benchmark-specific score gaps (Table 7): VPEG scored better than MiAlgo on every dataset — DIV2K-LSDIR 2.5024 vs 2.5383 (difference 0.0359), PIPAL 2.1294 vs 2.2554 (0.1260), RealSR 2.4335 vs 2.5840 (0.1505), Real47 2.4712 vs 2.5407 (0.0695), RealSRSet 2.3747 vs 2.4783 (0.1036). The average gap on these benchmarks was 0.0971, compared with 0.2497 on PSR4K.
• Per-class trends: architecture was the most favorable category for all methods, followed by animals and nature. Food was consistently the worst-performing category. Art, portraits and objects tracked the overall average. Text performed reasonably well on CLIPIQA and MANIQA but was penalized by PI.
• VPEG was the most stable method across classes: standard deviation of 0.2486 for PI (versus 0.5269 for Real-ESRGAN) and 0.0166 for CLIPIQA (versus 0.0389). Its MANIQA standard deviation (0.0307) was slightly higher than Real-ESRGAN's (0.0294) and BSRGAN's (0.0288).
• Qualitative analysis exposed a metric limitation: perceptual metrics assigned top scores to VPEG and MiAlgo outputs on RealSR and Real47 crops that showed noticeable artifacts, while Real-ESRGAN produced a clean Real47 reconstruction. The paper states that no standard quantitative measure for hallucinations or artifacts is available.
• Runtime: measured on an NVIDIA H100 80GB HBM3 GPU, VPEG required less than half the runtime of Real-ESRGAN across all evaluated datasets except RealSR. The paper cautions that these measurements should be interpreted carefully.

Methodology in Plain English

The organizers posed a constrained optimization problem. Starting from Real-ESRGAN as the reference model for perceptual quality, teams had to produce a super-resolution model that uses no more than 5M parameters and 2000 GFLOPs for a 960×540 input — about a third of the parameters and roughly a fifth of the compute of the reference. Training data was left to the teams; DIV2K, Flickr2K, LSDIR and OST were the most commonly used, and all teams based their degradation pipeline on the Real-ESRGAN pipeline at ×4 down-scaling.

Evaluation avoided the usual PSNR/SSIM comparison. Because PSR4K ships only low-resolution images with no matching high-resolution originals, scoring relies on no-reference quality assessment: PI (lower is better), CLIPIQA and MANIQA (higher is better). The metrics were picked for their high correlation with human judgment, and they were combined into a single per-method score relative to Real-ESRGAN using weightings of 0.5 for PI, 0.25 for CLIPIQA and 0.25 for MANIQA. All metrics were computed with the PIQA package.

During the eight-week training phase, participants validated locally on RealSRSet and Real47 because the Codabench platform lacked GPU support for the no-reference metrics. The test phase lasted one day: teams submitted code, a factsheet and output images for those two validation sets, and the organizers re-ran the code under controlled conditions to verify the results.

The three winning methods take different routes. VPEG shrinks the SAFMN architecture (the original exceeded the budget at 2888.23 GFLOPs) into "SAFMN-L", keeping 16 blocks but cutting channels from 128 to 96, then fine-tunes it with perceptual, LDL, GAN and AESOP losses in three stages. MiAlgo builds "TinyESRGAN" by cutting ESRGAN down to 17 RRDB blocks with 32 intermediate channels and a growth rate of 18, an approximately 79% compute reduction, and trains in two stages with MSE plus LPIPS first, then GAN loss. IPIU reuses EFDN, an extremely small edge-enhanced distillation-style network trained with L1 loss on Flickr2K for 15 hours.

Why This Matters

Impact on research: The study shows that perceptual quality and strict efficiency are not mutually exclusive, opening a direction the paper describes as relatively unexplored. It also establishes a reproducible benchmark design (private degradations, no ground-truth, no-reference metrics) and demonstrates a methodological weakness in current perceptual metrics, which reward outputs that contain visible artifacts. Several efficiency techniques common in the field — knowledge distillation, pruning, and re-parameterization (used only by IPIU) — were not used by the top methods, leaving room for improvement.

Real-world applications:

• Mobile and edge image enhancement, where the parameter and FLOP budgets in this benchmark are close to realistic device limits; the paper explicitly cites low latency and limited hardware resources on mobile and edge devices as the motivation.
• Upscaling user photos and phone-camera output to UHD (3840×2160), the resolution implied by the ×4 scaling of the 960×540 PSR4K inputs.
• Video streaming and media pipelines on GPUs, where VPEG's runtimes on an H100 (for example 35.7096 ms versus Real-ESRGAN's 118.6400 ms on DIV2K-LSDIR) indicate headroom for faster or higher-throughput processing.
• Restoration of archival or compressed footage, given that the benchmarks include JPEG compression artifacts and complex real-world degradations such as those in RealSRSet and Real47.

Industry relevance: The competing entries came from a mix of academic labs and industry — the VPEG team is from Nanjing University of Science and Technology, MiAlgo from Xiaomi Inc. and Capital Normal University, and IPIU from Xidian University — and the challenge sponsors were AI Witchlabs and the University of Würzburg Computer Vision Lab, with support from the Alexander von Humboldt Foundation. The workbench of constrained models with published code and datasets is directly usable as a comparison point for teams building deployable SR products.

Future Directions

• New perceptual metrics robust to artifacts and hallucinations. The paper's own qualitative section shows top-scoring outputs can be visually flawed; it explicitly calls for metrics that penalize such failures.
• Explore the efficiency-perception trade-off. The authors note that studied methods tend to produce visual artifacts despite better metric scores, raising the question of whether a genuine trade-off exists under tight compute budgets.
• Apply untapped efficiency techniques. Knowledge distillation, pruning, and broader use of re-parameterization were not exploited by the winning entries, suggesting a clear path to better quality within the same constraints.
• Extend the benchmark and analysis. PSR4K is presented as a step toward a 4K SR benchmark and includes a per-class breakdown; the paper's own category analysis (with food, sports, urban scenes and text posing problems) points toward further work on data coverage and metric behavior in those domains.

Target Audience

Researchers and engineers working on image restoration, super-resolution and generative models; practitioners deploying SR on mobile, embedded or GPU-constrained hardware; challenge organizers and benchmark designers interested in no-reference evaluation protocols; and graduate students looking for a compact, well-documented entry point into the perception–distortion tradeoff and efficient perceptual SR.

Authors’ abstract

This paper presents a comprehensive study and benchmark on Efficient Perceptual Super-Resolution (EPSR). While significant progress has been made in efficient PSNR-oriented super resolution, approaches focusing on perceptual quality metrics remain relatively inefficient. Motivated by this gap, we aim to replicate or improve the perceptual results of Real-ESRGAN while meeting strict efficiency constraints: a maximum of 5M parameters and 2000 GFLOPs, calculated for an input size of 960x540 pixels. The proposed solutions were evaluated on a novel dataset consisting of 500 test images of 4K resolution, each degraded using multiple degradation types, without providing the original high-quality counterparts. This design aims to reflect realistic deployment conditions and serves as a diverse and challenging benchmark. The top-performing approach manages to outperform Real-ESRGAN across all benchmark datasets, demonstrating the potential of efficient methods in the perceptual domain. This paper establishes the modern baselines for efficient perceptual super resolution.