UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Dataset

AI summary

Overview

Research area: Computer Vision — ultra-high-resolution (UHR) text-to-image (T2I) generation and dataset construction for diffusion models.

Technical level: Intermediate. The paper assumes familiarity with diffusion models, latent-space generation, and basic signal-processing concepts (Discrete Fourier Transform, Discrete Wavelet Transform, gray-level co-occurrence matrices), though the core ideas are explained clearly enough for a motivated non-specialist.

Scope: The paper introduces a 100K-image UHR text-to-image dataset (UltraHR-100K), a frequency-aware post-training recipe (FAPT) that improves fine-detail synthesis in existing T2I models, and a new 4K evaluation benchmark (UltraHR-eval4K).

What This Paper Is About

Most text-to-image diffusion models are trained at roughly 1024×1024 resolution and degrade badly when pushed to 4K and beyond, producing oversmoothed textures and structural artifacts. Progress has been blocked by two problems: there is no large, openly available, high-quality UHR image–caption dataset, and existing training methods optimize for speed rather than for the fine-grained, high-frequency detail that UHR images demand.

The paper's goal is to fix both bottlenecks at once — by curating a rigorously filtered 100K-image dataset with long VLM-generated captions, and by adding a post-training stage that explicitly biases the model toward learning high-frequency detail.

Key Contributions

1. UltraHR-100K dataset. A 100K-scale UHR text-to-image dataset where every image has an average of width and height exceeding 3K pixels. Images were scraped (roughly 400K candidates), then filtered through a two-stage pipeline and a three-criterion intersection, and annotated with long, fine-grained captions generated by Gemini 2.0.

2. Frequency-aware post-training (FAPT). A two-component training strategy for adapting pretrained T2I models to UHR synthesis: Detail-Oriented Timestep Sampling (DOTS), which reweights which denoising timesteps the model learns from, and Soft-Weighting Frequency Regularization (SWFR), which adds a DFT-based loss term that softly emphasizes high-frequency reconstruction.

3. UltraHR-eval4K benchmark. A held-out 2,000-image evaluation set at 4096×4096 for measuring UHR generation, complementing the existing 195-image Aesthetic-Eval@4096. Evaluation spans FID, FID-patch, IS, IS-patch, long CLIP, and fine-grained CLIP scores, plus a human user study.

4. Empirical validation. The paper shows the dataset alone improves over strong baselines, and the training strategy alone further improves detail fidelity, with ablations isolating each component and the effect of data scale.

Main Findings

• Patch-level fidelity improves dramatically. On UltraHR-eval4K, the full method reaches an FID-patch of 15.795, versus 32.198 for Pixart-σ, 38.515 for Diffusion4K, and 38.795 for SANA. This is the paper's most striking result — it targets precisely the local texture quality that other methods lose at 4K.

• Global quality also improves. Overall FID drops to 31.748 (from 33.171 for Pixart-σ and 39.857 for Diffusion4K), and IS rises to 12.995, the highest among all compared methods.

• Semantic alignment is roughly preserved, not improved. CLIP scores stay competitive (31.82) but are marginally below the best baseline (Pixart-σ at 31.78 is close; SD3.5+BSRGAN at 31.75). The authors acknowledge a slight text–image alignment degradation as a limitation.

• Human raters strongly prefer the method. In a user study with 5 volunteers over 50 random cases, the proposed approach won 70% overall preference, 78% detail quality, 72% text–image alignment, and 70% preference against Pixart-σ, SANA, and Diffusion4K.

• Results generalize to a public benchmark. On the independent Aesthetic-Eval@4096 set, the method achieves the best FID (142.965) and FID-patch (24.008) compared with Pixart-σ, SANA, and Diffusion4K.

• Both training components contribute. Ablation shows DOTS alone improves FID from 33.99 to 32.57 and FID-patch from 20.93 to 19.95; adding SWFR brings FID to 31.74 and FID-patch to 15.79. The frequency regularization is responsible for most of the detail gain.

• Data scale matters. Training on a 15K subset with the full recipe yields an FID-patch of 18.42, noticeably worse than 15.79 on the full 100K set, confirming that the dataset's size is doing real work.

• The timestep distribution is tunable and matters. A Beta(2, 4) sampling distribution outperformed alternatives like (α=1, β=4), (α=2, β=5), (α=3, β=4), and (α=2, β=3), validating the intuition that biasing toward later denoising steps helps detail learning.

Methodology in Plain English

The work proceeds in two parallel tracks.

Building the dataset. The team scraped roughly 400,000 high-resolution images (minimum 3840×2160) from the web. Resolution alone does not make an image useful for training a generator, so they applied a blunt first-pass filter — Laplacian variance to reject blurry images and a Sobel edge detector to reject flat, textureless ones. Then they ran three independent quality rankings on the survivors: a Gray-Level Co-occurrence Matrix score to measure fine texture richness, Shannon entropy to measure how visually complex and varied the content is, and the LAION aesthetic predictor to score human-perceived appeal. They kept the top half on each dimension and took the intersection. Only images that scored well on all three survived, yielding roughly 104,000 images. Each was then captioned by Gemini 2.0, producing descriptions far longer and more specific than prior datasets.

Improving the training. They started from a pretrained T2I model (SANA) and ran two stages. Stage one is a standard fine-tune on UltraHR-100K using SD3-style logit-normal timestep sampling, for 4K iterations, to give the model a feel for UHR data. Stage two, the novel part, runs for 8K iterations with two modifications. The first, DOTS, changes which noise levels the model is asked to denoise: instead of sampling timesteps uniformly, it draws them from a Beta distribution shaped to favor the late denoising steps where high-frequency detail is synthesized. The second, SWFR, adds an extra loss term computed in the Fourier domain. It takes the Discrete Fourier Transform of both the model's prediction and the ground-truth target, then weights the squared difference so that high-frequency components are penalized more heavily — using a smooth exponential weighting function based on distance from the center of the frequency plane. This contrasts with Diffusion4K's approach, which uses the Discrete Wavelet Transform and produces coarser, block-based frequency separation.

The overall training loss is simply the ordinary diffusion loss plus a scaled version of this frequency loss.

Why This Matters

Impact on research. The paper attacks a data bottleneck that has held back UHR generation research. Prior to this, the largest open UHR T2I dataset was Aesthetic-4K at roughly 12,000 images; UltraHR-100K is about ten times larger with a stricter and more transparent selection procedure. The soft, DFT-based frequency regularization also offers a more elegant alternative to wavelet-based approaches, and the DOTS idea — deliberately biasing timestep sampling toward detail-synthesis steps — is a lightweight technique that could transfer to other high-resolution generative tasks.

Real-world applications:

• Digital art and concept design — artists need 4K+ outputs with crisp textures rather than upscaled, smeared approximations.
• Virtual content and game asset creation — UHR textures and environments generated directly rather than via expensive upscaling pipelines.
• Commercial advertising and product visualization — print and large-format display demand genuine UHR detail, not interpolated pixels.
• Portrait and fashion imagery — the authors flag this as an area still underserved by their dataset, but it is a major commercial target.

Industry relevance. The work is a collaboration with vivo Mobile Communication, which signals direct mobile-imaging interest — on-device or cloud-based generation of high-resolution imagery for phone cameras and content creation. More broadly, the method is a post-training recipe, meaning any organization with a strong base T2I model and access to UHR data can apply it without retraining from scratch, which lowers the barrier to building UHR-capable systems.

Future Directions

• Fix the semantic alignment trade-off. The frequency regularization improves detail but slightly degrades text–image alignment. The authors explicitly plan more balanced training strategies to close this gap — for instance, decoupling or adaptively weighting the frequency and diffusion losses.

• Expand portrait coverage. The dataset is skewed away from portraits, which limits UHR portrait generation. Adding high-quality UHR portrait data is a stated priority.

• Scale beyond one base model. All training experiments were conducted solely on SANA due to compute constraints. Validating FAPT on Transformer-based architectures like FLUX, SD3.5, or Pixart-σ remains open.

• Broaden the frequency-domain toolkit. The SWFR design uses a single exponential soft-weighting function with two hyperparameters. Whether other weighting profiles (perceptual, learned, or content-adaptive) yield better high-frequency reconstruction is unexplored.

• Move toward true 8K and beyond. The dataset floor is roughly 3K and the benchmark sits at 4096×4096. The same techniques could plausibly extend further, but memory and token-compression limits remain unaddressed.

Target Audience

This paper is most valuable to researchers and engineers working on high-resolution generative models — particularly those training or fine-tuning diffusion transformers and looking for practical recipes to push beyond 1024×1024. It is also directly useful to dataset builders, since the filtering pipeline (Laplacian, Sobel, GLCM, entropy, aesthetic scoring, intersection) is a reusable template for curating large multimodal datasets. Practitioners in mobile imaging, digital content creation, and applied generative AI at product companies will find the post-training approach actionable, as it adapts existing checkpoints rather than requiring new pretraining. Readers new to diffusion models will need some background, but the dataset construction section is accessible on its own.

Authors’ abstract

Ultra-high-resolution (UHR) text-to-image (T2I) generation has seen notable progress. However, two key challenges remain : 1) the absence of a large-scale high-quality UHR T2I dataset, and (2) the neglect of tailored training strategies for fine-grained detail synthesis in UHR scenarios. To tackle the first challenge, we introduce \textbf{UltraHR-100K}, a high-quality dataset of 100K UHR images with rich captions, offering diverse content and strong visual fidelity. Each image exceeds 3K resolution and is rigorously curated based on detail richness, content complexity, and aesthetic quality. To tackle the second challenge, we propose a frequency-aware post-training method that enhances fine-detail generation in T2I diffusion models. Specifically, we design (i) \textit{Detail-Oriented Timestep Sampling (DOTS)} to focus learning on detail-critical denoising steps, and (ii) \textit{Soft-Weighting Frequency Regularization (SWFR)}, which leverages Discrete Fourier Transform (DFT) to softly constrain frequency components, encouraging high-frequency detail preservation. Extensive experiments on our proposed UltraHR-eval4K benchmarks demonstrate that our approach significantly improves the fine-grained detail quality and overall fidelity of UHR image generation. The code is available at \href{https://github.com/NJU-PCALab/UltraHR-100k}{here}.