SynWeather: Weather Observation Data Synthesis across Multiple Regions and Variables via a General Diffusion Transformer

AI summary

Overview

• Research area: Computer vision and generative modeling applied to meteorological remote sensing — specifically, synthesizing missing weather observation variables from satellite imagery.
• Technical level: Advanced. The paper assumes familiarity with diffusion models, latent autoencoders, transformers, and satellite remote sensing terminology.
• Scope: The paper introduces SynWeather, a dataset spanning four regions and four weather variables, and SynWeatherDiff, a text-prompt-driven Diffusion Transformer baseline for unified weather observation data synthesis.

What This Paper Is About

Weather observing instruments have intrinsic limitations — radar coverage is sparse in complex terrain or economically limited regions, and visible satellite imagery is unavailable at night — so raw weather data contain temporal and spatial gaps. Existing synthesis datasets and models are typically restricted to a single region and a single variable, and they rely on deterministic modeling, which produces over-smoothed outputs that miss high-intensity weather. This paper builds a unified multi-region, multi-variable dataset (SynWeather) and a general probabilistic model (SynWeatherDiff) that can fill those gaps across regions and variables within one framework.

Key Contributions

1. SynWeather dataset: The first standardized dataset supporting unified multi-region and multi-variable weather observation data synthesis, covering four regions (Continental United States, Europe, East Asia, and Tropical Cyclone regions) and four variables (Composite Radar Reflectivity, Hourly Precipitation, Visible Light, and Microwave Brightness Temperature), with infrared observations from six geostationary satellites (GOES-16/17/18, Himawari-8/9, Meteosat-11) as inputs.
2. SynWeatherDiff model: The first generative, probabilistic framework that unifies many variables and regions under a single text-prompt-driven interface, built on a Diffusion Transformer with a shared latent autoencoder.
3. Task suite and benchmark: Six standard synthesis tasks plus one out-of-distribution task (precipitation synthesis over East Asia), with comparisons against task-specific models (SRViT, Deep-STEP, TomoPE), a general model (WeatherGFM), and adapted deep learning baselines (UNet, ViT).
4. Systematic ablations: Analyses of task sampling ratios, input channel groups (SWIR, WV, LWIR, GAS), out-of-distribution generalization, and few-shot transfer.

Main Findings

• Dataset composition and scale: SynWeather provides high-resolution observations for four variables across four regions, including 142k CR samples over CONUS (target source: GREMLIN CONUS3 Dataset, 3 km), 20k CONUS precipitation samples (MRMS, 1 km), 372k Europe visible light samples (Meteosat-11, 3 km), 25k Europe precipitation samples (EURADCLIM, 1 km), 503k East Asia visible light samples (Himawari-8, 2 km), 15k East Asia precipitation samples (GPM, 10 km), and 9k MWBT samples over Tropical Cyclone regions (AMSR-2 and GMI via TC PRIMED, spanning 2015–2023).
• Strong universal capability: SynWeatherDiff outperforms the existing generalist model WeatherGFM across most tasks and can generate different variables from the same satellite input — for example, both CR and precipitation over CONUS, and both visible light and precipitation over Europe — which the authors attribute to the task-specific text prompts.
• CR synthesis (CONUS): SynWeatherDiff achieves RMSE 2.820, CSI/25 0.382, CSI/35 0.158, and CSI/40 0.101, compared with WeatherGFM's RMSE 3.124, CSI/25 0.366, CSI/35 0.166, CSI/40 0.086.
• Precipitation synthesis: Over CONUS, SynWeatherDiff records RMSE 0.976, CSI/2 0.312, CSI/5 0.223, and CSI/15 0.113; over Europe it records RMSE 0.569, CSI/2 0.084, and CSI/5 0.079, the latter clearly above the next-best CSI/5 of 0.044 (ViT).
• Visible light is a caveat: UNet achieves better results than SynWeatherDiff on visible light synthesis — for East Asia, SSIM 0.917, PSNR 21.67, CSI/50 0.711 versus SynWeatherDiff's SSIM 0.868, PSNR 19.79, CSI/50 0.690. The authors explain that UNet operates directly in pixel space, while visible light contains abundant high-frequency details that are hard to reconstruct through the autoencoder.
• MWBT synthesis (Tropical Cyclone regions): SynWeatherDiff attains RMSE 4.456, SSIM 0.837, PSNR 22.33, LPIPS 0.254, and CSI/300 0.795, the best RMSE, SSIM, PSNR, LPIPS, and CSI/300 among the compared models.
• Fine-grained and high-value detail reconstruction: Visual results show Unet, ViT, and WeatherGFM produce fewer individual cells, overlook weak signals, and merge small weather cells into larger areas; SynWeatherDiff restores the number and distribution of scattered cells and more accurately captures the location and shape of intensity centers.
• Out-of-distribution generalization: On East Asia precipitation synthesis (an OOD task), SynWeatherDiff achieves RMSE 1.150, SSIM 0.771, PSNR 29.80, CSI/2 0.235, and CSI/5 0.108, surpassing ViT (RMSE 1.233, SSIM 0.749, CSI/2 0.232, CSI/5 0.103) and UNet (RMSE 1.383, SSIM 0.671, CSI/2 0.161, CSI/5 0.076).
• Task sampling ratios matter: Setting CR as the primary task at a 0.5 sampling ratio improves CONUS CR (CSI/25 0.403, CSI/35 0.187, RMSE 2

Authors’ abstract

With the advancement of meteorological instruments, abundant data has become available. Current approaches are typically focus on single-variable, single-region tasks and primarily rely on deterministic modeling. This limits unified synthesis across variables and regions, overlooks cross-variable complementarity and often leads to over-smoothed results. To address above challenges, we introduce SynWeather, the first dataset designed for Unified Multi-region and Multi-variable Weather Observation Data Synthesis. SynWeather covers four representative regions: the Continental United States, Europe, East Asia, and Tropical Cyclone regions, as well as provides high-resolution observations of key weather variables, including Composite Radar Reflectivity, Hourly Precipitation, Visible Light, and Microwave Brightness Temperature. In addition, we introduce SynWeatherDiff, a general and probabilistic weather synthesis model built upon the Diffusion Transformer framework to address the over-smoothed problem. Experiments on the SynWeather dataset demonstrate the effectiveness of our network compared with both task-specific and general models.