Precipitation nowcasting of satellite data using physically-aligned neural networks

AI summary

Overview

Research area: Machine learning for geoscience — specifically precipitation nowcasting (short-term rain forecasting) from geostationary satellite imagery.

Technical level: Advanced. The paper assumes familiarity with deep learning architectures (variational autoencoders, vision transformers, GANs, diffusion models), optical-flow methods, and meteorological verification metrics such as CSI and HSS.

Scope: The paper introduces TUPANN, a satellite-only nowcasting model that decomposes forecasting into physical submodules — motion inference, latent evolution, and advection — and benchmarks it against optical-flow, deep learning, and hybrid baselines across four cities and two satellite datasets.

What This Paper Is About

Most accurate short-term rain forecasts depend on dense ground-based weather-radar networks, which are missing or degraded across much of South America, Africa, and Asia — precisely the regions most exposed to climate extremes. This paper builds a nowcasting model that uses only geostationary satellite precipitation estimates, so it can work anywhere with satellite coverage. It also attacks a second problem: purely data-driven nowcasters often produce realistic-looking rain maps without physically consistent motion fields, which undermines forecasters' trust. TUPANN addresses both by explicitly supervising the network's internal motion estimates with a numerical optical-flow method.

Key Contributions

1. A physically-aligned, satellite-only architecture. TUPANN separates the forecast into three learned parts: a variational encoder–decoder (VED) that infers motion and intensity fields from recent imagery under direct optical-flow supervision, a lead-time-conditioned MaxViT transformer that evolves the latent state, and a differentiable advection operator ("warp") that reconstructs future frames. Unlike prior work that learns motion implicitly from a final-frame loss, the VED is supervised by numerical optical flow, producing smooth and interpretable motion fields.

2. Lead-time conditioning instead of recurrent decoding. The MaxViT transformer is conditioned on the lead time k via one-hot encoding and linear embedding, so a single network produces all lead times, reducing memory requirements compared with recurrent decoding.

3. Extensive multi-region evaluation. Experiments on GOES-16 RRQPE and IMERG data across up to four cities — Rio de Janeiro, Manaus, Miami, and La Paz — at lead times from 10 to 180 minutes, using CSI and HSS over 4–64 mm/h thresholds, compared against PySTEPS (LK and DARTS), Earthformer, NowcastNet, and CasCast. The paper also analyzes a GAN variant, cross-city transfer, and multi-city joint training.

4. Operational discussion. The paper reports runtime and latency considerations and outlines limitations and future directions for satellite-based nowcasting.

Main Findings

• Best or second-best skill in most settings. TUPANN consistently ranks first or second on CSI across regions and thresholds, with the largest advantages at high rain-rate thresholds.

• Strongest in Rio de Janeiro. TUPANN achieves the highest CSI at all thresholds in Rio, with a CSI–M of 0.259 (POOL1) and 0.277 (POOL4), ahead of NowcastNet (0.244 / 0.269) and Earthformer (0.237 / 0.222). PySTEPS (LK) reaches 0.165 / 0.169 and PySTEPS (DARTS) 0.166 / 0.166.

• Miami: TUPANN leads, CasCast is competitive. TUPANN reaches CSI–M of 0.169 / 0.187 in Miami, versus CasCast at 0.146 / 0.170, Earthformer at 0.141 / 0.126, and NowcastNet at 0.137 / 0.160.

• Manaus and La Paz: baselines win at low thresholds, TUPANN wins at high ones. In Manaus, Earthformer (0.276 / 0.256) and NowcastNet (0.253 / 0.278) obtain slightly better low-threshold CSI, but TUPANN leads at CSI₆₄ with 0.200 / 0.193 versus NowcastNet's 0.130 / 0.137, CasCast's 0.126 / 0.141, and Earthformer's 0.124 / 0.104. In La Paz, TUPANN reaches CSI–M of 0.314 / 0.317 versus Earthformer 0.303 / 0.270 and NowcastNet 0.291 / 0.301, and leads at CSI₆₄ with 0.232 / 0.239.

• Extreme events are rare and hard for all models. The 64 mm/h CSI values are small across every model, reflecting the rarity of such intense events, though TUPANN's scores remain the highest.

• Lead time behavior. TUPANN maintains the highest or second-highest mean CSI across all lead times, and its advantage over NowcastNet grows at early lead times.

• HSS behaves differently. HSS results are similar in direction, but by that metric Earthformer is much more competitive.

• Motion fields are the interpretability win. TUPANN's motion fields are smooth and closely resemble the numerical optical flow computed by DARTS, whereas the motion fields from NowcastNet's Evolution Network exhibit unrealistic patterns.

• A GAN head sharpens images but does not reliably improve scores. GAN-TUPANN produces significantly sharper images. In Rio de Janeiro it raises low-threshold CSI (CSI–M 0.265 / 0.290 versus TUPANN's 0.259 / 0.277), but in Miami gains are non-existent or negative (0.152 / 0.174 versus 0.169 / 0.187), and it degrades in Manaus (0.274 / 0.285 versus 0.290 / 0.293) and La Paz (0.306 / 0.312 versus 0.314 / 0.317).

• Cross-city transfer is modestly degraded. Training on Rio and evaluating elsewhere (TUPANN–Rio) lowers CSI in Manaus (0.236 / 0.249 versus 0.290 / 0.293) and La Paz (0.279 / 0.288 versus 0.314 / 0.317). In Miami the Rio-trained model performs comparably at low thresholds but better at high ones (CSI₃₂ 0.138 / 0.152 and CSI₆₄ 0.085 / 0.103 versus 0.135 / 0.141 and 0.079 / 0.094). Overall degradation is at most 20%, and TUPANN–Rio still matches or beats baselines trained on the target city.

• Multi-city training helps. A jointly trained TUPANN–Multicity (including Toronto) improves CSI in most regions: Rio CSI–M 0.271 / 0.286, Miami 0.178 / 0.190, Manaus 0.291 / 0.296, La Paz 0.324 / 0.325.

• IMERG generalization holds. On IMERG for Rio de Janeiro, TUPANN achieves the best CSI across all thresholds without pooling (POOL1). With pooling (POOL4), generative models (NowcastNet, GAN-TUPANN) slightly outperform at low thresholds, but TUPANN remains competitive and leads at higher thresholds. CSI–M versus lead time shows TUPANN superior at most lead times, with small gaps only at 150 minutes.

• Near real-time operation. Inference typically takes under two seconds per forecast for all 18 lead times, run on a single NVIDIA A100 GPU.

Methodology in Plain English

The model takes a sequence of past satellite rain-rate images and predicts future ones. Rather than learning the whole problem end-to-end as a black box, the researchers split it into pieces that mirror how the atmosphere actually moves rain:

1. Estimate motion and intensity change. A variational encoder–decoder looks at recent frames and outputs two things: a motion field (which way rain is moving) and an intensity correction (how much rain is growing or fading). Crucially, the motion field is trained to match the output of a classical optical-flow algorithm — DARTS for GOES-16, Lucas–Kanade for IMERG — so the network's internal "wind" is directly supervised rather than inferred from the final image loss alone. The loss combines ℓ₁ losses on intensity and motion, a cosine-similarity term on motion, and a Kullback–Leibler divergence term keeping the latent space regular.

2. Evolve the state forward. Once the first latent representation is learned, its weights are frozen and a MaxViT transformer takes over. The transformer is told which lead time it is predicting via one-hot encoding, so one network handles all lead times at once.

3. Advect to reconstruct. A fixed, differentiable warp operator — following NowcastNet — uses the predicted motion and intensity fields to push the previous frame forward into the next one. Training then minimizes the ℓ₁ loss between the warped frame and the actual observed frame, assuming the previous frame equals the ground truth to avoid expensive recursive loss computation.

Data. The primary source is GOES-16 RRQPE, which covers the Americas every 10 minutes at 2 km resolution with roughly 5 minutes of latency, from January 2020 to December 2023. Rain events are defined as contiguous periods exceeding a threshold, sampled uniformly at random and split 70% training, 15% validation, 15% test. Four 512 km × 512 km subregions are studied: Rio de Janeiro (subtropical, coastal), La Paz (high-altitude), Manaus (rainforest), and Miami (subtropical monsoon). IMERG — 30-minute cadence, 10 km resolution, about 3.5 months latency — is used as an independent validation source over a 2560 km × 2560 km area around Rio de Janeiro.

Evaluation. CSI and HSS at 4, 8, 16, 32, and 64 mm/h thresholds, computed pixel-wise (POOL1) and as max-pooled scores over 4 × 4 blocks (POOL4), with CSI–M and HSS–M as the mean across thresholds.

Why This Matters

Impact on research. The paper shows that injecting physical structure — specifically, supervising learned motion fields against a numerical optical-flow solver — can improve forecast skill rather than trade it away, while also making the model's internals legible. It also demonstrates that a satellite-only model can compete with radar-trained architectures, opening a research path for regions without radar infrastructure, and that multi-city training transfers across very different climates.

Real-world applications:

• Flood and landslide early warning in cities with recurrent disasters and unreliable radar, such as Rio de Janeiro, where topographic blocking and limited infrastructure constrain radar coverage.
• Global nowcasting coverage for underserved regions across South America, Africa, and Asia that geostationary satellites already observe.
• Operational forecaster support, since smooth, interpretable motion fields that align with numerical optical flow are easier to trust and sanity-check than black-box outputs.
• Data-sparse-agency deployment, given that the model runs on a single NVIDIA A100 GPU with inference under two seconds per forecast, which is practical for smaller meteorological services.

Industry relevance. Insurance, agriculture, aviation, logistics, and emergency management all depend on short-horizon rainfall forecasts. A satellite-only model lowers the infrastructure barrier to offering those forecasts commercially in markets where radar networks do not exist. The transferability results also matter for vendors who want one model serving many geographies rather than retraining per city.

Future Directions

1. Closing the skill gap with radar-trained and diffusion-based models at low rain-rate thresholds. Earthformer and NowcastNet still edge out TUPANN at low thresholds in Manaus and La Paz, and on pooled IMERG data generative models lead at low thresholds. Understanding and narrowing that gap is a natural next step.

2. Improving extreme-event prediction more broadly. CSI at 64 mm/h remains small for every model tested, so rare heavy-rain regimes are still an open problem.

3. Making generative sharpening worthwhile. GAN-TUPANN improves visual realism but helps CSI only in Rio de Janeiro and hurts it elsewhere, so the trade-off between sharpness and verification skill is unresolved.

4. Extending to more regions and data sources. The multi-city experiment already includes Toronto alongside the four study regions; further cross-city and cross-satellite transfer studies, plus the limitations and future work discussed in the paper's Section 6, would test how universal the approach really is.

Target Audience

This paper is most useful to machine learning researchers working on spatiotemporal forecasting and physics-informed neural networks, and to meteorologists and operational nowcasting practitioners interested in satellite-based alternatives to radar. Climate-risk analysts, disaster-management agencies, and teams building weather products for regions with sparse ground infrastructure will also find the transferability and latency results directly actionable. Readers without a background in deep learning or meteorological verification metrics will find the architecture and results sections demanding.

Authors’ abstract

Accurate short-term precipitation forecasts predominantly rely on dense weather-radar networks, limiting operational value in places most exposed to climate extremes. We present TUPANN (Transferable and Universal Physics-Aligned Nowcasting Network), a satellite-only model trained on GOES-16 RRQPE. Unlike most deep learning models for nowcasting, TUPANN decomposes the forecast into physically meaningful components: a variational encoder-decoder infers motion and intensity fields from recent imagery under optical-flow supervision, a lead-time-conditioned MaxViT evolves the latent state, and a differentiable advection operator reconstructs future frames. We evaluate TUPANN on both GOES-16 and IMERG data, in up to four distinct climates (Rio de Janeiro, Manaus, Miami, La Paz) at 10-180min lead times using the CSI and HSS metrics over 4-64 mm/h thresholds. Comparisons against optical-flow, deep learning and hybrid baselines show that TUPANN achieves the best or second-best skill in most settings, with pronounced gains at higher thresholds. Training on multiple cities further improves performance, while cross-city experiments show modest degradation and occasional gains for rare heavy-rain regimes. The model produces smooth, interpretable motion fields aligned with numerical optical flow and runs in near real time due to the low latency of GOES-16. These results indicate that physically aligned learning can provide nowcasts that are skillful, transferable and global.