The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators

AI summary

Overview

Research area: Scientific machine learning — specifically neural operators for solving partial differential equations (PDEs), and their ability to generalize across discretizations.

Technical level: Intermediate. The paper assumes some familiarity with Fourier analysis, sampling theory (Nyquist frequency), and neural operator architectures, but its core argument is accessible.

Scope: A systematic empirical evaluation of whether machine-learned operators (MLOs) can perform accurate inference at resolutions different from their training resolution ("zero-shot multi-resolution"), plus a proposed data-driven fix.

What This Paper Is About

Machine-learned operators such as the Fourier Neural Operator (FNO) are designed to represent continuous physical systems from discrete data, and they have been claimed to perform "zero-shot super-resolution" — training at a low resolution and then serving accurate inference at a higher resolution without additional high-resolution training data. This paper tests that claim directly by decomposing multi-resolution inference into two separate capabilities (interpolating across sampling rates and extrapolating to new frequency content) and evaluating whether trained models actually possess either one. The authors then study two proposed remedies and introduce a training-time alternative.

Key Contributions

1. A decomposition and empirical audit of multi-resolution generalization. The authors separate zero-shot multi-resolution inference into resolution interpolation (fixed frequency content, changing sampling rate) and information extrapolation (fixed sampling rate, changing frequency content), and show that FNOs fail at both. They conclude that changing resolution at inference time is effectively out-of-distribution inference.

2. Evaluation of two proposed corrective approaches. They test physics-informed optimization constraints (following Li et al. 2024b) and band-limited learning approaches — convolutional neural operators (CNO) and the Cross-Resolution Operator-Learning (CROP) pipeline — and find that neither enables reliable multi-resolution generalization.

3. A proposed multi-resolution training protocol. They train on data drawn from multiple resolutions simultaneously and show this substantially improves multi-resolution inference without a significant increase in training cost.

4. A data-cost/performance analysis of training-set composition. They profile which mixtures of resolutions are most efficient, finding that datasets composed mostly of cheap low-resolution data plus small amounts of expensive high-resolution data remain competitive across test resolutions.

Main Findings

• Zero-shot resolution interpolation fails. FNOs trained at a fixed Darcy resolution with a constant low-pass limit of 8f consistently assign incorrect high energy to frequencies greater than 8f at every other test resolution. The same failure appears for Burgers (low-pass limit 64f, resolutions 128, 256, 512, 1024) and Navier Stokes (low-pass limit 32f, resolutions 64, 128, 255, 510).

• Zero-shot information extrapolation fails. With sampling rate held constant and frequency content varied, residual spectra rise sharply in high frequencies. Errors increase as the test filter limit diverges from the training filter limit — including when the test data contains fewer frequencies than training data.

• Zero-shot super- and sub-resolution inference produces aliasing. In Figure 1, a model trained at resolution 16 and evaluated at resolutions 16, 32, 64, and 128 shows striation artifacts at resolution 128 that are visible in the predicted fields and in the energy spectra.

• Errors scale substantially with resolution mismatch. Test losses vary by 1x, 2x, and 10x across test resolutions for the Darcy, Burgers, and Navier Stokes datasets, respectively, indicating that models trained at one resolution do not achieve low loss across all test resolutions.

• Aliasing artifacts compound over time in time-dependent PDEs. For Navier Stokes, high-frequency artifacts in a model trained at resolution 255 and evaluated at resolution 510 become more prevalent over successive time steps.

• Physics-informed constraints do not rescue multi-resolution inference. Across weightings of the physics loss of w ∈ {0, 0.1, 0.25, 0.5}, the purely data-driven loss always outperformed any objective containing a physics constraint. Among physics-constrained objectives, lower physics weighting performed better. Models trained with physics constraints even failed to accurately fit their own training distribution and failed to generalize to both super- and sub-resolution data.

• Band-limited learning trades aliasing for an inability to predict high frequencies. CNO accurately learns the band-limited representation of its training data — its spectrum matches ground truth up to frequency 8f and then drops sharply by design — but it cannot predict frequencies beyond those seen in training. CROP+FNO fits lower frequencies well but struggles with higher frequencies across resolutions. The authors note that band-limiting a model's predictive capacity is counter to the goal of multi-resolution inference.

• Multi-resolution training improves multi-resolution inference. In pairwise (dual-resolution) training, test performance improves mainly at the two training resolutions, with no consistent gains at the other resolutions — confirming models perform best at resolutions they were trained on. Including data from all four resolutions improves test performance across all resolutions.

• Low-resolution-heavy mixtures are efficient. Datasets with compositions (0.7, 0.1, 0.1, 0.1) and (0.9, 0.5, 0.3, 0.2) across resolutions remained competitive across test resolutions, and "All Res." datasets generally form the Pareto front of average data size versus test loss.

Methodology in Plain English

The authors work with three standard PDEBench datasets — Darcy flow, Burgers' equation, and turbulent incompressible Navier Stokes — and primarily with the Fourier Neural Operator.

To isolate interpolation, they keep the frequency content of the data fixed (applying the same low-pass filter everywhere) while varying the sampling resolution. To isolate extrapolation, they do the reverse: hold the resolution fixed while varying how many frequencies remain after low-pass filtering. They then run the combined task — changing resolution and frequency content together — to mimic real super- and sub-resolution settings.

FNO hyperparameters were selected by grid search over learning rates {1e-2, 1e-3, 1e-4, 1e-5} and weight decays {1e-5, 1e-6, 1e-7}, with each model trained for 150 epochs. For the physics-informed experiments they used a dual objective combining a data loss with a physics loss, sweeping the physics weight over {0, 0.1, 0.25, 0.5}. For the band-limited experiments they trained CNO and CROP+FNO models using a separate grid search over learning rates {1e-3, 1e-4, 1e-5}.

For the proposed fix, they compose training sets by randomly sampling different proportions of data at each of n = 4 resolutions, first in pairwise combinations (yielding n(n−1)/2 dual-resolution sets) and then across all resolutions at once, including deliberately lopsided mixtures that favor cheap low-resolution data.

Diagnostics are spectral: they compare average energy spectra of predictions against ground truth and inspect normalized residual spectra, where lower residual energy at all frequencies is better.

All experiments ran on Python 3.10 with an NVIDIA A100-PCIE-40GB (driver 550.163.01, CUDA 12.4.131, PyTorch 2.9.1+cu126, neuraloperator 2.0.0).

Why This Matters

Impact on research. The paper challenges a widely repeated claim about mesh-invariant architectures — that a model trained at one resolution can be deployed "for free" at another. It reframes the failure not as a missing architectural feature but as a standard out-of-distribution generalization problem: the model simply never saw the relevant sampling rates or frequency content. That reframing matters because it directs effort away from architectural fixes (anti-aliasing activations, band-limiting) and toward data composition.

Real-world applications (grounded in the settings the paper discusses — PDE-governed simulation and modeling):

• Adaptive mesh refinement in fluid flow. Scientists increase resolution in regions requiring high accuracy (e.g., turbulence) and coarsen it elsewhere; a surrogate that cannot be queried accurately at a changed resolution undercuts this workflow.
• Cheap surrogate models of expensive PDE solvers. A model that is only valid at its training resolution has far less value as a drop-in replacement for numerical methods.
• Turbulent flow modeling. The Navier Stokes results are specifically in the turbulent regime, where unresolved high-frequency content is precisely the information a super-resolution model would need to supply.
• Time-dependent simulation rollouts. Because aliasing artifacts compound across time steps, errors in long trajectories are an especially serious concern.

Industry relevance. Engineering and scientific workflows that rely on field simulation — such as design iteration in aerospace or energy — benefit from surrogates that can be re-queried at multiple fidelities. The paper's practical guidance is that training must include the resolutions of interest, and that this can be done cheaply by skewing the training set toward low-resolution data. The specific industries and application domains beyond the fluid-dynamics examples are not enumerated in the paper.

Future Directions

1. Automated selection of multi-resolution training data. The authors explicitly name active learning strategies as a promising direction for choosing which resolutions and what proportions of data to include.

2. Optimizing the ratio across resolutions. The paper states that optimizing the proportions of data across all resolutions remains an open direction; the tested mixtures are not claimed to be optimal.

3. Better anti-aliasing approaches that do not band-limit predictive capacity. Since CNO and CROP prevent aliasing but cannot predict unseen high frequencies, a method that both avoids aliasing and extends the resolvable frequency range is still missing.

4. Generalization beyond training distributions as the core obstacle. The authors frame their findings within the broader observation that machine-learned models cannot typically generalize beyond their training data, raising the question of whether any architectural innovation can deliver resolution generalization without corresponding training data.

Target Audience

Researchers and practitioners in scientific machine learning and computational physics who work with neural operators, PDE surrogates, or mesh-invariant architectures. It is also relevant to numerical analysts evaluating whether learned surrogates can substitute for traditional solvers in multi-fidelity workflows, and to machine learning researchers interested in out-of-distribution generalization in continuous-signal settings. Some background in Fourier analysis and sampling theory is helpful for the diagnostic sections.

Authors’ abstract

A core challenge in scientific machine learning, and scientific computing more generally, is modeling continuous phenomena which (in practice) are represented discretely. Machine-learned operators (MLOs) have been introduced as a means to achieve this modeling goal, as this class of architecture can perform inference at arbitrary resolution. In this work, we evaluate whether this architectural innovation is sufficient to perform "zero-shot super-resolution," namely to enable a model to serve inference on higher-resolution data than that on which it was originally trained. We comprehensively evaluate both zero-shot sub-resolution and super-resolution (i.e., multi-resolution) inference in MLOs. We decouple multi-resolution inference into two key behaviors: 1) extrapolation to varying frequency information; and 2) interpolating across varying resolutions. We empirically demonstrate that MLOs fail to do both of these tasks in a zero-shot manner. Consequently, we find MLOs are not able to perform accurate inference at resolutions different from those on which they were trained, and instead they are brittle and susceptible to aliasing. To address these failure modes, we propose a simple, computationally-efficient, and data-driven multi-resolution training protocol that overcomes aliasing and that provides robust multi-resolution generalization.