Principled Operator Learning in Ocean Dynamics: The Role of Temporal Structure

AI summary

Overview

Research area: Machine learning for physical sciences, specifically neural operator learning applied to high-resolution ocean dynamics and sea-level prediction.

Technical level: Advanced. The paper assumes familiarity with neural operators, Fourier Neural Operators (FNOs), partial differential equations (PDEs), the shallow water equations, and spectral analysis.

Scope: A comparative study of the standard Fourier Neural Operator (FNO) against a modified variant (FNOtD) that adds temporal Fourier modes, evaluated on hourly sea-level prediction in the Baltic Sea.

What This Paper Is About

Neural operators are increasingly used to learn solutions to the PDEs that govern weather and ocean systems, but they struggle with long-term prediction stability and with respecting physical laws, particularly for high-frequency processes. This paper tackles that problem by changing how the integral kernels inside an FNO are parameterized: instead of mapping one time step to the next at a fixed interval, the modified model (FNOtD) learns the kernel jointly over space and time, letting it internalize the ocean's dispersion relation. The authors then test whether this architectural change improves physical fidelity and stability over the standard FNO.

Key Contributions

1. A temporally aware FNO variant (FNOtD). The integral kernels are parameterized jointly over space and time rather than solely over spatial frequency modes, so the operator can internalize the dispersion relation of ocean waves. The standard FNO is the special case with τ = 1, while FNOtD uses a fixed temporal window (a two-day window was found suitable for regional-scale wave propagation).

2. A controlled comparison of physical fidelity. Both models are trained under identical conditions — same dataset, same supervision target, same number of learnable parameters (FNO: 16.71 M; FNOtD: 16.19 M) — and evaluated on a sensitivity-to-initial-conditions experiment and on spectral bias.

3. Improved long-term stability and spectral behavior. FNOtD converges stably after roughly three weeks, whereas the standard FNO diverges, and FNOtD avoids the error propagation from high to low frequencies that the standard FNO exhibits at the first out-of-distribution event.

4. Competitive skill at lower cost. FNOtD achieves predictive skill comparable to a state-of-the-art numerical ocean model while requiring significantly lower computational cost, demonstrating principled zero-shot super-resolution by training on a low-resolution grid and evaluating on a high-resolution grid.

Main Findings

• Initial-condition sensitivity: With initial conditions rolled forward and iterative predictions, the standard FNO shows notable instability and diverges rather than converges after a few iterative steps, contrary to expectations. FNOtD exhibits stable convergence after approximately three weeks (Figure 1d), analogous to the spin-up phase in ocean modeling. The authors note a similar three-week period for reconstructing ocean circulation under atmospheric forcing with a deep learning model was observed by Hordoir et al. [16].

• Accuracy after spin-up: For November and December 2024, FNOtD yields an RMSE of 0.09 m (relative RMSE of 0.18) against the Nemo model, evaluated 30 days after the initial condition.

• Spectral bias: The standard FNO exhibits error propagation from high to low frequencies upon encountering the first out-of-distribution event, while FNOtD demonstrates robust stability in iterative predictions. In relative RMSE in the spectral domain, FNOtD outperforms the standard FNO and shows superior spectral stability in long-term predictions.

• Comparison with observations: Relative to geoid-referenced tide gauge observations, RMSE is 0.12 m for FNOtD and 0.08 m for Nemo outputs. Separately, the Nemo model's own sea-level fields exhibit RMSE of 0.08 m relative to tide gauge observations along the Baltic coast.

• Mode and parameter counts: The standard FNO uses truncated modes k_x^max = k_y^max = 16, while FNOtD uses k_x^max = k_y^max = 8 and ω^max = 4. The authors report that in their experiments the standard FNO with 8 modes demonstrates comparable performance to that with 16 modes.

• Comparison with numerical modeling: FNOtD provides competitive predictive skill relative to a state-of-the-art numerical ocean model while requiring significantly lower computational cost; the paper states FNO-based models can generate large ensemble forecasts at computational costs several orders of magnitude lower than state-of-the-art numerical models.

Methodology in Plain English

The underlying physics is the shallow water equations. The standard FNO learns a map from the current state at time t to the state at t + 1Δt, applying Fourier transforms only in the spatial dimensions through a 2D FFT with truncated modes. FNOtD instead learns a map from a window of past states, X̄(x,y,t−τ+1:t), to a window of future states, X(x,y,t+1:t+τ), with the complex-valued weight tensor R extended to include the temporal frequency dimension, so the kernel acts on space and time together.

Both architectures use four Fourier layers (L = 4), a point-wise encoder P, a point-wise decoder Q, and a point-wise multilayer perceptron M at each layer to enhance local nonlinear representation. Training minimizes the relative L2 error over the spatial dimensions with a land mask excluding land grid points. Although FNOtD can distribute the loss across the predicted lead-time window, the authors compute the loss using only the first lead time step (l = 1) so both models are optimized under identical supervision.

The case study is the Baltic Sea, described as a shallow, non-tidal, stratified, semi-enclosed basin in Northern Europe with complex morphology and bathymetry. Data comes from the Baltic Sea Physics Analysis and Forecast product (the "Nemo" model), providing hourly three-dimensional outputs at one nautical mile horizontal resolution with data assimilation from in situ temperature and salinity profiles, satellite-derived sea surface temperature, and sea ice concentration. Inputs are sea level, sea surface temperature, sea surface salinity, reference water depth, and atmospheric forcing (sea level pressure and 10-meter zonal and meridional wind components from ERA5); sea level is the only output variable.

The dataset spans November 2021 to December 2024, with training from November 2021 to September 2024 and testing from September to December 2024, separated by a ten-day temporal gap. Both models were trained for 50 epochs using the Adam optimizer with a cosine annealing learning rate schedule starting at 1×10⁻³, on a single NVIDIA A100-80GB GPU, for approximately six hours per model.

Training used a low-resolution version of the data obtained by bilinear-interpolation resampling of the high-resolution grid, with randomized resampling start points that effectively slide the coarse grid over the fine-resolution domain. Evaluation was on the high-resolution grid, producing hourly sea-level predictions in an autoregressive inference setting under prescribed atmospheric forcing; because the forcing comes from a reanalysis dataset, the authors call the task "prediction" rather than forecasting. A fixed time step of Δt = 6 hours is used throughout.

Why This Matters

Impact on research. The paper argues that establishing a form of continuous–discrete equivalence is essential for a neural architecture to faithfully learn an underlying operator, citing Bartolucci et al. [10]. By entangling space and time in the training of integral kernels, the work proposes a principled path toward operator learning that respects physical laws rather than relying on extensive training data, and it points toward AI-enabled ocean digital twins.

Real-world applications:

• Climate resilience and adaptation planning, supported by rapid and computationally efficient ensemble forecasting.
• Maritime safety, where accurate and timely ocean forecasting is essential.
• Early warning systems for extreme ocean events amplified by climate change, supporting decision-making.
• Sustainable marine resource management and operational ocean applications.

Industry relevance. Operational ocean systems currently rely on physics-based numerical models that face high computational costs, discretization requirements, and dependence on parameterized subgrid processes, constraining scalability, adaptability, and efficient ensemble forecasting. The paper positions neural operators as scalable alternatives capable of providing ensemble predictions at orders of magnitude lower computational cost, which matters for operational forecasting centers, climate services, and marine industries.

Future Directions

1. Validation beyond a single region. The approach has so far been evaluated only in one representative region (the Baltic Sea); the authors call for validation across diverse geographical settings and oceanic conditions.

2. Larger datasets and more variables. Extending training data, adding predictive variables, and further fine-tuning could improve accuracy and the ability to capture extreme events with better uncertainty quantification; variables from multiple depths are suggested, along with extension to three-dimensional emulation.

3. Training on sparse and irregular observations. The authors see potential for training or fine-tuning FNOtD directly on sparse (in time and space) observational data, such as in-situ or satellite observations, and note that a spatiotemporal training objective with Fourier-domain kernel parameterization offers a robust framework for constraining operator learning on limited, irregular data.

4. Addressing architectural costs. Compared with the standard FNO, FNOtD increases memory demand during training because of the temporal modes, and it is more prone to training plateaus, particularly when the dataset contains high noise levels that obscure physical relationships such as dispersion relations or when input variables inadequately represent the governing dynamics.

Target Audience

Researchers and practitioners working at the intersection of machine learning and the geophysical sciences: neural operator and scientific machine learning researchers, oceanographers and meteorologists interested in data-driven forecasting, and engineers building operational ocean or weather prediction systems. Readers need a working understanding of PDEs, Fourier analysis, and neural network architectures to follow the method, though the motivation and results are accessible to a broader Earth-science audience.

Authors’ abstract

Neural operators are becoming the default tools to learn solutions to governing partial differential equations (PDEs) in weather and ocean forecasting applications. Despite early promising achievements, significant challenges remain, including long-term prediction stability and adherence to physical laws, particularly for high-frequency processes. In this paper, we take a step toward addressing these challenges in high-resolution ocean prediction by incorporating temporal Fourier modes, demonstrating how this modification enhances physical fidelity. This study compares the standard Fourier Neural Operator (FNO) with its variant, FNOtD, which has been modified to internalize the dispersion relation while learning the solution operator for ocean PDEs. The results demonstrate that entangling space and time in the training of integral kernels enables the model to capture multiscale wave propagation and effectively learn ocean dynamics. FNOtD substantially improves long-term prediction stability and consistency with underlying physical dynamics in challenging high-frequency settings compared to the standard FNO. It also provides competitive predictive skill relative to a state-of-the-art numerical ocean model, while requiring significantly lower computational cost.