Sumudu Neural Operator for ODEs and PDEs

AI summary

Overview

Research area: Machine learning for scientific computing, specifically neural operators — neural network architectures that learn mappings between infinite-dimensional function spaces in order to approximate solution operators for differential equations.

Technical level: Intermediate. The paper assumes familiarity with the Fourier Neural Operator (FNO) and the Laplace Neural Operator (LNO), integral transforms, and the relative L² error metric standard in this subfield.

Scope: The paper introduces the Sumudu Neural Operator (SNO), which replaces the Fourier transform used in FNO with the Sumudu transform implemented via polynomial regression, and benchmarks it against FNO and LNO on six ODE settings and five PDE tasks, plus zero-shot super-resolution and runtime comparisons.

What This Paper Is About

Neural operators like FNO and LNO learn to approximate solutions of differential equations by transforming inputs into a frequency-like or pole-residue space and parameterizing weights there. FNO struggles with transient responses and undamped signals, which motivated LNO's pole-residue formulation. This paper asks whether a third, much less studied integral transform — the Sumudu transform — can serve as the basis for a neural operator, exploiting a known relationship between the polynomial expansions of Sumudu transform pairs. The goal is to test whether a polynomial-based transform decomposition yields accuracy or efficiency advantages on specific classes of ODEs and PDEs.

Key Contributions

1. A new operator architecture. The authors define the Sumudu Neural Operator (SNO), which lifts an input function to a higher dimension with a shallow linear network, applies a Sumudu transform to decompose the input into polynomial coefficients, parameterizes weights in Sumudu space with a linear transform as bias, then normalizes, applies an activation, and projects back with a second network.

2. A polynomial formulation of the Sumudu transform. The paper exploits the property that the Sumudu transform of a function approximates its polynomial expansion up to a factorial scaling factor (sum of n!·c_n·t^n), allowing the transform to be computed via polynomial regression with a Vandermonde matrix solved using the Moore-Penrose inverse.

3. A theoretical complexity argument. The authors compare the FFT Cooley-Tukey algorithm's O(n log n) time complexity against polynomial regression's O(d²n), arguing that for low degree and sufficiently large n the degree parameter becomes negligible and complexity scales as O(n) — theoretically more efficient than both FNO and LNO.

4. A combined ODE/PDE benchmark with runtime and super-resolution evaluation. SNO is evaluated on Duffing Oscillator, Lorenz System, and Driven Pendulum (ODEs) and Euler-Bernoulli Beam, Burger's Equation, Diffusion, Diffusion-Reaction, and Brusselator (PDEs), with comparisons to FNO and LNO, runtime measurements, and zero-shot super-resolution plots.

Main Findings

• SNO beats FNO on every PDE task reported. On Euler-Bernoulli Beam the errors are SNO 0.002100 vs FNO 1.518; on Burger SNO 0.3331 vs FNO 3.389; on Diffusion SNO 0.0006779 vs FNO 1.246; on Diffusion-Reaction SNO 0.1185 vs FNO 7.76775652; on Brusselator SNO 0.1093 vs FNO 0.1283.

• SNO achieves the lowest error of the three operators on two PDEs. It records the best score on the Euler-Bernoulli Beam (0.002100 vs LNO 0.007900) and the Diffusion equation (0.0006779 vs LNO 0.001300), which the authors describe as outperforming LNO by factors of 3.8× and 1.9× respectively. On Burger's equation SNO (0.3331) outperforms FNO but LNO is lower at 0.05310. On Diffusion-Reaction SNO (0.1185) and LNO (0.1123) are close, while on Brusselator LNO is highest at 0.1834.

• SNO is weaker than both baselines on the ODE tasks. For Duffing Oscillator with c=0 the errors are SNO 0.9125, FNO 0.8761, LNO 0.2777; with c=0.5, SNO 0.7147, FNO 0.3343, LNO 0.1372. For Pendulum with c=0: SNO 0.9100, FNO 0.5150, LNO 0.1961; with c=0.5: SNO 0.7386, FNO 0.1893, LNO 0.1332. For the Lorenz System with ρ=10: SNO 0.5565, FNO 0.4129, LNO 0.3024; with ρ=5: SNO 0.1025, FNO 0.06110, LNO 0.008700.

• Error patterns track damping and equation type. The authors report that SNO achieves lower error on PDEs than ODEs and does better on damped systems than undamped ones, visible in the errors. They attribute the ODE gap to SNO lacking LNO's pole-residue formulation.

• Runtime results are problem-dependent. Measured in seconds, SNO records 1913.45 (Duffing c=0), 2291.55 (Duffing c=0.5), 261.03 (Diffusion), 84.21 (Diffusion-Reaction), and 2241.85 (Brusselator), versus FNO at 46.33, 49.27, 31.47, 30.84, and 562.41, and LNO at 2193.89, 3244.41, 3099.65, 2532.03, and 805.80. The paper reports an 11.9× speedup over LNO on Diffusion and a 30× speedup on Diffusion-Reaction, while SNO lags on Brusselator. The authors caution that these times are not wholly representative of actual efficiencies due to differences in code and optimization, and note FNO is fast partly because of its simple implementation and the wide research into the Fourier transform.

• Zero-shot super-resolution shows resolution-robust behavior. Using weights from the end of training, SNO was evaluated on test data and compared against ground truth and base resolutions. For ODEs the ground truth and base resolution were interpolated for comparison; for PDEs everything was compared at base resolution. SNO performed better on damped tasks than undamped ones, with only slight differences across resolutions. The corresponding heatmaps are in Appendix A.

Methodology in Plain English

The pipeline starts by taking an input function f(t) and "lifting" it into a higher-dimensional representation using a shallow linear neural network L. Next, instead of using a Fourier transform as FNO does, the model decomposes this lifted signal with a Sumudu transform. Because the Sumudu transform of a function mirrors its polynomial expansion up to factorial scaling, the authors compute it by fitting a polynomial: they build a Vandermonde matrix from the input points, solve it (via the Moore-Penrose inverse) by minimizing squared error to obtain coefficients a₀ through aₙ, and use those coefficients to represent the signal in Sumudu space. Learned weights are parameterized in that space, a local linear transform W acts as a bias, and the result is normalized, passed through a nonlinear activation, and projected back down to the original dimension by a second network P.

Training used the Adam optimizer with a learning rate of 1×10⁻³ and batch size of 20 for 1000 epochs, except the Pendulum which used 1200 epochs and batch size 50. The loss function was the relative L² error between predicted and ground-truth solutions, following Li et al. (2021), and all input data were normalized first. Accuracy was measured by relative L² error on out-of-distribution functions, and performance was compared against LNO and FNO on each task. Runtimes were compared using torch synchronization, and zero-shot super-resolution was tested using trained weights without retraining.

Why This Matters

Impact on research. The paper tests a low-studied integral transform as a neural operator design principle, extending a line of work that has moved from Fourier (FNO) to Laplace with pole-residue decomposition (LNO). It reports that a polynomial-based transform decomposition can outperform FNO on PDEs and, on two PDE tasks, outperform LNO — evidence that transform choice should be matched to problem structure rather than treated as fixed.

Real-world applications (as suggested by the problem classes tested):

• Structural and mechanical engineering, where beam bending problems such as the Euler-Bernoulli Beam arise.
• Fluid dynamics and transport modeling, covered by Burger's equation and diffusion problems.
• Chemical kinetics and reaction modeling, covered by Diffusion-Reaction and Brusselator systems.
• Dynamical system forecasting for oscillators and chaotic systems, covered by the Duffing, Pendulum, and Lorenz tasks.

Industry relevance. Neural operators are marketed on efficiency relative to traditional numerical solvers, which can be time and resource-intensive. The paper's runtime measurements — including an 11.9× speedup over LNO on Diffusion and 30× on Diffusion-Reaction — speak to that claim, though the authors are explicit that implementation differences confound the comparison. The project also releases code at github.com/Lapuleu/SNO for reproduction.

Future Directions

• Increase architectural complexity. The authors state that observed limitations in accuracy and behavior highlight the need to increase the complexity of the current architecture.
• Improve polynomial regression. They call for more accurate and efficient methods of polynomial regression, which is the core transform computation in SNO.
• Mixed interpolation for transient responses. A footnote reports prior experimentation in which replacing polynomial regression with a Taylor series approximation centered at a point of importance gave significantly higher accuracy near that point but less average accuracy, suggesting that a mixed interpolation technique might improve both transient-state and steady-state responses.
• Deeper efficiency analysis and problem-structure study. The authors ask for a more nuanced time-efficiency analysis and deeper investigation of the relationship between problem structure and transform choice, since computational efficiency proved problem-dependent. They also suggest further testing on the specific PDE types where SNO performs well.

Target Audience

This paper is most useful to machine learning researchers working on neural operators and scientific machine learning, particularly those comparing integral-transform-based architectures such as FNO and LNO. It also suits applied mathematicians and computational scientists interested in the Sumudu transform or polynomial-based spectral decomposition, and engineers working on beam mechanics, diffusion, and reaction-diffusion problems who are curious whether neural surrogates can compete on their problem classes. Readers seeking a mature, thoroughly optimized method should note the authors' own framing of these as preliminary findings with recognized runtime and accuracy limitations.

Authors’ abstract

We introduce the Sumudu Neural Operator (SNO), a neural operator rooted in the properties of the Sumudu Transform. We leverage the relationship between the polynomial expansions of transform pairs to decompose the input space as coefficients, which are then transformed into the Sumudu Space, where the neural operator is parameterized. We evaluate the operator in ODEs (Duffing Oscillator, Lorenz System, and Driven Pendulum) and PDEs (Euler-Bernoulli Beam, Burger's Equation, Diffusion, Diffusion-Reaction, and Brusselator). SNO achieves superior performance to FNO on PDEs and demonstrates competitive accuracy with LNO on several PDE tasks, including the lowest error on the Euler-Bernoulli Beam and Diffusion Equation. Additionally, we apply zero-shot super-resolution to the PDE tasks to observe the model's capability of obtaining higher quality data from low-quality samples. These preliminary findings suggest promise for the Sumudu Transform as a neural operator design, particularly for certain classes of PDEs.