Fast PINN Eigensolvers via Biconvex Reformulation

AI summary

Overview

• Research area: Scientific machine learning, specifically physics-informed neural networks (PINNs) applied to differential eigenvalue problems, plus convex optimization.
• Technical level: Advanced. The paper assumes familiarity with PINN loss formulations, differential operators, boundary conditions, linear least squares, and biconvex optimization.
• Scope: A reformulation of PINN eigensolvers that restricts training to the network's output layer, turning the problem into a biconvex program solved by alternating convex search (ACS) with closed-form updates, and demonstrated on three benchmark eigenvalue problems.

What This Paper Is About

Eigenvalue problems for differential operators mix a forward problem (finding the function for a given eigenvalue) with an inverse problem (finding the eigenvalue from a function). PINNs can solve these mesh-free, but they are typically orders of magnitude slower than classical numerical schemes because the PINN loss is highly non-convex and its terms conflict. The paper's goal is to make PINN eigensolvers fast and provably convergent by reformulating the loss as a biconvex problem and solving it with alternating convex search using analytically optimal updates.

Key Contributions

1. Biconvex PINN reformulation of the differential eigenvalue problem. Unlike prior PINN approaches, the reformulation accommodates varied boundary conditions and applies even when the eigenvalue appears explicitly in the boundary operator, such as Euler's buckling of a column with a free end.

2. Alternating convex search (ACS) for PINN optimization. Starting from a random initial eigenvalue estimate, ACS fixes one variable (eigenvalue or eigenfunction) and updates the other through analytic results. It converges rapidly and can be parallelized over a population of estimates to discover multiple eigenpairs simultaneously.

3. Demonstration on three problems. Euler's buckling, the Helmholtz equation on an L-shaped domain, and natural frequencies of a thin plate governed by a fourth-order biharmonic equation, with speedups exceeding 10x and up to 500x compared to gradient-based PINN training.

4. Provable convergence. Theorem 1 establishes that the loss sequence generated by ACS converges monotonically.

Main Findings

• Monotonic convergence is guaranteed and observed. Theorem 1 states that the ACS-generated loss sequence converges monotonically, because the squared loss terms are bounded below by zero and each analytically optimal update decreases the loss. Experiments agree with this prediction, while gradient descent using Adam oscillates due to conflicting loss terms.
• Large speedups. PINN-ACS is at least 10x faster than gradient descent, and for the pin-pin and fixed-fixed Euler buckling cases it achieves a 500x speedup while maintaining generally higher accuracy.
• Gradient descent struggles on higher modes. For the Helmholtz equation on the L-shaped domain, gradient descent results are omitted because of convergence difficulties within a reasonable time, especially for higher-order modes. In contrast, PINN-ACS achieves high accuracy for both low- and high-frequency eigenfunctions.
• Gradient descent fails to converge reliably for plate modes. For free vibrations of a thin plate, gradient descent results are excluded because it fails to converge reliably for higher modes; PINN-ACS shows smooth convergence of eigenvalues to ground truth.
• Accuracy verified against fine-mesh finite difference solutions for the Laplace operator, and PINN-ACS closely approximates that ground truth.
• ACS converges to the eigenpair nearest the initial estimate. This motivates running a population of random eigenvalue estimates in parallel, with already-discovered eigenfunctions inserted into the orthogonality term to steer later generations toward distinct eigenpairs.
• Modest hardware suffices. A shallow neural net with 500 neurons was enough for the 1D and 2D test problems. All codes were written in MATLAB R2023b without GPU acceleration and run on a single 11th Gen Intel Core i5-1135G7 processor with 8 GB RAM at approximately 2.40 GHz.
• Random Fourier features help. Cosine activations were used to learn high-frequency eigenfunctions, with the randomization bandwidth selected appropriately; larger bandwidths facilitate learning high-frequency patterns.

Methodology in Plain English

The researchers keep the usual PINN loss, which combines a differential-equation residual, a boundary-condition term, a reference-point term to block the trivial zero solution, and an orthogonality term against previously found eigenfunctions. Their key trick is to train only the network's final output layer and leave the hidden layers fixed at values chosen randomly or from a heuristic. Because the differential operators considered are linear, the physics residual becomes linear in the unknown output weights. The consequence is that the loss is convex (a linear least-squares problem) when the eigenvalue is held fixed, and convex in the eigenvalue when the output weights are held fixed, making the overall problem biconvex.

They then alternate: fix the eigenvalue, update the weights; fix the weights, update the eigenvalue. Each subproblem has a closed-form solution via the Moore-Penrose pseudoinverse applied to assembled design matrices, so no gradient descent is needed. When design matrices are ill-conditioned, a Tikhonov-regularized closed-form solution provides a numerically stable approximation that converges to the pseudoinverse as the regularization coefficient goes to zero. Since each step is optimal for its subproblem, the loss decreases monotonically and must converge. A reference-point constraint is deliberately preferred over a unit-norm constraint, because normalization would introduce nonlinear interactions among parameters and destroy the biconvex structure. Because ACS typically lands on the eigenpair closest to its starting eigenvalue, many strands are run in parallel from different starts to collect multiple eigenpairs, with found eigenfunctions feeding the orthogonality term in later generations.

Why This Matters

• Impact on research: It addresses a known weakness of PINNs for eigenvalue problems, replacing unreliable gradient-based training with a convergent, closed-form alternating scheme. It also widens the range of admissible boundary conditions beyond Dirichlet-only treatments, including cases where the eigenvalue appears in the boundary operator, and offers a general biconvex template for inverse problems where field functions and parameters are sought simultaneously.
• Real-world applications:
  • Critical buckling loads of columns and structural members under axial load.
  • Natural frequencies and vibration modes of thin plates.
  • Thermal response, stability, and natural-mode characterization of engineering systems, which the paper names as the motivation for eigenvalue problems.
  • Wave and resonance problems on irregular geometries such as L-shaped domains.
• Industry relevance: The method runs without GPU acceleration on ordinary hardware and in MATLAB, which lowers the barrier to adoption in mechanical, civil, and structural engineering workflows. Order-of-magnitude speedups matter for design exploration and for repeatedly solving eigenvalue problems across many parameter settings. The released code repository supports reproducibility and reuse.

Future Directions

• Extend the approach to nonlinear operators, with the Picard method proposed as one route: iteratively linearize at the differential-equation level using previous solution approximations so that alternating linear least-squares subproblems remain applicable.
• Scale up to larger-scale physics applications beyond the 1D and 2D test problems reported here.
• Automate the selection of the random feature bandwidth, for example by adapting the method cited as reference 26, since bandwidth choice affects learning of high-frequency eigenfunctions.
• Further exploit population-based parallel search to map entire spectra within prescribed eigenvalue search bounds, and broaden ACS into a general framework for inverse problems where unknown field functions and parameters are sought together.

Target Audience

Researchers and graduate students in scientific machine learning, computational mechanics, and numerical analysis who work on PINNs, mesh-free solvers, or eigenvalue problems. It is also relevant to structural and mechanical engineers interested in buckling, plate vibration, and modal analysis, and to practitioners who need fast eigenpair solutions on modest hardware. Readers without a background in convex optimization, PINN loss design, or differential eigenvalue problems will find the paper demanding.

Note: the paper does not report specific accuracy error values, wall-clock times in seconds, or dataset sizes; those quantities are not stated in the content provided. Work was supported by the Ramanujan Fellowship from the Anusandhan National Research Foundation, Government of India (Grant No. RJF/2022/000115), and code is released at the linked GitHub repository.

Authors’ abstract

Eigenvalue problems have a distinctive forward-inverse structure and are fundamental to characterizing a system's thermal response, stability, and natural modes. Physics-Informed Neural Networks (PINNs) offer a mesh-free alternative for solving such problems but are often orders of magnitude slower than classical numerical schemes. In this paper, we introduce a reformulated PINN approach that casts the search for eigenpairs as a biconvex optimization problem, enabling fast and provably convergent alternating convex search (ACS) over eigenvalues and eigenfunctions using analytically optimal updates. Numerical experiments show that PINN-ACS attains high accuracy with convergence speeds up to 500$\times$ faster than gradient-based PINN training. We release our codes at https://github.com/NeurIPS-ML4PS-2025/PINN_ACS_CODES.