From Evaluation to Design: Using Potential Energy Surface Smoothness Metrics to Guide Machine Learning Interatomic Potential Architectures

AI summary

Overview

Research area: Machine learning interatomic potentials (MLIPs) for computational chemistry and materials science, with a focus on evaluation metrics and architecture design.

Technical level: Advanced. The paper assumes familiarity with potential energy surfaces, molecular dynamics, DFT, and Transformer-style neural network architectures, though its main argument is conceptually accessible.

Scope: The paper proposes a low-cost benchmark (BSCT) and metric (FSD) for measuring the physical smoothness of MLIP-predicted potential energy surfaces, and uses it to iteratively redesign a Transformer-based MLIP backbone.

What This Paper Is About

Machine learning interatomic potentials are trained to minimize energy and force regression error, but a low error does not guarantee that the predicted potential energy surface (PES) has the smooth curvature of the true quantum mechanical surface. Non-smooth features—discontinuities, artificial minima, spurious forces—can destabilize molecular dynamics simulations even when standard test-set errors look good. The paper introduces an efficient benchmark called the Bond Smoothness Characterization Test (BSCT) to detect these artifacts, and then demonstrates how the metric it defines can be used as an "in-the-loop" signal to guide architectural design of a new MLIP called MinDScAIP.

Key Contributions

1. The Bond Smoothness Characterization Test (BSCT) — a benchmark that probes the PES by systematically stretching and compressing single bonds, a regime where the true PES is known to be smooth and where MLIP training data are sparse (far from equilibrium).

2. The Force Smoothness Deviation (FSD) metric — a logarithmic-derivative-based measure of how MLIP force norms change relative to DFT references along the bond scan, designed to detect artificial extrema and inflection points. Lower FSD indicates a smoother, more physically sound PES.

3. The BSCT-SPICE dataset — 485 molecules, each with 100 DFT single-point calculations at the ωB97M-D3(BJ)/def2-TZVPPD level of theory computed with Psi4, the same level of theory and code used by SPICE.

4. A differentiable k-nearest-neighbors algorithm (Diff-kNN) — a soft-ranking replacement for the non-differentiable hard ranking in standard kNN graph construction, allowing conservative force prediction via automatic differentiation. Combined with controllable Gaussian smearing and temperature-controlled attention, it is used to refine the MinDScAIP testbed architecture.

Main Findings

• FSD correlates with MD stability at far lower cost. Across three MinDScAIP models with FSD values of 97.4 (vanilla), 76.3 (weight decay), and 43.2 (smearing and temperature), the maximum kinetic temperature jump in a 10 fs window at 2000 K dropped from 9734 to 1904 to 490. Corresponding values at 3000 K were 813, 681, and 614; at 5000 K they were 597, 509, and 514. Computing FSD takes roughly 40 minutes on one A6000 GPU, versus roughly 40 hours for MD simulations.

• BSCT exposed a real artifact that motivated a design change. Inspecting a C11H12NO2 molecule from BSCT-SPICE, the authors found a spike in the ratio of MLIP to DFT force norms that traced back to rapidly changing attention scores, which led them to introduce temperature-controlled attention.

• Smoothness must be enforced at both local and non-local scales. Increasing the Gaussian smearing width smoothed compressed-bond regions (FSD compress 32.3 versus 65.2 for weight decay), while increasing attention temperature smoothed stretched-bond regions (FSD stretch 62 versus 55.1). Combining both gave the lowest overall FSD of 43.2, with compress 32.8 and stretch 38.1.

• Direct force regression is smooth but non-conservative. Direct-force prediction attained a low FSD of 71.8 (compress 69.4, stretch 42.8) but an NVE energy drift of 2.6e5 meV/atom, far worse than the conservative Diff-kNN model's 0.678.

• Standard kNN graphs break energy conservation. Gradient-based forces with standard kNN produced an NVE energy drift of 19.10 meV/atom, versus 0.678 for the same setup with Diff-kNN, confirming the non-conservative nature of hard kNN truncation.

• Bigger models are more accurate but less smooth far from equilibrium. Scaling from small (3.8M parameters) to medium (15M) to large (60M) improved test energy MAE from 0.23 to 0.12 to 0.09 meV/atom, but worsened FSD from 80.2 to 93.2 to 97.4.

• The smoothness-guided model compares favorably to established baselines. MinDScAIP with smoothness designs reached 0.12 meV/atom energy MAE and 2.94 meV/Å force MAE at FSD 43.2, versus MACE (4.7M, energy 0.79, forces 14.3, FSD 62.1) and GemNet-T (energy 0.30, forces 7.11, FSD 33.8 overall but 28.8 on stretch and 20.5 on compress).

• Smoothness gains transfer to near-equilibrium materials benchmarks. Training MinDScAIP-30M on MPTrj with weak, moderate, and strong smoothness settings gave Matbench Discovery F1 scores of 0.807, 0.811, and 0.817, while κ_SRME improved from 0.77 to 0.63 to 0.49, and RMSD from 0.092 to 0.089 to 0.088.

• A related model using Diff-kNN is a leaderboard leader. The paper states that as of May 2026, AllScAIP (which incorporates Diff-kNN) is the top-performing model on the OMol25 leaderboard among energy-conserving MLIPs, measured by multiple molecular physics-based evaluations.

Methodology in Plain English

The authors start from a physical intuition: if you take a molecule and slowly pull two fragments apart along a bond, the true energy curve should vary smoothly. So they build a benchmark that does exactly that—rigidly displacing one fragment relative to another along the bond axis—and runs DFT reference calculations for every displaced structure. Any jagged, spiky, or bumpy behavior in an MLIP's predictions along that scan is an artifact.

To turn this into a number, they track how the norm of the force (relative to the force at the energy minimum) changes as the displacement parameter increases. They compare the MLIP's relative rate of change to DFT's using a logarithmic derivative, then take the worst-case absolute difference over the whole scan. This is the FSD. The logarithmic form means the metric penalizes artifacts equally in high-force and low-force regions, and the ratio-to-DFT form lets it detect both artificial minima (small denominator) and inflection points (small numerator).

To show the metric is useful, they need an MLIP to test it on. They deliberately build an expressive, minimally constrained Transformer-style backbone called MinDScAIP, without embedding all the usual physical symmetries, so that any source of non-smoothness can be traced to a specific design choice. The backbone uses k-nearest-neighbor graphs arranged into a regular array, and alternating in-neighborhood and out-neighborhood attention inspired by Swin-Transformer's shifted windows.

Because hard kNN truncation is not differentiable and therefore breaks energy conservation, they replace the hard ranking with a sigmoid-based soft ranking and combine it with a smooth envelope function and an optional soft radius cutoff. They then identify three sources of nonlinearity—Gaussian smearing, activation functions, and the softmax in attention—and propose regularizing each with wider smearing, weight decay, and a temperature parameter in attention. Each change is ablated independently and measured three ways: energy/force MAE near equilibrium, FSD, and energy drift in NVE molecular dynamics.

Why This Matters

Impact on research: Standard MLIP evaluation relies on test-set energy and force errors, which the paper shows can miss physically serious defects. BSCT offers a fast, interpretable check that is reported to take minutes rather than hours, and the paper demonstrates that it can be used not just to screen finished models but to steer architecture design. It also contributes a differentiable kNN algorithm that makes sparse graph construction compatible with conservative force fields, and reports that this idea has already been adopted by a leaderboard-topping model.

Real-world applications:

• Drug discovery and molecular simulation, where unstable trajectories from nonphysical forces can waste compute or produce misleading results.
• Catalyst design, where accurate energies and stable relaxation of adsorbate systems are needed.
• Materials discovery and screening, where benchmarks like Matbench Discovery assess whether models can correctly predict structure stability and phonon behavior.
• Long-timescale or high-temperature simulations, where far-from-equilibrium bond breaking is common and training data are sparse.

Industry relevance: MLIPs are being commercialized as fast surrogates for DFT in chemistry and materials workflows. A cheap, actionable reliability metric lowers the cost of validating models before deployment, and the paper's in-the-loop design methodology gives model developers a concrete signal to optimize against. The release of the BSCT dataset and evaluation scripts on GitHub, and the Diff-kNN algorithm in the Fairchem package, makes the work directly usable by practitioners.

Future Directions

1. Build a BSCT benchmark for OMol25. The authors propose this explicitly, describing OMol25 as the largest and most diverse molecular dataset to date.

2. Test sensitivity to DFT setting mismatches. The paper notes that it has not yet systematically tested how FSD behaves when the MLIP was trained on a different functional or basis set than the BSCT reference.

3. Combine BSCT with multi-directional probes. Because BSCT relies on 1D rigid fragment displacements, the authors describe it as a necessary but not sufficient condition for full high-dimensional PES smoothness, and recommend pairing it with torsional and angular distortion benchmarks.

4. Validate refined models with a broader benchmark suite. The authors recommend that after BSCT-guided refinement, models be re-evaluated with energy conservation tests, thermal conductivity calculations, and other benchmarks to confirm final performance.

Target Audience

MLIP developers and machine learning researchers working on atomistic simulation will get the most from this paper, particularly those designing attention-based or graph-based architectures and looking for diagnostics beyond energy and force errors. Computational chemists and materials scientists who use MLIPs in molecular dynamics or screening pipelines will benefit from understanding what BSCT measures and when standard error metrics can be misleading. Benchmark designers may find the metric construction—comparing logarithmic derivatives of force norms against a DFT reference—useful as a template. Readers need familiarity with potential energy surfaces and neural network training to follow the architecture ablations, but the central argument about physical smoothness is stated clearly enough for a broader computational science audience.

Authors’ abstract

Machine Learning Interatomic Potentials (MLIPs) sometimes fail to reproduce the physical smoothness of the quantum potential energy surface (PES), leading to erroneous behavior in downstream simulations that standard energy and force regression evaluations can miss. Existing evaluations, such as microcanonical molecular dynamics (MD), are computationally expensive and primarily probe near-equilibrium states. To improve evaluation metrics for MLIPs, we introduce the Bond Smoothness Characterization Test (BSCT). This efficient benchmark probes the PES via controlled bond deformations and detects non-smoothness, including discontinuities, artificial minima, and spurious forces, both near and far from equilibrium. We show that BSCT correlates strongly with MD stability while requiring a fraction of the cost of MD. To demonstrate how BSCT can guide iterative model design, we utilize an unconstrained Transformer backbone as a testbed, illustrating how refinements such as a new differentiable $k$-nearest neighbors algorithm and temperature-controlled attention reduce artifacts identified by our metric. By optimizing model design systematically based on BSCT, the resulting MLIP simultaneously achieves a low conventional E/F regression error, stable MD simulations, and robust atomistic property predictions. Our results establish BSCT as both a validation metric for practitioners to assess MLIP utility and as an "in-the-loop" model design proxy that alerts MLIP developers to physical challenges that cannot be efficiently evaluated by current MLIP benchmarks. The BSCT dataset and evaluation are available on https://github.com/ryanliu30/bsct.git