Optimizing the Optimizer: Language Models Discover Faster Molecular Relaxation

AI summary

Overview

Research area: Computational and quantum chemistry (physics.chem-ph), specifically molecular geometry optimization, combined with automated algorithm discovery by large language models ("autoresearch").

Technical level: Advanced. The paper assumes familiarity with force evaluations, potential energy surfaces, quasi-Newton optimizers, model Hessians, internal coordinates, and density-functional theory (DFT).

Scope: The paper describes a gated agentic search loop that rewrites the source code of a molecular geometry optimizer to reduce the number of force evaluations required per relaxation, producing two new optimizers (AutoSella-F and AutoSella-A) and evaluating them on held-out molecules and on potentials never used during the search.

What This Paper Is About

Molecular geometry optimization is expensive because every optimization step needs one force evaluation, and at the DFT level that evaluation dominates the wall time. The authors ask whether a language model can autonomously improve the best existing open-source optimizer through an automated research loop, rather than through traditional manual algorithm design. The goal is an optimizer that reaches the same convergence criteria in fewer force calls, and whose improvements hold up on unseen molecules and on potentials it never saw during the search.

Key Contributions

1. A gated autoresearch loop for molecular geometry optimizers. The loop uses a single editable file, a fixed benchmark, a keep-or-discard decision, and two admission gates — a validity gate (rejects candidates with evaluation errors or mean energy recovery below the reference, r̄ < 1) and a generalization gate (rejects training-set improvements that fail on a validation split), with a significance threshold δ = 10⁻⁴.

2. AutoSella, a family of two evolved optimizers — AutoSella-F and AutoSella-A — produced by two independent autoresearch runs using Claude Fable 5.1 (Max) and GPT-6 Astra (High), respectively, each run for 66 active research hours starting from Sella 2.5.0. A third run with Grok-4.6 (xhigh) is reported in Appendix H.

3. Extensive evaluation of generalization along two axes: four held-out datasets (500 PubChem molecules selected to be dissimilar from training and validation, 673 Dipeptides, 475 amino acid–ligand pairs, and 100 Solvated PubChem systems) and two potentials never seen during evolution (the GFN-FF force field and r2SCAN-3c DFT).

4. Ablations of the evolved mechanisms, showing which groups of changes account for the largest validation force-call increases in each optimizer.

The code is publicly available at https://github.com/vdeshchenya/autosella.

Main Findings

• Sella was the strongest starting point. Among the 27 modern optimizer configurations the authors tested (from six packages, by one count in the text and "five packages" in the Figure 1 caption), Sella 2.5.0 used the fewest force calls, and its code became the seed for the search.

• AutoSella-F is the strongest result on the evolution potential (GFN2-xTB). It is cheapest in every held-out dataset, at 39.86–73.25% of the baseline's force calls per molecule (C_per-mol), with mean energy recovery r̄ > 1 on all four datasets. Pooled cost (C_pooled) ranges from 38.95% to 72.05%.

• AutoSella-A is weaker and regresses on the most complex systems. It uses 78.14–92.57% of baseline force calls per molecule on the Test, Dipeptides, and AA–ligand datasets on GFN2-xTB, but requires 118.77% on Solvated PubChem while recovering less energy (r̄ = 0.999125).

• The savings transfer to potentials never used in the search. On r2SCAN-3c DFT, AutoSella-F requires 77.20% (Test, N = 500), 73.14% (Dipeptides, N = 673), 68.61% (AA–ligand pairs, N = 475) and 40.24% (Solvated PubChem, N = 100) of Sella's force calls per molecule. The abstract summarizes this as 40.2–77.2% of Sella's force calls at the same energy reduction, achieved although the agent never used a DFT gradient.

• Largest DFT saving. Under r2SCAN-3c, the largest saving occurs on Solvated PubChem, where AutoSella-F requires only 40.24% of Sella's force calls while achieving 1.27% greater mean energy recovery.

• AutoSella-A under DFT. It reaches 109.06% of Sella's force calls per molecule on Solvated PubChem and recovers less energy (r̄ = 0.993028). AutoSella-F saves more force calls than AutoSella-A in every available comparison.

• Final evolved programs. AutoSella-F and AutoSella-A have mean per-molecule force-call ratios of 65.13% / 74.51% on training and 64.99% / 76.66% on validation, relative to original Sella.

• Different agent rhythms. Within 66 active research hours, Fable committed 108 evaluated candidates and accepted 37 improvements; Astra committed 672 and accepted 29. Fable had fewer evaluation errors (3, or 2.8%, versus 35, or 5.2%) and fewer candidates rejected for insufficient mean energy recovery (8, or 7.4%, versus 115, or 17.1%). Mean gaps between commits were 36.32 ± 32.08 minutes for Fable and 5.88 ± 3.66 minutes for Astra (107 and 671 intervals).

• Ablation results. Removing the model-Hessian-and-coordinates block and the fragment-handling block produced the two largest validation force-call increases: about 18.7 and 11.3 percentage points for AutoSella-F, and 8.8 and 5.7 points for AutoSella-A. The third-largest increase came from removing Hessian updates in AutoSella-F (10.2 points) and the Gaussian process in AutoSella-A (4.7 points).

• New optimizers reach lower energies on most molecules. The median final-energy difference between AutoSella-F and Sella ranges from −8 × 10⁻⁶ to −5 × 10⁻⁶ kcal mol⁻¹ on Test, Dipeptides, and AA–ligand pairs, and is −1.30 kcal mol⁻¹ on Solvated PubChem.

Methodology in Plain English

The authors treat optimizer design as a search problem. They pick Sella 2.5.0, condense its code into a single file (5975 source lines, derived from a package of 9136 lines whose minimization path touches 1418 lines and 131 functions across eight modules), and hand that file to an agent that may only edit it. Each proposed edit is committed and scored by a remote evaluator that runs full molecular relaxations on a training split of 469 structures (225 diverse PubChem molecules, 219 DES370K dimers, plus 25 registered drugs from a published benchmark) and a validation split of 465 structures (250 PubChem molecules and 215 DES370K dimers).

Because DFT is too expensive to evaluate hundreds of candidates, the search is run entirely on GFN2-xTB; a full candidate evaluation on the 469-structure training set takes roughly 60–90 seconds of wall time. Convergence follows the five-criterion Gaussian-style test used by geomeTRIC (thresholds of 10⁻⁶ E_h on energy change, 4.5 × 10⁻⁴ E_h a₀⁻¹ on maximum gradient, 3 × 10⁻⁴ E_h a₀⁻¹ on RMS gradient, 1.8 × 10⁻³ Å on maximum displacement and 1.2 × 10⁻³ Å on RMS displacement).

A candidate is only promoted if it passes the validity gate (no evaluation errors, mean energy recovery not below the reference, and a force-call cost below the champion's by more than δ = 10⁻⁴) and then the generalization gate (the same conditions on the validation split). The first gate prevents "cheating" by stopping early; the second prevents overfitting to the training molecules. The agent sees a per-molecule table of force calls, energies, energy differences and convergence status, which lets it form chemically targeted hypotheses. The authors did not let either gate inspect geometries, so they separately compared optimized conformations to Sella's using heavy-atom RMSD and vibrational analysis (Appendix E).

Evaluation uses four held-out sets: 500 out-of-distribution PubChem molecules, 673 dipeptides (out of 676 fixed identities), 475 amino acid–ligand pairs (25 systems for each of 19 amino acids), and 100 Solvated PubChem systems (one solute with 20 waters each). Relaxations on these sets were also tested with GFN-FF and with r2SCAN-3c DFT through ORCA (TightSCF, ENGRAD). Sella converges on all 1,648 structures in the PubChem test, Dipeptides and amino acid–ligand sets using 38 force calls on average and at most 200, and on 98 of 100 Solvated PubChem systems using 209 force calls on average and at most 366; of the remaining two, one reaches the 500-call cap and one fails during Sella's geometry update.

Why This Matters

Impact on research. The paper offers evidence that inexpensive surrogate potentials can drive automated algorithm development when direct search at the target level is too expensive, and that the resulting improvements can transfer to a much costlier level of theory. It also demonstrates a concrete anti-reward-hacking design: gates that block premature stopping and non-generalizing edits, rather than only optimizing a single scalar objective.

Real-world applications (as implications of the paper's drug-design focus):

• Faster DFT-level geometry relaxations for drug-like molecules, where each saved force call reduces wall time proportionally at the DFT level.
• Cheaper preparation of molecular conformations and non-covalent complexes (dimers, amino acid–ligand pairs) used in structure-based workflows.
• Lower cost for multi-fragment systems such as a solvated solute with surrounding water molecules, where the evolved optimizers show their largest savings.
• Reduced compute for large-scale relaxation benchmarks that curate DFT optimization trajectories of organic molecules.

Industry relevance. The authors frame the work around drug design and note that at the DFT level gradient calculations dominate wall time, so force-call reductions translate directly into lower cost per job. The evaluation used production-style tooling (xTB 6.7.1 with accuracy parameter 0.2, ORCA 6.1.1 through OPI, a Redis-backed pool of 192 single-thread workers across four remote evaluation hosts with Intel Xeon Gold 6348 CPUs), which is representative of the infrastructure such methods would need to run on.

Future Directions

• Extending the search budget and agent diversity. The paper reports a third run with Grok-4.6 (xhigh) in Appendix H; the relative value of different agents, longer budgets, or multi-agent variants (as in the related AI Scientist, Agent Laboratory, AlphaLab, AutoSOTA and AutoScientists lines of work) remains open.

• Closing the gap to DFT-guided search. The search never evaluated a DFT gradient. Whether incorporating limited DFT feedback improves the transfer further is not reported.

• Fixing the regression on complex multi-fragment systems. AutoSella-A consistently regresses on Solvated PubChem under both GFN2-xTB (118.77%) and r2SCAN-3c (109.06%), which the authors describe as limited generalization to complex multi-fragment systems.

• Understanding which mechanisms generalize. The ablations identify blocks whose removal hurts most (model Hessian and coordinates, fragment handling, Hessian updates, Gaussian process), but the paper does not establish a general theory of why those mechanisms transfer across potentials or how far the transfer extends beyond the four datasets and two unseen potentials tested here.

Target Audience

Researchers in computational and quantum chemistry who run geometry optimizations and care about force-call cost; developers of molecular simulation software such as ASE, geomeTRIC, pysisyphus and Sella; and machine-learning researchers working on LLM-driven program search, automated research loops, and reward-hacking-resistant evaluation design. Readers looking for introductory material on geometry optimization or DFT will find the paper advanced, since it assumes prior familiarity with those topics.

Authors’ abstract

Geometry optimization is a major cost in many quantum-chemical workflows: each optimization step requires one force evaluation, and at the density-functional level that evaluation dominates the wall time. Research in this area has produced a broad range of optimization methods, and we ask whether a language model can improve on the best of them through autoresearch. An agent rewrites the optimizer itself to minimize force-call counts, restrained by two admission gates that reject premature stopping and improvements that do not generalize to unseen molecules. Starting from Sella, the fastest open-source optimizer available, the search produces AutoSella, a family of two optimizers. Both of them deliver consistent force-call reductions relative to Sella across held-out molecular benchmarks and potentials not used during the search. Most notably, at the \texttt{r2SCAN-3c} DFT level, the best variant requires only $40.2$--$77.2\%$ of Sella's force calls while achieving the same energy reduction, even though agent used no DFT gradients.