LLMs Depend on Specialist Tools for Out-of-Distribution Generalization in MS/MS Structural Elucidation

AI summary

Overview

Research area: Machine learning for chemistry, specifically agentic large language models applied to mass spectrometry and small-molecule structure identification.

Technical level: Intermediate. The paper is readable without deep ML background, but assumes familiarity with tandem mass spectrometry (MS/MS), SMILES strings, InChIKeys, and the distinction between library matching and de novo structure generation.

Scope: A controlled ablation study asking whether a frontier LLM can perform de novo MS/MS structural elucidation on its own, and whether giving it specialist tools (subformula annotation, forward spectral prediction, and de novo structure generators) closes the gap on compounds that are essentially undocumented in the literature.

What This Paper Is About

A recent result reported that a frontier LLM could solve 1D NMR structural elucidation with no tools, raising the question of whether that ability carries over to tandem mass spectrometry. MS/MS fragmentation is stochastic and non-local, so it cannot be inverted from substructure rules the way NMR chemical shifts roughly can. The authors therefore test whether an LLM's apparent MS/MS skill is real spectral reasoning or recall of well-documented compounds, and measure how much each individual tool contributes when added to a fixed agent harness.

Key Contributions

1. A 100-analyte benchmark split by "textual footprint." The authors draw spectra from the MassSpecGym v1.5 test fold (17,556 spectra, yielding 2,992 unique standardized analytes) and build two 50-analyte arms: an in-distribution (ID) arm with at least 100 linked PubMed records, a MeSH entry, and a common-name title, and an out-of-distribution (OOD) arm with zero linked PubMed records, no MeSH annotation, and a systematic (IUPAC-style) title. Pairs are matched 1:1 on precursor m/z, fragment-peak count, adduct (48 of 50 pairs) and instrument (41 of 50), and the 100 analytes were fixed before any experiment ran.

2. Evidence that unaided LLM accuracy tracks documentation rather than spectral content. With the model, harness, and output format held fixed, the LLM reaches 52% Top-1 in-distribution but only 2% out-of-distribution.

3. A per-tool ablation ladder under a fixed harness. Tools are added one at a time (subformula annotation, then forward prediction, then a de novo generator), so differences are attributable to the tools rather than to prompt or harness changes.

4. A demonstration that the agent is a strong re-ranker but a weak generator out-of-distribution. The out-of-distribution gain is bounded by the candidate coverage of the de novo generator the agent is given.

Main Findings

• Unaided LLMs behave like retrieval engines. At the L0 baseline, Claude Opus 5 achieves 52% Top-1 (26/50) on ID analytes and 2% (1/50) on OOD analytes, for 27% across all 100. Tanimoto@1 also separates the arms sharply (0.66 ID against 0.31 OOD), indicating the OOD predictions are structurally wrong, not merely mis-ranked. Because the arms do not differ on adduct (p=0.68), precursor m/z (p=0.98), or peak count (p=0.70), the gap cannot be attributed to spectral difficulty.

• The spectra alone do carry the information. Standalone de novo models reach 18% (MS-GPT) and 32% (LSM-Gen) Top-1 on the same OOD arm, so OOD spectra are not simply less informative than ID spectra.

• Hypothesis-testing tools help only in-distribution. Subformula annotation (L1) raises ID Top-1 from 52% to 62%, and forward prediction (L2) to 56%, while OOD accuracy stays at 2% in both cases. Forward prediction is mildly counterproductive where it acts, costing six points of ID Top-1 relative to L1.

• Only de novo generators move out-of-distribution accuracy. Adding FRIGID raises OOD Top-1 to 12%, MS-GPT to 26%, and the in-house LSM-Gen to 38%, lifting all-analyte Top-1 to 40%, 56%, and 59% respectively. The best agent configuration (L3 + LSM-Gen) reaches 59% all-analyte Top-1 against 46% for LSM-Gen standalone, 56% against 33% for MS-GPT, and 40% against 15% for FRIGID.

• The agent is a strong re-ranker. When the true structure appears anywhere in the generator's candidate pool, the agent ranks it first for 93–97% of ID analytes and 67–95% of textually OOD analytes. Since OOD analytes have no PubMed-linked records or MeSH entry, this is unlikely to be recall.

• Out-of-distribution gains are bounded by generator coverage. On the ID arm the agent exceeds the generator's candidate coverage (68% against 30% for FRIGID, 86% against 58% for MS-GPT, 80% against 64% for LSM-Gen, i.e. 20, 15, and 9 structures the pool never held; exact McNemar p<0.01). On the OOD arm it does not: 26% against 36% coverage for MS-GPT and 38% against 40% for LSM-Gen. FRIGID is the exception, gaining two out-of-pool structures while losing two in-pool ones (12% against 12% coverage).

• Candidate generation, not ranking, is the bottleneck. Across all experiments at most 14% of errors occur where the agent had the correct structure but failed to return it first; in nearly all the rest, the structure was never proposed by the agent or its tools.

• A second leakage channel exists in PubChem. 31 of the 100 analytes have MS/MS spectra displayed on their PubChem pages. The LLM identifies 61% of these (n=31) against 12% of the rest (n=69; odds ratio 12.1, p<0.001, Fisher exact test), while the de novo generators show no such difference (FRIGID 16% against 14%, LSM-Gen 45% against 46%, MS-GPT slightly reversed at 29% against 35%).

• The ID/OOD gap is not unique to the LLM, but is far larger for it. The LLM's Top-1 drops by a factor of 26 from ID to OOD; the three generator-only runs drop by roughly a factor of two to four (24% to 6% for FRIGID, 48% to 18% for MS-GPT, 60% to 32% for LSM-Gen). Nearest-neighbor similarity to the shared encoder training data also splits by arm (median ECFP4 Tanimoto 0.90 ID against 0.82 OOD).

• Cost: the highest-performing agents are the cheapest in the ladder, at 1.15–1.26 USD per spectrum against 1.27 at L0 and 1.58 at L1, because the model stops enumerating candidates from memory and answers in about three tool calls. Cost per correct identification falls from 4.70 USD at L0 to 2.05 USD at the cheapest L3 experiment. The agent still costs roughly twenty times more per correct identification than calling its generator directly (2.05 USD against 0.10 USD for LSM-Gen alone).

Methodology in Plain English

The authors first need a way to tell whether a model actually reasoned from a spectrum or just remembered a compound. Since frontier model training data is undisclosed, they use a proxy: how much has been written about a molecule. Compounds with many PubMed-linked records, MeSH entries, and common names are treated as likely memorized ("in-distribution"); compounds with no PubMed records, no MeSH entry, and only a systematic chemical title are treated as likely unseen ("out-of-distribution"). They build two sets of 50 spectra each, matched on mass, peak count, adduct, and instrument so that only documentation differs.

Then they hold everything else constant. The same model (Claude Opus 5, high reasoning effort, 64,000-token output allowance, a ceiling of twenty tool calls), the same prompt structure, the same output contract, and the same scorer are used in every run. The only variable is which tools the agent may call. They build a ladder: no tools (L0), plus subformula annotation (L1), plus forward spectral prediction (L2), plus a de novo generator (L3). At L3, generator candidates are added to the agent's candidate set rather than replacing it. Each generator is also run standalone as a non-agentic baseline. Tools are supplied through two interchangeable suites: a public suite (MIST-CF's subformula script, GLACIER for forward prediction, MS-GPT and FRIGID for generation) and an in-house suite substituting proprietary models while retaining GLACIER.

Every spectrum is queried with fresh context, tool outputs are screened for ground-truth leakage, and failed or truncated runs are re-run until all 100 analytes are answered. Accuracy is scored by whether the first 14-character InChIKey block (connectivity, ignoring stereochemistry) of any of the top five returned SMILES matches the truth, supplemented by Tanimoto@1 over 2048-bit ECFP4 fingerprints and MCES@1 graph edit distance. Statistical comparisons use McNemar tests for accuracy and Wilcoxon signed-rank tests for the similarity metrics, all against L0.

Why This Matters

Impact on research. The paper separates two things that are often conflated in chemistry LLM papers: a model's ability to retrieve a known answer and its ability to reason from experimental evidence. Its fixed-harness, one-tool-at-a-time design produces attribution that earlier MS/MS agent papers did not, since those compared a full toolbox against no tools at all. It also sets a methodological template, the textual-footprint split, for auditing memorization in domains where training data cannot be inspected.

Real-world applications:

• Untargeted metabolomics and "dark matter" annotation. Most spectra detected in a sample are never assigned to a structure; the results indicate which fraction of that gap a better generator, rather than a better LLM, would close.
• Novel drug metabolite and environmental transformation product identification. These compounds are frequently absent from spectral libraries, which is precisely the OOD regime studied here.
• Triage design for high-throughput pipelines. The cost analysis (2.05 USD per correct identification for the agent, 0.10 USD for the generator alone, roughly a twentyfold premium) supports a tiered workflow: library matching and in-silico retrieval first, generator alone for most of the remainder, and the full agentic workflow reserved for the few spectra that matter most.
• Benchmark auditing. The PubChem MS/MS finding (61% against 12% accuracy depending on whether fragmentation text appears on the compound page) shows that contamination auditing must cover spectral data, not just names and citations.

Industry relevance. The authors are all employees and shareholders of Matterworks, Inc., and one of the three generators (LSM-Gen) is their proprietary model; performance rankings and cost figures in the paper are therefore partly internal product comparisons. For vendors and pharma/biotech groups building annotation tools, the practical message is that investment in de novo generation quality pays off more than investment in the reasoning layer, and that agentic re-ranking is a genuine but bounded multiplier.

Future Directions

• Tighten encoder decontamination. The three generators share an encoder trained on synthetic spectra from 457,818 PubChem structures, filtered only by exact InChIKey match. The authors recommend removing close structural analogues, not just exact matches, from the encoder's training data, since nearest-neighbor similarity splits by benchmark arm (median 0.90 ID against 0.82 OOD) and may explain why the generators also perform worse out-of-distribution.
• Build better generators, not just better agents. Because the agent's OOD accuracy is capped by candidate coverage (26% against 36% for MS-GPT, 38% against 40% for LSM-Gen), progress on the dark matter of metabolomics depends primarily on de novo models that propose correct structures for undocumented compounds.
• Broaden the tool set and the evaluation. The authors call for agents that draw on a wider range of structural elucidation tools, and note their study covers a single model at a single date, without sweeping providers, temperatures, or token budgets, and that the model used is not uniformly the strongest LLM on scientific tasks.
• Validate the distribution proxy. Textual footprint cannot be verified without access to undisclosed training data, and the analytes do exist in PubChem, so a compound with no literature record may still have been seen; better proxies for training-set membership would sharpen future audits.

Target Audience

Machine learning researchers working on scientific agents and tool-use evaluation; mass spectrometry and metabolomics researchers interested in what LLM-based annotation can and cannot deliver; cheminformatics and analytical chemistry groups evaluating de novo structure generation; and benchmark designers who need practical methods for detecting memorization when training data is undisclosed. Readers without chemistry background will need to look up MS/MS fragmentation, InChIKey, and the library-matching versus de novo distinction, but the experimental logic is accessible.

Authors’ abstract

A frontier language model was recently reported to solve 1D NMR structural elucidation without tools. We ask whether this capability transfers to tandem mass spectrometry (MS/MS) in the de novo setting, where the model must propose a structure directly from the spectrum without a candidate list or reference library. To separate reasoning from recall, we build a 100-analyte benchmark from MassSpecGym, split by each analyte's textual footprint into in-distribution (well documented in literature) and out-of-distribution (essentially undocumented) arms. We hold the model, harness, and output format fixed and vary only the LLM's access to domain-specific tools. Without tools, a frontier LLM reaches 52% Top-1 accuracy in-distribution but only 2% out-of-distribution. Only access to de novo structure generators improves out-of-distribution accuracy, raising it to as much as 38% Top-1, and the resulting agent outperforms the standalone de novo models both in and out of distribution. These results suggest that LLMs are not effective at de novo MS/MS structural elucidation on their own, and that further gains depend on the specialist tools available to the agent rather than on the LLM itself.