MS-BART: Unified Modeling of Mass Spectra and Molecules for Structure Elucidation

AI summary

Overview

Research area: Machine learning for analytical chemistry — specifically deep learning for molecular structure elucidation from mass spectrometry (MS) data, positioned at the intersection of language modeling, cheminformatics, and metabolomics.

Technical level: Advanced. The paper assumes familiarity with Transformer encoder-decoder architectures, masked-token pretraining, cross-entropy and contrastive rank losses, molecular fingerprints, SELFIES/SMILES strings, and spectral similarity metrics (Tanimoto, MCES).

Scope: The paper introduces MS-Bart, a BART-Base language model that represents both mass spectra (as fingerprints) and molecules (as SELFIES) in one shared token vocabulary, trained under a pretraining–finetuning–alignment pipeline, and evaluates it on the NPLIB1 and MassSpecGym benchmarks.

What This Paper Is About

Determining a molecule's chemical structure from its mass spectrum is hard because annotated experimental spectra are scarce, and raw spectra for the same molecule change substantially with collision energy, adduct type, and instrument settings. MS-Bart sidesteps raw spectral variability by using molecular fingerprints — binary vectors encoding chemical substructures — as a stable intermediate representation of spectra, then models fingerprints and molecular SELFIES strings together as a single token sequence. The goal is to generate the correct molecular structure from a spectrum more accurately and much faster than existing de novo methods.

Key Contributions

1. First language-model formulation of MS structure elucidation. The authors state they are the first to leverage a language model for this task by introducing a unified token vocabulary covering both fingerprint tokens and SELFIES tokens, combined with multi-task pretraining on a large corpus of fingerprint–molecule pairs.

2. A four-task multi-task pretraining scheme. Pretraining combines SELFIES denoising, fingerprint-to-molecule translation, and hybrid denoising in two input orderings, all optimized with cross-entropy loss, on 4 million fingerprint–molecule pairs.

3. Finetuning on experimental spectra plus chemical-feedback alignment. The model is finetuned using MIST-predicted fingerprints to absorb dataset-specific noise and bias, then aligned with a contrastive rank loss based on Tanimoto similarity to reduce molecular hallucination (chemically valid but structurally wrong outputs).

4. State-of-the-art results on two public benchmarks. MS-Bart achieves SOTA on 5/12 key metrics on MassSpecGym and NPLIB1 and is reported as faster by one order of magnitude than competing diffusion-based methods. Data and code are released at https://github.com/OpenDFM/MS-BART.

Main Findings

• NPLIB1 similarity metrics: MS-Bart leads all baselines on every similarity metric, reporting Top-1 MCES 9.66 and Top-1 Tanimoto 0.44, versus the second-best (DiffMS) at MCES 11.95 and Tanimoto 0.35 — described as a 19.16% (MCES) and 25.71% (Tanimoto) improvement over the second-best method in the Top-1 setting.
• NPLIB1 accuracy is not the top result: MS-Bart reaches Top-1 accuracy 7.45% and Top-10 accuracy 10.99% on NPLIB1, below DiffMS at 8.34% and 15.44%. The authors attribute this to their stricter pretraining-data filtering (removing molecules with MCES distance less than two from the test fold), whereas DiffMS only removes NPLIB1 and MassSpecGym test/validation molecules.
• MassSpecGym results: MS-Bart reports Top-1 accuracy 1.07%, MCES 16.47, Tanimoto 0.23, and Top-10 accuracy 1.11%, MCES 15.12, Tanimoto 0.28. DiffMS reports higher accuracy (2.30% Top-1, 4.25% Top-10) and Tanimoto (0.28, 0.39), while MS-Bart has better MCES (16.47 vs 18.45; 15.12 vs 14.73 — the Top-10 MCES comparison favors DiffMS). Excluding DiffMS, the paper states MS-Bart shows SOTA performance across all similarity metrics.
• Gold-fingerprint ceiling: When given the fingerprint computed from the true structure instead of MIST's prediction, MS-Bart reaches 73.50% Top-1 accuracy, MCES 2.14, Tanimoto 0.90 on NPLIB1 and 47.56% Top-1 accuracy, MCES 3.26, Tanimoto 0.85 on MassSpecGym. The authors present this as evidence that improving MIST is a promising direction.
• Pretraining ablations matter: On NPLIB1, no pretraining gives Top-1 accuracy 1.71% and Tanimoto 0.27; SELFIES denoising alone is worse (0.37%, 0.24); fingerprint-to-molecule translation alone is much better (6.23%, 0.42); hybrid denoising gives 5.13% and 0.41; the full MS-Bart gives 7.45% and 0.44. The denoising task alone degrades performance because it is not well aligned with structure elucidation.
• Progressive hallucination reduction on MassSpecGym: The pretrained-only model scores 0.00% Top-1 and 0.01% Top-10 accuracy; after finetuning, 1.07% and 1.12%; the alignment stage further improves Tanimoto similarity and MCES.
• Hyperparameter insensitivity: Across fingerprint thresholds ε from 0.10 to 0.20, Top-1 Tanimoto similarity on MassSpecGym ranges narrowly from 0.1636 to 0.1678. Rank loss weight α of 1, 3, and 5 on NPLIB1 gives Top-1 Tanimoto 0.44, 0.44, and 0.44 respectively, with Top-1 accuracy of 7.08%, 7.20%, and 7.45%.
• Decoding speed: With beam widths from 10 to 100, Top-1 and Top-10 accuracy improve consistently while latency scales linearly. On an RTX 4090 at beam width 100, average inference is about 3 seconds per spectrum, stated as 53x faster than DiffMS's approximately 160 seconds.
• Dataset difficulty context: MassSpecGym contains 231k high-quality mass spectra over 29k unique molecular structures; its [M+Na]+ training samples constitute only 15.52% of the data, and [M+Na]+ fragmentation differs substantially from [M+H]+. NPLIB1 lacks a scaffold-based split, and its test set contains molecules with Tanimoto similarity greater than 0.85 to training molecules.

Methodology in Plain English

The approach follows the pretraining–finetuning–alignment recipe familiar from NLP.

Representation. Instead of feeding raw peak lists, the model represents a spectrum as a 4096-bit circular Morgan fingerprint. Each active bit becomes a token like <fp0123>. Molecules are written as canonical SELFIES strings drawn from a vocabulary of 185 tokens, which guarantees that any generated string corresponds to a chemically valid molecule.

Pretraining. The authors computed fingerprints from 4 million unlabeled molecules (sourced from MassSpecGym) using RDKit, producing fingerprint–molecule pairs without needing any experimental spectra. Four self-supervised tasks are trained jointly: mask 30% of SELFIES tokens and recover them; generate SELFIES conditioned on fingerprint tokens; and two "hybrid" variants that concatenate fingerprint tokens and masked SELFIES in either order, separated by a special <fps_sep> token, and predict the full SELFIES sequence. All are optimized with cross-entropy loss.

Finetuning. For real experimental spectra, a pretrained model called MIST predicts a fingerprint probability vector conditioned on the chemical formula. Probabilities above a threshold ε become binary bits (ε = 0.2 for NPLIB1, ε = 0.11 for MassSpecGym). The model is then finetuned with cross-entropy on these predicted fingerprints, which lets it learn MIST's systematic biases.

Alignment with chemical feedback. Because the model can still hallucinate — producing valid molecules that differ from the truth — the authors sample n candidate molecules per input and rank them by Tanimoto similarity to the ground truth. A contrastive rank loss pushes the model's own log-probabilities into the same order, with a margin γ_ij = (j − i) × γ scaled by rank difference. The total loss is L = L_ce + α L_rank. During alignment the encoder is frozen and only the decoder is updated.

Implementation. The backbone is BART-Base initialized from scratch. Pretraining uses a maximum sequence length of 512; finetuning and alignment use fixed input and output lengths of 256. Pretraining ran on four NVIDIA A800-SXM4-80GB GPUs in bfloat16 with per-device batch size 96 and two gradient accumulation steps (effective batch 768) for three epochs, a cosine schedule with 10,000 warm-up steps, peak learning rate 6e-4 decaying to 1e-5, taking roughly 34 hours. Finetuning used one A800-SXM4-80GB, learning rate 5e-5, batch size 128, and early stopping on Top-1 Tanimoto similarity (checked every 400 steps on MassSpecGym, every 200 on NPLIB1). To prevent leakage, the pretraining set excluded molecules with MCES distance below two from any MassSpecGym test molecule, and separately removed molecules with maximum Tanimoto similarity above 0.5 to any NPLIB1 test molecule — roughly 3% of the pretraining molecules were similar to NPLIB1 test molecules before filtering.

Evaluation. Top-k accuracy uses full InChIKey exact match (2D InChIKey results appear in Appendix C), Top-k maximum Tanimoto similarity uses Morgan fingerprints with radius 2 and 2048 bits, and Top-k minimum MCES measures graph edit distance. Results are reported for k = 1 and k = 10. To match DiffMS's protocol, 100 molecules are sampled per spectrum and reranked by atomic-count distance to the given formula (Eq. 6), with ties broken by estimated log-probability.

Why This Matters

The work shows that the pretrain–finetune–align paradigm from NLP can be transferred to mass spectrometry if the raw, highly variable signal is first replaced by a stable, chemically meaningful intermediate representation. It also quantifies a ceiling: the large gap between MS-Bart's results with predicted fingerprints versus gold fingerprints (for example, 7.45% versus 73.50% Top-1 accuracy on NPLIB1) signals that the spectral inference front-end, not the generative model, is the current bottleneck.

Real-world applications:

• Drug discovery and metabolite identification: rapid candidate structure proposals for small molecules from MS data without reference-library matches.
• Environmental biochemistry: identifying unknown or novel compounds in environmental samples where no reference spectrum exists.
• Materials science and chemical analysis: computer-aided structure assignment alongside existing analytical workflows.
• High-throughput screening: the reported speed (about 3 seconds per spectrum on a consumer-grade RTX 4090 at beam width 100) makes batch processing of large spectral collections practical.

Industry relevance: the method's one-order-of-magnitude speed advantage over diffusion-based generation, its open-source code release, and its ability to propose structurally similar candidates (Top-1 Tanimoto values close to or exceeding 0.5) are directly relevant to labs that need ranked candidate lists rather than a single guaranteed answer.

Future Directions

• Improving the spectral front-end. The authors explicitly point to the gold-fingerprint results as evidence that further work should target the MIST model, since fingerprint prediction quality dominates end accuracy.
• Handling adduct diversity. MS-Bart filters out [M+Na]+ adducts before finetuning and alignment and retains only [M+H]+, while still evaluating on the mixed test set — extending the method to diverse adduct types without fragmentation-pattern conflicts is an open problem.
• Closing the accuracy gap with diffusion models. MS-Bart trails DiffMS on Top-1 and Top-10 accuracy on both benchmarks; the paper attributes this partly to stricter pretraining-data filtering, but does not demonstrate that relaxing the filter would recover the gap.
• Resolving the validation/test mismatch. The paper reports that the hyperparameters chosen on validation (ε = 0.11, α = 5) are not the best overall under ablation, which the authors attribute to MassSpecGym's difficulty and distribution mismatch between its validation and test sets — a data-split issue that remains unresolved.

Target Audience

Machine learning researchers working on scientific applications of language models and cross-modal representation learning; computational chemists and metabolomics researchers interested in de novo structure elucidation; and practitioners in analytical chemistry, pharma, and environmental science who need automated or semi-automated candidate generation from mass spectra. Readers without a background in cheminformatics (fingerprints, SELFIES, MCES, Tanimoto similarity) or in Transformer pretraining objectives will need supplementary reading.

Authors’ abstract

Mass spectrometry (MS) plays a critical role in molecular identification, significantly advancing scientific discovery. However, structure elucidation from MS data remains challenging due to the scarcity of annotated spectra. While large-scale pretraining has proven effective in addressing data scarcity in other domains, applying this paradigm to mass spectrometry is hindered by the complexity and heterogeneity of raw spectral signals. To address this, we propose MS-BART, a unified modeling framework that maps mass spectra and molecular structures into a shared token vocabulary, enabling cross-modal learning through large-scale pretraining on reliably computed fingerprint-molecule datasets. Multi-task pretraining objectives further enhance MS-BART's generalization by jointly optimizing denoising and translation task. The pretrained model is subsequently transferred to experimental spectra through finetuning on fingerprint predictions generated with MIST, a pre-trained spectral inference model, thereby enhancing robustness to real-world spectral variability. While finetuning alleviates the distributional difference, MS-BART still suffers molecular hallucination and requires further alignment. We therefore introduce a chemical feedback mechanism that guides the model toward generating molecules closer to the reference structure. Extensive evaluations demonstrate that MS-BART achieves SOTA performance across 5/12 key metrics on MassSpecGym and NPLIB1 and is faster by one order of magnitude than competing diffusion-based methods, while comprehensive ablation studies systematically validate the model's effectiveness and robustness.