Do Language Models Dream of Binding Molecules? Benchmarking LLMs under Spatial Constraints

AI summary

Overview

• Research area: Structure-based drug design (SBDD), 3D molecule generation, and evaluation of large language models on spatial reasoning tasks.
• Technical level: Advanced. The paper assumes familiarity with protein pockets, docking scores, pharmacophores, fragment-based design, and diffusion models, though its central question — can LLMs follow 3D geometric instructions? — is understandable without that background.
• Scope in one sentence: The paper introduces 3D-Fit, a token-efficient benchmark that tests whether current general-purpose LLMs can generate valid 3D drug-like molecules satisfying protein-pocket and other spatial constraints, and compares them against specialist diffusion models.

What This Paper Is About

Designing a molecule that fits a protein pocket is hard; designing one that simultaneously respects a fixed fragment, specific pharmacophore points, and required protein-ligand contacts is harder still. Specialized diffusion models lead at pocket-conditioned generation but are difficult to extend to many heterogeneous constraints at once, while LLMs handle multi-constraint instructions naturally but have never been systematically tested on 3D molecular geometry. The paper builds a benchmark to measure how well off-the-shelf LLMs understand, reason about, and generate molecules under these spatial constraints.

Key Contributions

1. The 3D-Fit benchmark: a framework for applying and evaluating multiple 3D conditions for molecular generation, built around four condition types — protein pocket, mandatory pocket-ligand interaction points, anchor fragments, and pharmacophore points — where any subset can be combined per test example.
2. Token-efficient textual representations of 3D conditions and a structured output format ("Simplified SDF"): conditions are described as compact text (residue names with heavy-atom coordinates, anchor atoms as spheres with a radius, pharmacophore points as typed positions), and outputs are written as a reduced MolBlock retaining atom index, symbol, coordinates, charge, and bond pairs only. An ablation found Simplified SDF consistently outperforms the SMILES+XYZ format used in prior work on nearly all metrics.
3. Two annotated test sets: 948 CrossDocked2020 v1.3 complexes and 505 PLINDER 2024-06/v2 complexes, each with sampled spatial conditions, filtered by valid loading (ProDy and RDKit), binding quality (RMSD < 0.5 Å and Vina score < -6 for CrossDocked2020), ligands with more than 25 heavy atoms, and pockets defined by residues with at least one atom within 10 Å of the reference ligand center.
4. A head-to-head evaluation of eleven LLM baselines against specialist diffusion models across four conditioning settings, including a four-condition setting (pocket + mandatory interaction + anchor + pharmacophore) that no diffusion model in the study natively supports.

Main Findings

• LLMs follow spatial constraints, but imperfectly. The models generally parsed the required output format successfully, and their success rates rose when conditions were expressed in a form that could be copied, mirrored, or locally reconstructed — anchor fragments and pharmacophore points. They struggled more with mandatory interaction points, which are more abstract.
• Raw LLM poses are poor binders. Before local UniDock optimization, every LLM's UniDock score was above the -6 kcal/mol threshold and often strongly positive, indicating severe steric clashes with the pocket or highly unfavorable placement.
• Optimization helps but does not close the gap. After local UniDock optimization, most LLM scores moved into roughly -6.0 to -7.0 kcal/mol, still behind the best pocket-specific diffusion models such as MolSnapper, PocketXMol, and PDMD.
• Diffusion models keep their binding mode; LLMs do not. Diffusion models produced poses with roughly the same binding mode before and after optimization, whereas LLM poses required substantial ligand repositioning to become reasonable.
• More conditions improved pocket-related metrics. Across experiments, adding constraints made LLM pocket metrics substantially better than pocket-only conditioning, suggesting seed-ligand information helps the models place molecules inside the pocket and encourages more accurate exploration of chemical space.
• Inter-molecular validity is not a proxy for binding quality. DiffSBDD, DiffPharma, SeFMol, IPDiff, and BindDM could pass many PoseBusters inter-molecular filters while still having weak optimized UniDock scores, whereas MolSnapper combined high inter-molecular validity with the best docking scores. The authors conclude UniDock scores remain the more direct ranking metric and PoseBusters filters are complementary checks.
• Internal structure quality varies. PoseBusters intra-molecular filters (bond lengths, bond angles, internal steric clashes, aromatic ring flatness, non-aromatic ring non-flatness, double bond flatness) revealed variability in LLM-generated conformations, and LLM molecules showed weaker intra-molecular validity than the strongest specialized models.
• Baseline reporting note. The text lists ten LLM baselines (GPT 5.5, GPT 5.4, Claude 4.7 Opus, Claude 4.6 Opus, Claude 4.6 Sonnet, Gemini 3.1 Pro, Grok 4.1 Fast Reasoning, Qwen-3.5 397b-a17b, DeepSeek v3.2, GLM-5), while the results tables report eleven model rows including Opus 4.8 and Grok 4.3 and do not include DeepSeek v3.2.

Methodology in Plain English

The researchers take known protein-ligand complexes and turn each one into a text prompt. The protein pocket is written out residue by residue, with each heavy atom's name and coordinates; the authors deliberately omit hydrogens and redundant PDB fields to save tokens. On top of that they add up to three extra conditions sampled from the reference ligand: one interaction that must be reproduced with a named residue (taken from ProLIF annotations, covering hydrophobic contacts, hydrogen bonds, π-π stacking, van der Waals contacts, ionic interactions, and cation-π/π-cation interactions), one small fragment of 3-8 heavy atoms extracted by BRICS fragmentation, and one pharmacophore point of a given type within 0.5 Å of a stated coordinate. To prevent coordinate memorization from test-set leakage, each complex is randomly shifted by up to 50 Å along each axis.

The LLM is asked to return a molecule using the Simplified SDF format inside <sdf> tags, with an explicit instruction to prioritize chemical validity if conditions cannot all be satisfied. Models were run without web search or external tools at temperature 1.0, top p 1.0, up to 8192 new tokens excluding reasoning tokens, with reasoning effort set to "high" where supported.

Every generated molecule is then scored on two pillars. Molecular validity covers whether the molecule parses and sanitizes, its heavy-atom count, Lipinski's rule of five, and PoseBusters intra-molecular checks. Condition success covers whether specified interactions, fragments, or pharmacophore points are actually placed correctly in 3D — evaluated only on 3D placement, not 2D connectivity — plus the median UniDock docking score and PoseBusters inter-molecular checks for the pocket. Success rates are computed over all outputs, and each condition metric is reported both before (Raw) and after (Optimized) local ligand pose optimization with UniDock.

Why This Matters

This work tests a question the field has largely assumed rather than measured: whether general-purpose LLMs can reason about 3D molecular space at all. The answer is nuanced — they show emerging ability to follow spatial instructions and handle many constraints at once, which diffusion models cannot easily do, but they do not yet produce physically plausible binding poses. That distinction matters because it tells researchers where LLMs are already useful (flexible multi-condition conditioning, instruction-following) and where they are not (pose quality, steric realism without post-hoc optimization).

Real-world applications:

• Lead optimization where medicinal chemists need to keep a fragment or interaction fixed while modifying the rest of the molecule.
• Multi-objective design campaigns that combine pocket fit with pharmacophore requirements and required contacts in a single prompt rather than retraining a specialized model.
• Fast hypothesis generation and triage, using LLM proposals as starting points for docking and optimization pipelines.
• Benchmark-driven model selection, giving teams a standard way to compare LLMs and diffusion models on the same spatial tasks.

Industry relevance: the work comes from Insilico Medicine, and it speaks directly to the practical question of whether off-the-shelf foundation models can be dropped into an existing SBDD stack. The authors also flag dual-use risk, noting that widely available models could be misused to generate harmful or toxic compounds, making misuse-aware filtering, access controls, and chemical-structure safeguards relevant for deployment.

Future Directions

• Closing the physical plausibility gap: figures such as the poor raw UniDock scores suggest future work on domain-specific training and improved 3D molecular representations rather than only better prompting.
• Scaling the condition set: the authors state their sampling strategy already supports more than one pharmacophore point and more than one mandatory interaction, so evaluating denser multi-condition settings is a natural next step.
• Runtime and reinforcement-learning sampling: the 3D-Fit framework allows additional or runtime sampling of protein-ligand complexes for applications such as reinforcement learning, provided test complexes are never used for training.
• Stronger anchors and significance testing: the authors identify random fragments as anchors (rather than chemically meaningful scaffolds) and the lack of statistical significance testing as limitations to address.

Target Audience

Researchers and practitioners in computational drug discovery and molecular machine learning, especially those evaluating or deploying LLMs for structure-based design. The paper also suits benchmark designers interested in how to encode 3D geometric information as text, and model developers who want a concrete target for improving LLM spatial reasoning. Readers without a structural biology background will need to look up docking scores, pharmacophores, and fragment-based design to follow the results tables.

Authors’ abstract

Structure-based drug design (SBDD) leverages the 3D structure of protein targets, often complemented by other spatial constraints, to generate candidate binding molecules. While diffusion models have dominated as a leading paradigm for high-quality 3D molecule generation, LLM-based methods are rapidly emerging in molecular design and have shown competitive performance in pocket-conditioned molecular generation. However, their ability to reason about physics and 3D spatial environments is largely underexplored. In this work, we systematically analyze whether current general-purpose LLMs are capable of navigating complex 3D constraints compared to established baselines such as specialized diffusion models. We consider 3D ligand generation conditioned on protein pockets together with ligand- and interaction-derived spatial constraints, including anchor fragments, pharmacophore points, and mandatory pocket-ligand interactions. To enable this evaluation, we introduce 3D-Fit - a token-efficient benchmarking strategy for assessing LLM performance on multi-conditioned spatial molecule generation. Our findings reveal a clear pattern in LLM spatial capabilities: while they still lag behind state-of-the-art approaches, they are promising and can handle multiple spatial constraints simultaneously, enabling scaling to heterogeneous setups.