S$^2$Drug: Bridging Protein Sequence and 3D Structure in Contrastive Representation Learning for Virtual Screening

AI summary

Overview

Research area: Computational drug discovery — specifically virtual screening (VS), protein representation learning, protein language models, and contrastive learning. The paper sits at the intersection of machine learning and structural biology.

Technical level: Intermediate. The framework is conceptually clear, but full engagement requires familiarity with contrastive objectives (InfoNCE), protein language model backbones (ESM2), 3D molecular encoders (Uni-Mol), and standard VS benchmarks.

Scope: The paper proposes S2Drug, a two-stage contrastive learning framework that pretrains on protein sequence–ligand data and then fuses sequence with 3D pocket structure to improve virtual screening, evaluated on two VS benchmarks and three binding-site prediction benchmarks.

What This Paper Is About

Virtual screening aims to find small molecules that bind a given protein pocket, but most deep learning methods today consume only 3D structural data — which is expensive to obtain (X-ray crystallography, Cryo-EM) and leaves out protein sequences, which are far more abundant and cheaper to produce. Simply adding sequence data is not trivial, however, because large protein–ligand interaction datasets contain redundancy and noise on both the protein and ligand sides. S2Drug addresses this by learning protein representations from sequence first, then fusing them with 3D structure in a fine-tuning stage while simultaneously predicting which residues form the binding site.

Key Contributions

1. A two-stage contrastive learning framework (S2Drug) that performs protein sequence–ligand contrastive pretraining followed by sequence–structure fusion fine-tuning, so that both modalities are used for virtual screening rather than structure alone.
2. A bilateral data sampling strategy for using large-scale protein–ligand data such as ChemBL. On the protein side it applies homology-aware downweighting (MMseqs2 clustering at a 40% identity threshold, sampling probability scaled by 1/|C|^α with α = 0.5) and functional deduplication via UniProt/Gene Ontology annotations. On the ligand side it applies affinity variability filtering (retain only pairs with σ < δ, δ = 1.0) and frequent hitter removal (discard ligands binding more than T = 20 proteins, unless most affinities indicate strong binding, plus PAINS filtering).
3. A residue-level gating fusion module plus an auxiliary binding site prediction task, added during fine-tuning on PDBBind. The gating mechanism adaptively weights sequence versus structure information per residue, and the auxiliary task (trained with summed binary cross-entropy) localizes binding residues along the sequence, helping the model capture how they assemble into a 3D pocket.
4. Empirical validation across benchmarks, showing consistent gains over docking-based and learning-based baselines on DUD-E and LIT-PCBA under a zero-shot setting, plus competitive binding-site prediction results on HOLO4K, COACH420, and ASD.

Main Findings

• DUD-E virtual screening: S2Drug reaches AUROC 92.46, BEDROC 79.25, EF 0.5% 58.37, EF 1% 43.06, EF 5% 18.82. The strongest retrieval-based baselines are DrugHash (AUROC 83.73, BEDROC 57.16) and DrugCLIP (AUROC 79.45, BEDROC 47.82). The paper reports the gap to DrugCLIP and DrugHash as 13.01 and 8.73 AUROC points respectively, with all S2Drug improvements marked significant at p < 0.01.

• LIT-PCBA virtual screening: S2Drug records the best BEDROC (8.69) and enrichment factors (EF 0.5% 11.44, EF 1% 7.38, EF 5% 2.97), all marked significant. Its AUROC of 58.23 is not marked significant and is below Banana (62.78) and Gnina (60.93), both of which appear without significance markers.

• Homology exclusion generalizability: Under identity cutoffs of 90%, 60%, 30%, and HMM-based exclusion between test targets and training data, S2Drug consistently outperforms DrugCLIP. Under the 90% and 60% cutoffs, S2Drug still beats DrugHash and DrugCLIP results obtained with no exclusion, suggesting reduced dependence on high train–test similarity.

• Ablation — fusion matters most: Removing the sequence–structure fusion module (SSF) drops DUD-E BEDROC from 79.25 to 69.85 and LIT-PCBA BEDROC from 8.69 to 6.88 — decreases of 9.40 and 1.81 points. Removing bilateral data sampling (DUD-E AUROC 88.73, LIT-PCBA AUROC 56.12) and removing the binding site prediction task (DUD-E AUROC 89.58, LIT-PCBA AUROC 56.47) each reduce performance versus the full model.

• Binding site prediction: S2Drug achieves the best results on COACH420 (F1 47.32, PR-AUC 42.62) and ASD (F1 48.79, PR-AUC 43.96), both marked significant, against baselines P2Rank, VN-EGNN, DiffDock, and VN-EGNNrank. On HOLO4K it ranks second with F1 51.66 and PR-AUC 46.97, behind DiffDock (F1 52.91, PR-AUC 47.52); the authors attribute this to DiffDock's diffusion-based generative handling of symmetric complexes common in HOLO4K.

• Hyperparameter robustness: Performance is relatively stable around the defaults λ = 0.50 (loss balancing coefficient) and τ = 0.1 (contrastive temperature). Values of λ above 1.0 cause optimization conflict and underfitting; extreme τ values cause over-sharpening or over-smoothing.

Methodology in Plain English

The approach has two sequential stages.

Stage 1 — sequence pretraining. Starting from ChemBL, described as 745K protein–ligand–affinity triplets, the authors clean and subsample the data from both sides. Proteins are clustered by sequence similarity so that members of large homologous families are sampled less often; functionally redundant proteins are collapsed to one representative each, prioritizing those with greater ligand diversity. Ligands are filtered for inconsistent affinity measurements across assays and for promiscuous "frequent hitters" and known problematic substructures. The remaining data is subsampled with rebalancing that downweights high-frequency ligands. A protein sequence encoder initialized from the 650M-parameter ESM2 model is then trained with a symmetric InfoNCE contrastive objective against ligand embeddings from the Uni-Mol molecular encoder, pulling binding pairs together and pushing non-binding pairs apart in a shared embedding space.

Stage 2 — sequence–structure fusion fine-tuning. On the smaller PDBBind dataset, which supplies high-resolution pocket structures, the model adds a Uni-Mol-initialized pocket structure encoder. For each pocket residue, atom-level structural embeddings are pooled into a residue-level vector, and a gating module computes a per-residue weight β to blend the sequence representation and the structural representation. Two Transformer layers then produce a fused pocket representation. In parallel, an auxiliary head samples K ligand probes and uses a shared attention projection over sequence residues only (to avoid information leakage) to predict, for every residue in the full sequence, the probability that it belongs to the binding site. The total loss combines the contrastive fusion loss and the weighted binding site prediction loss (L_total = L_fc + λ · L_bsp).

Training setup. Pretraining uses 10 epochs, learning rate 1e-3, and batch size 128; fine-tuning uses 50 epochs and batch size 48 with the learning rate unchanged. Experiments ran on a server with 1.6 TB RAM, 8 NVIDIA RTX A6000 GPUs, and dual Intel Xeon Silver 4309Y processors (16 cores, 32 threads @ 2.80 GHz). Each experiment was run 5 times and averaged, with paired t-tests and significance at p < 0.01.

Why This Matters

Impact on research. The work tests the principle that "sequence determines structure, and structure determines function" inside a virtual screening pipeline, and shows that sequence and structure are complementary rather than competing sources of signal. It also demonstrates that large, noisy, sequence-level interaction datasets like ChemBL — long available but underexploited for VS — can supervise learning if paired with careful de-redundancy and de-noising.

Real-world applications:

• Early-stage hit identification, where a compound library is ranked against a target before any expensive experimental screening.
• Screening against targets whose 3D structures are unavailable or low quality, since sequence data is much easier to obtain than crystal structures.
• Prioritizing candidate molecules under realistic hit rates, as simulated by the harder LIT-PCBA benchmark.
• Identifying binding-site residues directly from sequence, useful for annotating new or poorly characterized proteins and for guiding mutagenesis or pocket-engineering work.

Industry relevance. Pharmaceutical and biotech discovery pipelines are bottlenecked by the cost, time, and labor of empirical screening over very large chemical libraries, and by the availability of reliable protein structures. A method that extracts more value from cheap sequence data while still exploiting structural context when available addresses both bottlenecks. The reported robustness under remote-homology exclusion (HMM setting) is particularly relevant to industrial use, where novel targets rarely have close training-set relatives.

Future Directions

• Incorporating protein surface and solvent features, which the authors explicitly name as the direction for future work.
• Closing the gap on symmetric complexes, since DiffDock outperformed S2Drug on HOLO4K and the authors attribute this to diffusion-based sampling of multi-modal pose distributions and mitigation of geometric redundancy from symmetric units.
• Improving AUROC on realistic benchmarks, given that S2Drug's LIT-PCBA AUROC of 58.23 trailed Banana (62.78) and Gnina (60.93) even as it led on BEDROC and enrichment factors.
• Reducing dependence on structural supervision, since the fine-tuning stage still relies on PDBBind structures; the paper does not report results for a variant trained without any structural data at fine-tuning time.

Target Audience

Machine learning researchers working on multimodal representation learning and contrastive objectives; computational chemists and structural biologists interested in virtual screening and binding-site prediction; and drug-discovery practitioners evaluating whether sequence-based protein models can complement or partially replace structure-based pipelines. Readers with a background in protein language models or molecular graph/3D learning will get the most out of the methodological details.

Authors’ abstract

Virtual screening (VS) is an essential task in drug discovery, focusing on the identification of small-molecule ligands that bind to specific protein pockets. Existing deep learning methods, from early regression models to recent contrastive learning approaches, primarily rely on structural data while overlooking protein sequences, which are more accessible and can enhance generalizability. However, directly integrating protein sequences poses challenges due to the redundancy and noise in large-scale protein-ligand datasets. To address these limitations, we propose \textbf{S$^2$Drug}, a two-stage framework that explicitly incorporates protein \textbf{S}equence information and 3D \textbf{S}tructure context in protein-ligand contrastive representation learning. In the first stage, we perform protein sequence pretraining on ChemBL using an ESM2-based backbone, combined with a tailored data sampling strategy to reduce redundancy and noise on both protein and ligand sides. In the second stage, we fine-tune on PDBBind by fusing sequence and structure information through a residue-level gating module, while introducing an auxiliary binding site prediction task. This auxiliary task guides the model to accurately localize binding residues within the protein sequence and capture their 3D spatial arrangement, thereby refining protein-ligand matching. Across multiple benchmarks, S$^2$Drug consistently improves virtual screening performance and achieves strong results on binding site prediction, demonstrating the value of bridging sequence and structure in contrastive learning.