Enhancing Multimodal Protein Function Prediction Through Dual-Branch Dynamic Selection with Reconstructive Pre-Training

AI summary

Overview

• Research area: Computational biology / bioinformatics, specifically multimodal deep learning for protein function prediction using Gene Ontology (GO) labels.
• Technical level: Advanced. The paper combines Mamba state-space models, Transformer attention, cross-attention, and a Mixture-of-Experts strategy, and assumes familiarity with these components.
• Scope: This paper proposes DSRPGO, a dual-branch neural architecture with reconstructive pre-training that fuses protein sequence, interaction network, subcellular location, and domain data to predict protein functions across BPO, MFO, and CCO GO categories.

What This Paper Is About

Predicting what a protein does is hard because the evidence comes from many different sources (amino acid sequences, protein-protein interaction networks, where the protein lives in the cell, which structural domains it contains), and each source has its own weaknesses: sequences can be dissimilar for proteins that do the same job, structures are expensive to obtain, and interaction networks are noisy. The authors build a model that learns from all these sources at once, first by training each source's encoder to reconstruct its own input (capturing fine-grained detail), then by letting the two branches exchange information and dynamically pick the most useful feature channels for each prediction.

Key Contributions

1. A multimodal protein function prediction framework (DSRPGO) that fuses sequence features with spatial-structure features (PPI networks, subcellular localization, protein domains), overcoming the limited information available from any single modality.
2. A reconstructive pre-training scheme built on a BiMamba-based encoder-decoder for spatial data and a Transformer-based encoder-decoder for sequences, designed to extract low-semantic, fine-grained features (local network topology, connection strength, amino acid frequency patterns) before the high-semantic prediction task.
3. The Bidirectional Interaction Module (BInM), which uses cross-attention so that sequence and spatial branches mutually inform each other, rather than being fused one-directionally.
4. The Dynamic Selection Module (DSM), an improved Mixture-of-Experts mechanism that scores expert confidence, keeps only experts above a threshold, and re-weights them, allowing the model to adaptively choose the feature channels most relevant to the specific protein and function label.

Main Findings

• Broad performance gains on human datasets: DSRPGO achieved the best F-max, m-AUPR, F1, and ACC across BPO, MFO, and CCO, improving F-max over the strongest multimodal baseline (CFAGO) by 1.9% in BPO, 1.8% in MFO, and 8.6% in CCO. Against unimodal methods the gains were larger: at least 6.4%, 7.7%, and 15.5% respectively.
• One metric exception: The model did not lead on M-AUPR (macro-averaged AUPR), which the authors attribute to that metric weighting rare GO classes equally and reflecting per-class difficulty rather than overall capability.
• Both branches are needed: Using only the shared-learning branch or only the interactive-learning branch underperformed the combined model across nearly every metric, confirming that shared and interactive representations are complementary.
• BInM and DSM both contribute: Removing BInM caused drops particularly in CCO F-max (0.452 to 0.333); removing DSM caused drops particularly in BPO F-max (0.458 to 0.397) and BPO m-AUPR.
• Pre-training is critical: Removing reconstructive pre-training produced the steepest declines, with BPO F-max falling from 0.458 to 0.297, MFO m-AUPR from 0.166 to 0.093, and CCO M-AUPR from 0.239 to 0.200.
• Modalities are complementary: Keeping only spatial features or only sequence features sharply degraded performance, confirming that the two families of data provide non-redundant information.
• Feature quality improves through the pipeline: Davies-Bouldin scores showed DSM_embedding produced the tightest, best-separated clusters, and t-SNE visualizations showed DSRPGO's learned features clustered far more cleanly than raw inputs or CFAGO's embeddings.

Methodology in Plain English

The approach unfolds in two training stages.

Stage one — reconstructive pre-training. Think of this as teaching the model to describe protein data to itself. Two encoders are built: one handles spatial information (the protein interaction network as an adjacency matrix, plus subcellular location and domain data), and one handles the amino acid sequence. Each encoder compresses its input into a small latent representation, and a matching decoder tries to rebuild the original input from that compression. Because the model is penalized when it cannot reconstruct fine details, it is forced to preserve low-level patterns like which neighbors a protein connects to and how strongly, or which fragments of the sequence are distinctive. The spatial encoder uses BiMamba blocks, which scan the data both forward and backward to capture context from either end; the sequence encoder uses Transformer self-attention blocks and reuses a frozen pre-trained protein language model (ProtT5) to parse sequences.

Stage two — the DSRPGO model. The pre-trained encoders now feed a two-branch prediction network. In the shared branch, features from all modalities are concatenated and analyzed in one joint space. In the interactive branch, the BInM module lets each branch's features act as queries against the other's keys and values via cross-attention, so sequence features can be refined by structural context and vice versa. Each branch produces a three-channel output (PPI, sequence, and location-plus-domain). All six channels feed the DSM, which works like a committee of experts: a small network assigns each expert a confidence score, only experts above a threshold are activated, and their contributions are renormalized and concatenated into the final feature. A fully connected predictor then outputs a score for each GO term. Because positive annotations are far rarer than negative ones, an asymmetric loss is used to prevent the model from simply predicting "no function" everywhere.

Evaluation. The model was trained on 19,385 proteins and fine-tuned on separate human BPO, MFO, and CCO splits, then compared against nine baselines ranging from BLAST and GeneMANIA to Graph2GO, DeepGraphGO, and CFAGO.

Why This Matters

• Impact on research: The paper shows that how you prepare features (reconstructive pre-training) and how you select them (dynamic expert selection at prediction time) can matter as much as which modalities you include. It also demonstrates that specialized state-space architectures like BiMamba are a viable alternative to graph neural networks for protein interaction data, avoiding the over-smoothing and noise amplification problems that GNN-based approaches face.
• Real-world applications:
  • Annotating uncharacterized proteins in newly sequenced genomes, where experimental validation is too slow and expensive.
  • Drug target discovery, where knowing a protein's molecular function and cellular role helps identify candidates for therapeutic intervention.
  • Enzyme engineering and synthetic biology, where designers need to predict whether a modified sequence will retain or acquire a desired function.
  • Metagenomic analysis of microbial communities, where most discovered proteins have no known function.
• Industry relevance: Pharmaceutical and biotechnology companies with large proprietary sequence and interaction datasets could use this architecture as a drop-in annotation engine; the open-source release of code and materials lowers the barrier to adoption. The general recipe — multimodal pre-training plus adaptive expert selection — also transfers to other hierarchical multi-label problems such as disease classification or gene regulatory role prediction.

Future Directions

• Better handling of rare classes: The model underperformed on M-AUPR, meaning it does not serve long-tail GO terms as well as it serves common ones. Addressing this directly, perhaps through class-balanced expert routing, would broaden its usefulness.
• Scaling and generalizing beyond human: All experiments used human protein data. Testing on other organisms, including model organisms and microbes, would reveal whether the architecture generalizes or has learned human-specific patterns.
• Incorporating additional modalities: Structures from AlphaFold, expression profiles, or text from the biological literature are natural additions that the framework could absorb, potentially replacing noisy PPI data with better-evidenced signals.
• Reducing reliance on pre-training cost: Pre-training took 5,000 epochs. Investigating more efficient pre-training objectives or transfer from existing protein foundation models could make the approach practical for groups without large compute budgets.

Target Audience

This paper is best suited for machine learning researchers working on multimodal representation learning and bioinformatics practitioners developing protein annotation pipelines. Readers should be comfortable with attention mechanisms, state-space models, and multi-label classification. Biologists seeking a practical tool will find the open-source implementation useful but may need to skim the architectural details; graduate students entering computational biology will find the two-stage pre-train-then-fuse design a useful template.

Authors’ abstract

Multimodal protein features play a crucial role in protein function prediction. However, these features encompass a wide range of information, ranging from structural data and sequence features to protein attributes and interaction networks, making it challenging to decipher their complex interconnections. In this work, we propose a multimodal protein function prediction method (DSRPGO) by utilizing dynamic selection and reconstructive pre-training mechanisms. To acquire complex protein information, we introduce reconstructive pre-training to mine more fine-grained information with low semantic levels. Moreover, we put forward the Bidirectional Interaction Module (BInM) to facilitate interactive learning among multimodal features. Additionally, to address the difficulty of hierarchical multi-label classification in this task, a Dynamic Selection Module (DSM) is designed to select the feature representation that is most conducive to current protein function prediction. Our proposed DSRPGO model improves significantly in BPO, MFO, and CCO on human datasets, thereby outperforming other benchmark models.