Protein as a Second Language for LLMs

AI summary

Overview

• Research area: Machine learning applied to computational biology, specifically protein function understanding using large language models (LLMs) and in-context learning, with no task-specific fine-tuning.
• Technical level: Intermediate. The core idea is conceptually simple and accessible, but the paper assumes familiarity with protein language models, retrieval-style in-context exemplar selection, and ROUGE-L evaluation, and it also reports GO-term graph pruning and sequence-clustering procedures that require some domain background.
• Scope (one sentence): The paper proposes and evaluates a framework that treats amino-acid sequences as a "second language" that generic LLMs can interpret through adaptively constructed bilingual (protein-sequence plus natural-language) in-context exemplars.

What This Paper Is About

Deciphering the function of an unseen protein sequence is a hard scientific problem, and most existing methods rely either on task-specific adapters or on large-scale supervised fine-tuning (and, for protein–language alignment models, on large paired datasets and re-fine-tuning whenever the output format changes). This paper asks whether a generic LLM can instead learn to "read" protein sequences the way a human acquires a second language: by seeing unfamiliar symbols repeatedly in context and inferring their meaning. The goal is a zero-shot, training-free route to protein understanding that generalizes across tasks and even surpasses domain-specialized protein models.

Key Contributions

1. The "Protein-as-Second-Language" conceptual framework, which recasts amino-acid sequences as a second symbolic language to be acquired via in-context learning, enabling protein understanding without parameter updates.
2. A protein–natural-language bilingual dataset of 79,926 protein–QA triples, built from 573,661 Swiss-Prot entries with gene ontology (GO) annotations and covering four task families: attribute-based QA, True or False QA, descriptive-text QA, and extended-information QA.
3. An adaptive context construction mechanism that assembles bilingual learning contexts per query using both amino-acid sequence homology (computed with MMseqs2) and similarity between descriptive texts or QA pairs and the query question.
4. Empirical demonstration that frozen, general-purpose LLMs and GPT-4o augmented with this method gain up to 17.2% in ROUGE-L (average +7%) and can surpass fine-tuned protein-specific LLMs, without any training.

Main Findings

• Consistent ROUGE-L gains across models and datasets: The method raises average ROUGE-L by 7% across diverse open-source models and GPT-4o, with a maximum gain of 17.2%, on ProtDescribe, Protein2Text-QA, and Mol-Instructions, with no parameter updates.

• Larger models benefit more: The paper reports that greater model capacity enhances the ability to leverage contextual cues, which the authors interpret as consistent with learning protein meaning through in-context analogy and reasoning.

• Frozen augmented LLMs beat fine-tuned protein LLMs: ProLLaMA-7B did not surpass frozen LLMs augmented with the method, which the authors attribute to limited training coverage and task-specific rigidity. Reported ProLLaMA-7B scores include 12.77 (ProtDescribe), 10.09 (Protein2Text-QA), and 16.85 (Mol-Instructions average); BioT5+ scored 9.97, 6.96, and 3.60 respectively.

• Selected headline numbers from Table 1 (ROUGE-L): GPT-4o moved from 18.29 to 35.53 on ProtDescribe (Δ gain +17.22) and from 20.84 to 26.86 on Protein2Text-QA (Δ gain +6.02). Qwen2.5-3B moved from 18.45 to 27.32 on ProtDescribe (+8.87) and from 23.21 to 28.66 on Protein2Text-QA (+5.45). Mistral-7B-Instruct-v0.3 moved from 15.02 to 29.39 (+14.37) and from 20.97 to 28.59 (+7.62). Qwen3-14B moved from 23.20 to 35.53 (+12.33) and from 21.02 to 25.93 (+4.91). kimi-k2 moved from 26.74 to 35.91 (+9.17) and from 17.33 to 21.04 (+3.71).

• Human evaluation agrees with the automatic metric: Five evaluators with biological-research experience rated answers on a 0–5 scale across six ordinal categories (Garbled, Inaccurate, Partially informative, Moderately accurate, Mostly accurate, Completely correct). Inter-rater consistency was reported as substantial (Krippendorff's α = 0.72%, as written in the paper). Models with context-driven exposure achieved higher or comparable ratings on most tasks, with the clearest improvements on Protein2Text-QA and several Mol-Instructions subtasks; pairwise win/lose analyses showed outputs with context exposure were preferred in the majority of comparisons, with win rates systematically exceeding loss rates. Each pairwise comparison used 8 randomly selected cases per subset, 48 cases total across six subsets.

• Optimal number of exemplars is task-dependent: Performance generally improves as the number of exemplars (k) increases, but only up to a task-dependent optimum. ProtDescribe peaks at k = 10–11, while Protein2Text-QA peaks earlier at k = 3–4. The experiments adopt k = 11 for ProtDescribe, k = 4 for Protein2Text-QA, and k = 4 for Mol-Instructions. The search space was k ∈ [1, 12]; the upper bound was set after a coarse scan up to k = 50 showed saturation around 2–12 exemplars.

• Both selection criteria matter (ablation, Table 2): Using both sequence homology and text/QA similarity (Dual) outperforms either alone. On average across three datasets, sequence-only selection reduces performance by 5.2% and text/QA-similarity-only selection reduces it by 2.8% relative to Dual, though all variants still outperform zero-shot models. For example, Qwen3-14B on ProtDescribe scored 35.53 (Dual), 22.90 (SeqOnly), and 30.52 (QAOnly); GPT-4o on Protein2Text-QA scored 26.86, 23.90, and 26.45 respectively.

• Dataset diversity and composition: After deduplication, the dataset covers 4,135 species across diverse evolutionary lineages, 63,749 families, and 1,115 superfamilies, with a long-tail distribution across sequence space and functional categories. Sample counts for the four protein-QA types range from 11,693 (attribute-based QA) to 32,444 (true/false QA). Amino-acid sequences constitute nearly 70% of corpus tokens.

• Dataset quality check: From the 79,926 automatically constructed examples, 500 pairs were randomly sampled and manually reviewed by domain experts on semantic fidelity, biological plausibility, and translation fluency, with a 95% pass rate.

• Qualitative case studies: Context-driven exposure produced concise, function-specific descriptions consistent with UniProt annotations; in two examples the model correctly identified "intrinsically disordered regions" and "[4Fe-4S] RNA methyltransferase activity," whereas zero-shot outputs remained generic.

• Exploratory real-world evaluation (Appendix): On biologically relevant queries involving uncharacterized Homo sapiens proteins, models guided by the framework produced plausible hypotheses aligned with biological knowledge. The paper does not report quantitative metrics for this exploratory setting.

Methodology in Plain English

The authors treat a protein sequence as a sentence in an unknown language, and they try to teach an LLM that language purely through examples placed in its context window, rather than by changing any model weights.

Step 1 — Build a bilingual textbook. They start with 573,661 Swiss-Prot entries carrying gene ontology annotations. To avoid massive redundancy, they prune the GO directed acyclic graph to keep representative functional categories, using two rules: a node is kept if it meets a depth-adjusted minimum support threshold, and a parent node is kept if its children's protein counts are too imbalanced (measured as the ratio of the largest to the smallest child protein count, compared against a depth-adjusted threshold). Proteins are grouped by the retained GO terms.

Step 2 — Remove redundancy twice. Within each group they cluster sequences with MMseqs2 at a 70% amino-acid sequence similarity threshold and pick one representative per cluster. Because sequence similarity below 70% does not imply functional divergence, they also deduplicate on annotation semantics: each protein gets a "Protein Functional Information Content" score equal to the sum of the information content of its GO terms plus all their ancestors, and proteins are sampled by unique rounded scores. A species quota strategy keeps the Eukaryota, Bacteria, Archaea, and Viruses proportions balanced.

Step 3 — Generate question–answer pairs. DeepSeek-R1 is prompted to produce four QA types: attribute-based (factual properties such as molecular function, cellular component, or family), knowledge-based (questions spanning expression, localization, mechanism, and interactions), descriptive text (longer coherent functional summaries), and True/False statements with brief explanations. The exact prompts are listed in the appendix.

Step 4 — Adaptively build context per query. At inference time, the system retrieves candidate examples for each incoming query using two complementary signals: sequence homology between candidate proteins and the query sequence (via MMseqs2), and similarity between the candidates' descriptive texts or QA pairs and the query question. The selected examples are structured into a coherent bilingual context and presented to the LLM alongside the query as in-context examples, so the model can reason by analogy and integrate evidence.

Step 5 — Evaluate. Evaluation uses ROUGE-L plus a human rating by five evaluators with at least two years of biology research experience, on ProtDescribe (553,052 protein–text pairs), Protein2Text-QA (209,847 open-ended QA pairs covering 5,574 unique proteins), and Mol-Instructions (2.04 M instruction instances, of which the protein-oriented section contributes 505 K). Models tested under identical prompting protocols include Qwen2.5-3B, Mistral-7B-Instruct-v0.3, Qwen3-14B, kimi-k2, and GPT-4o, with BioT5-plus-base and ProLLaMA as fine-tuned protein-oriented baselines.

Why This Matters

Impact on research. The paper argues that generic LLMs, when guided with protein-as-language cues, can outperform domain-specialized models. If that holds, protein understanding becomes a prompt-engineering and retrieval problem rather than a large-scale pre-training problem, removing the need for expensive paired-data curation, task-specific adapters, and repeated re-fine-tuning when the output format changes. The paper also frames its method as avoiding the catastrophic-forgetting risk associated with scaling gradient-based adaptation to larger LLMs.

Real-world applications (drawn from the tasks and settings the paper describes):

• Functional annotation of uncharacterized proteins: The exploratory evaluation on uncharacterized Homo sapiens proteins produced plausible hypotheses aligned with biological knowledge, which the authors suggest could offer interpretable preliminary insights into proteins lacking experimental annotation and potentially guide future biological investigations.
• Enzyme and catalytic activity prediction: Mol-Instructions includes a catalytic-activity subtask (reported as "Cat.") where the method improved scores for multiple models.
• Domain and motif recognition: Mol-Instructions includes a domain/motif recognition subtask (reported as "Dom."), relevant to identifying functional regions within sequences.
• Automated protein description generation: ProtDescribe-style captioning tasks produce natural-language summaries of a protein from its sequence, useful for databases and literature-facing resources.

Industry relevance. The paper's note that annotation generation includes "drug-target potential" as a knowledge dimension, together with the ethics statement's warning about "potential misuse for harmful purposes," points to relevance in biotech, pharmaceutical discovery, and protein-engineering workflows. Because no training is required and the approach works on frozen off-the-shelf LLMs and GPT-4o, deployment cost is limited to retrieval and prompting infrastructure rather than training compute. The ethics statement also restricts harmful or unsafe applications of the dataset.

Future Directions

• Finding the right amount of context automatically. The optimal number of exemplars varied sharply by task (10–11 for ProtDescribe versus 3–4 for Protein2Text-QA), so an open question is whether k can be predicted per query rather than fixed per task.
• Closing the gap between selection signals. Sequence-only and text/QA-similarity-only selection each underperformed the dual criterion (by 5.2% and 2.8% on average), suggesting there is headroom in how protein homology and textual relevance are combined or weighted.
• Quantifying real-world utility. The exploratory evaluation on uncharacterized Homo sapiens proteins was qualitative only; the paper does not report metrics or experimental validation for those hypotheses, leaving open how often such generated hypotheses hold up.
• Explaining why larger models benefit more. The paper observes that larger models gain more from context-driven exposure but does not isolate the mechanism; understanding this could inform model selection and whether capability gaps narrow with scale.

Target Audience

This paper is most useful to machine learning researchers working on in-context learning, retrieval-augmented generation, and cross-modal grounding; to computational biologists and bioinformaticians interested in training-free alternatives to protein language models and protein–text alignment models; and to practitioners in biotech or pharma evaluating whether general-purpose LLMs plus curated context can substitute for domain-specialized models. Readers need no deep protein-modeling background to follow the central idea, though the dataset construction sections assume some familiarity with GO terms, sequence clustering, and ROUGE-L.

Authors’ abstract

Deciphering the function of unseen protein sequences is a fundamental challenge with broad scientific impact, yet most existing methods depend on task-specific adapters or large-scale supervised fine-tuning. We introduce the "Protein-as-Second-Language" framework, which reformulates amino-acid sequences as sentences in a novel symbolic language that large language models can interpret through contextual exemplars. Our approach adaptively constructs sequence-question-answer triples that reveal functional cues in a zero-shot setting, without any further training. To support this process, we curate a bilingual corpus of 79,926 protein-QA instances spanning attribute prediction, descriptive understanding, and extended reasoning. Empirically, our method delivers consistent gains across diverse open-source LLMs and GPT-4, achieving up to 17.2% ROUGE-L improvement (average +7%) and even surpassing fine-tuned protein-specific language models. These results highlight that generic LLMs, when guided with protein-as-language cues, can outperform domain-specialized models, offering a scalable pathway for protein understanding in foundation models.