From Knowledge to Treatment: Large Language Model Assisted Biomedical Concept Representation for Drug Repurposing

AI summary

Overview

Research area: Computational drug repurposing, biomedical knowledge graphs, and knowledge graph embedding (KGE) augmented with large language model (LLM) derived semantics.

Technical level: Advanced. The paper assumes familiarity with knowledge graph embeddings, link prediction objectives, contrastive/regularized training losses, and LLM prompting and embedding pipelines.

Scope: The paper proposes LLaDR, a framework that extracts text descriptions of biomedical entities from LLMs and uses them to fine-tune knowledge graph embeddings on the Drug Repurposing Knowledge Graph (DRKG) for drug-disease link prediction.

What This Paper Is About

Existing knowledge-graph-based drug repurposing methods mostly learn from graph structure alone, so they miss common-sense biomedical concept knowledge — for example, mechanistic priors indicating that certain drugs are fundamentally incompatible with specific treatments, such as acetylcholinesterase inhibitors being contraindicated in Parkinson's disease psychosis. The paper's goal is to inject LLM-derived, treatment-relevant textual representations of biomedical entities (compounds, diseases, genes, cell lines) into KGE training, so that embeddings capture both topology and semantic meaning, especially for under-studied or mechanistically complex indications.

Key Contributions

1. The authors propose what they describe as the first framework that integrates common-sense biomedical concept knowledge into KG-based drug repurposing.
2. They introduce a knowledge graph fine-tuning approach that uses LLM-derived concept knowledge to improve the semantic expressiveness of knowledge graph embeddings.
3. They report extensive experiments showing state-of-the-art performance on standard benchmarks, plus robustness under KG noise and semantic perturbations.
4. They provide code at https://github.com/xiaomingaaa/LLaDR.

Main Findings

• Best relative gains on DRKG: The paper reports "Best Imprv" values of 24.5% on Mean Rank, 6.7% on Hits@10, and 3.1% on AUC for the DistMult backbone; 16.5% MR, 3.6% H@10, 1.7% AUC for TransE; and 13.6% MR, 2.7% H@10, 1.4% AUC for RotatE. Because MR is lower-is-better, these are reductions in rank.
• TransE results (MR / H@10 / AUC): Base 6.28 / .840 / .835; LLaDR-gpt-3.5 5.39 / .871 / .850; LLaDR-gpt-4o-mini 5.46 / .864 / .849; LLaDR-LLama 5.40 / .867 / .850.
• DistMult results: Base 7.55 / .786 / .817; LLaDR-gpt-3.5 6.06 / .839 / .843; LLaDR-gpt-4o-mini 6.42 / .821 / .836; LLaDR-LLama 6.32 / .826 / .838.
• RotatE results: Base 6.06 / .850 / .837; LLaDR-gpt-3.5 5.33 / .873 / .849; LLaDR-gpt-4o-mini 5.52 / .862 / .844; LLaDR-LLama 5.49 / .862 / .844.
• Baseline comparisons (MR / H@10 / AUC): GraphSAGE 6.66 / .813 / .844; GAT 6.64 / .812 / .839; KGT5 10.48 / .650 / .806; CSProm-KG 6.64 / .812 / .839 in the main results table. The appendix table lists CSProm-KG (BERT) at MR 9.36, MRR .422, H@3 .491, H@10 .705, AUC .829, and KGT5 (T5) at MR 10.48, MRR .393, H@3 .443, H@10 .650, AUC .806.
• Additional metrics in the appendix: TransE base reaches MRR .476 and H@3 .568, versus .533 and .634 for the gpt-3.5 variant; DistMult base .423 and .497 versus .499 and .588 for gpt-3.5; RotatE base .493 and .588 versus .550 and .650 for gpt-3.5.
• Prompt design matters: With GPT-4o-mini, a "good prompt" outperformed an original prompt and a no-prompt setting. TransE: 5.64 / .856 / .840 (no prompt), 5.46 / .864 / .849 (original), 5.43 / .867 / .850 (good prompt). DistMult: 7.05 / .800 / .822, 6.42 / .821 / .836, 6.34 / .824 / .838. RotatE: 5.82 / .851 / .832, 5.52 / .862 / .844, 5.51 / .863 / .844.
• Low temperature is roughly comparable: Reducing generation temperature from 0.7 to 0.1 produced only small metric differences — TransE 5.75 / .856 / .845 versus 5.46 / .864 / .849; DistMult 6.44 / .821 / .836 versus 6.42 / .821 / .836; RotatE 5.71 / .857 / .840 versus 5.52 / .862 / .844.
• Semantic masking degrades performance: Masking 20%, 40%, and 60% of LLM-generated descriptions progressively worsened results. For TransE: original 5.46 / .864 / .849, mask 20% 5.96 / .839 / .843, mask 40% 6.04 / .838 / .838, mask 60% 6.12 / .835 / .836. For DistMult: 6.42 / .821 / .836 down to 6.74 / .809 / .828 at 60%. For RotatE: 5.52 / .862 / .844 down to 6.03 / .839 / .836 at 60%.
• Robustness to noisy KGs: Under injected corruption levels of 20%, 40%, and 60% and removal of corresponding proportions of triples, LLaDR maintained stable performance while baseline models without comparable mechanisms showed progressive accuracy decline.
• Alternative components tested: Replacing the text embedding deviation constraint with Kullback-Leibler divergence gave worse results (for example, DistMult 6.89 / .434 / .512 / .804 / .827 versus 6.42 / .482 / .568 / .821 / .836 for the original). Using bge-v1.5-small instead of Text-embedding-3-small also gave lower results on all three backbones.
• Case study: For Alzheimer's disease, LLaDR's top-ranked compounds included Dasatinib and Quercetin, which the authors connect to supporting literature (Krzystyniak et al., 2022). The top 10 predictions are listed in Appendix A.7.
• Compute details: Experiments used an RTX 3080, 100,000 training steps, with one DRKG training session taking 2 hours.

Methodology in Plain English

LLaDR runs in two phases.

First, the authors take every entity in the biomedical knowledge graph and query an LLM with a prompt to produce a short, structured text description of that entity. Each description is converted into a vector with a sentence embedding model, and this text vector is concatenated with the entity's own embedding to form an enriched representation.

Second, these enriched representations seed the fine-tuning of a knowledge graph embedding model. Two objectives are combined. A semantic anchoring constraint pulls the fine-tuned embedding back toward its sliced text embedding, so the model does not drift away from the entity's original meaning — this acts as a regularizer, which the paper says is especially important for large clusters of diverse entities and for sparse graphs. A link prediction objective then trains the model to score true triples higher than corrupted ones, using a margin-based positive term and averaged negative-sample penalties. The overall training loss is a weighted sum of the anchoring loss and the link loss, with weights that control the trade-off between semantic fidelity and structural consistency.

The method is designed to be a general wrapper: it is applied on top of TransE, DistMult, and RotatE backbones, each tested with descriptions from GPT-3.5-turbo, GPT-4o-mini, and a LLaMA variant. Evaluation uses tail entity replacement, sampling 50 candidate disease tail entities per test triple across three therapeutically relevant relation types, repeated five times with averaged results. Metrics are Mean Rank, Mean Reciprocal Rank, Hits@N, and AUC.

Why This Matters

The paper's central claim is that structural knowledge graph embeddings are not enough for safe and effective treatment decisions because they cannot represent concept-level constraints such as contraindications and mechanistic incompatibilities. Bridging LLM semantic knowledge and graph structure is intended to make repurposing predictions more reliable for rare or mechanistically complex diseases, and the masking experiments support the idea that the textual semantics carry real signal rather than noise.

Real-world applications:

• Screening existing drugs for new therapeutic uses in diseases where trial data and approved options are scarce, such as Alzheimer's disease in the paper's case study.
• Flagging likely drug-disease relationships while awareness of concept-level contraindications matters, which is where the paper positions its advantage over structure-only models.
• Extending the same LLM-plus-KGE approach to related biomedical graphs such as disease-gene networks, as the conclusion suggests.
• Supporting clinical decision support systems, which the limitations section names as a potential use if interpretability concerns are addressed.

Industry relevance: Pharmaceutical and biotech research teams looking to prioritize repurposing candidates before expensive wet-lab validation, and computational drug discovery groups that already maintain biomedical knowledge graphs and want a lightweight way to add language-model semantics without full prompt-based retraining. The reported 2-hour DRKG training run on a single RTX 3080 makes the approach comparatively cheap to iterate on.

Future Directions

• Addressing the dependence on LLMs, since errors or biases in generated descriptions from GPT-3.5-turbo or GPT-4o-mini can directly affect performance and reliability.
• Solving scalability, because computational cost may rise substantially for graphs with millions of entities.
• Improving interpretability, since the "black-box" nature of LLMs complicates clinical adoption and stakeholder trust.
• The conclusion also points to extending the method to larger graphs and to multimodal data. Related open questions include domain generalization beyond DRKG and how to identify and exclude factually incorrect generated descriptions before they enter training.

Target Audience

Researchers and practitioners working on computational drug repurposing, biomedical knowledge graphs, and knowledge graph representation learning; NLP researchers interested in how LLM-generated text can be fused with structured embeddings; and pharmaceutical or clinical informatics teams evaluating whether LLM semantics can make graph-based repurposing predictions more trustworthy. Readers need a working understanding of KGE scoring functions and link prediction metrics to get the most from the results tables.

Authors’ abstract

Drug repurposing plays a critical role in accelerating treatment discovery, especially for complex and rare diseases. Biomedical knowledge graphs (KGs), which encode rich clinical associations, have been widely adopted to support this task. However, existing methods largely overlook common-sense biomedical concept knowledge in real-world labs, such as mechanistic priors indicating that certain drugs are fundamentally incompatible with specific treatments. To address this gap, we propose LLaDR, a Large Language Model-assisted framework for Drug Repurposing, which improves the representation of biomedical concepts within KGs. Specifically, we extract semantically enriched treatment-related textual representations of biomedical entities from large language models (LLMs) and use them to fine-tune knowledge graph embedding (KGE) models. By injecting treatment-relevant knowledge into KGE, LLaDR largely improves the representation of biomedical concepts, enhancing semantic understanding of under-studied or complex indications. Experiments based on benchmarks demonstrate that LLaDR achieves state-of-the-art performance across different scenarios, with case studies on Alzheimer's disease further confirming its robustness and effectiveness. Code is available at https://github.com/xiaomingaaa/LLaDR.