Neurosymbolic Retrievers for Retrieval-augmented Generation

AI summary

Overview

Research area: Retrieval-augmented generation (RAG), neurosymbolic AI, knowledge graphs, and interpretable retrieval for high-stakes clinical decision support.

Technical level: Intermediate. The paper combines dense retrieval, knowledge-graph traversal, and graph-based ranking losses, but each component is explained with worked clinical examples rather than assumed prior expertise.

Scope: The paper proposes and evaluates a family of neurosymbolic retrieval architectures — MAR, KG-Path RAG, and Proknow-RAG — that embed symbolic knowledge directly into the retrieval pipeline to make document selection interpretable and to enforce domain-specific procedural workflows.

What This Paper Is About

Standard RAG pipelines consist of a retriever, a re-ranker, and a generator, all of which are neural "closed boxes": developers cannot say why a particular document was retrieved, users cannot verify which passages shaped an answer, and specialists cannot confirm that a system followed an established workflow. This paper argues that the fix is not better post-hoc explanation but inherently interpretable architecture, and it introduces Neurosymbolic RAG, which couples symbolic structures (symbolic features, knowledge graphs, clinical questionnaires) with neural retrieval.

The motivating failure case is clinical: a system may retrieve the right guideline content yet present care steps out of order — for example recommending Cognitive Behavioural Therapy before risk assessment or diagnosis — producing an output that looks evidence-based but mis-implements the workflow. The paper asks whether retrievers can expose an interpretable basis for document selection (RQ1), whether knowledge graphs can simultaneously improve and explain retrieval (RQ2), and whether procedural knowledge can force retrieval to follow validated domain workflows (RQ3).

Key Contributions

1. Knowledge Modulation Aligned Retrieval (MAR): Introduces modulation networks that adjust query and document embeddings using interpretable symbolic features extracted from conversation turns, so that embedding dimensions correspond to controllable, explainable "semantic dials." Symbolic features accumulate across turns rather than being recomputed independently per utterance.

2. KG-Path RAG: Treats knowledge-graph traversal, query enrichment, and retrieval as a joint optimization problem rather than sequential components. A graph-based ranking loss (analogous to PageRank propagation) iteratively refines the retrieval distribution, and hyperparameters γ = 0.15 and β = 0.40 are used to produce the reported results.

3. Process Knowledge-infused RAG (Proknow-RAG): Uses domain-specific procedural instruments — the Patient Health Questionnaire (PHQ-9) for depression and the Generalized Anxiety Disorder scale (GAD-7) for anxiety — as an external scaffold to reorder retrieved evidence so that it follows the same stepwise assessment sequence clinicians use.

4. Empirical and expert evaluation: Benchmarks KG-Path RAG, Proknow-RAG, and a MentalLLAMA-33B baseline across eight mental health detection tasks on the IMHI dataset, and reports a human evaluation by three licensed clinical psychologists (agreement of 75 percent) on interpretability, clinical relevance, and computational efficiency.

Main Findings

• KG-Path RAG leads on symptom-interpretation tasks: It achieves the best reported accuracy in Depression Detection (84.7 percent), PTSD Detection (82.1 percent), and Stress Analysis (81.2 percent) on the IMHI benchmark, with the T5 model used in KG-Path RAG and Proknow-RAG having 3 billion parameters.

• Proknow-RAG leads on high-risk procedural tasks: It posts the highest score in Suicide Risk Detection (89.1 percent accuracy, F1 0.885, precision 0.897, recall 0.873) and is strong in Self-harm Risk Detection (83.2 percent accuracy, F1 0.819). The paper attributes this to the questionnaire-based process layer enforcing a validated clinical workflow.

• The MentalLLAMA-33B baseline is not uniformly beaten: MentalLLAMA-33B reports higher accuracy than both proposed systems on Anxiety Detection (79.8 percent vs 78.9 percent for KG-Path RAG and 75.6 percent for Proknow-RAG), Bipolar Detection (74.3 percent vs 73.4 percent and 70.1 percent), and Eating Disorders (68.7 percent vs 69.8 percent and 66.5 percent). The paper states that across the full benchmark the two neurosymbolic systems "consistently deliver stronger, more clinically aligned performance," with MentalLLAMA-33B credited for balanced results and strong explanation generation but lacking semantic scaffolding and procedural reasoning.

• Sample sizes vary substantially by task: The benchmark tasks range from 543 samples (Eating Disorders) to 2,341 samples (Stress Analysis), with Depression Detection at 2,150, Anxiety Detection at 1,890, PTSD Detection at 1,456, Suicide Risk at 987, Self-harm Risk at 834, and Bipolar Detection at 678.

• Expert evaluation favors Proknow-RAG: In the human study with three licensed clinical psychologists, Proknow-RAG scores highest in interpretability, clinical relevance, and computational efficiency, with KG-Path RAG performing similarly well and MentalLLAMA-33B scoring lower on interpretability and clinical alignment. The figure uses a 0 to 100 score axis; specific numerical scores per system are not reported in the text.

• Symbolic accumulation changes multi-turn retrieval: Worked examples show the accumulated feature set growing from {depressed mood} at turn 1, to {depressed mood, anhedonia} at turn 2, to {depressed mood, anhedonia, chronic insomnia, rumination, ACE disclosure, childhood abuse} at turn 3, so that the system recognizes one integrated clinical syndrome instead of three unrelated issues.

• Modulation strength scales with clinical complexity: The paper defines modulation strength as a sigmoid function of a complexity metric with three terms — symptom count, summed graph connection weights, and summed clinical severity scores — so reliance on structured knowledge increases as the presentation grows more interconnected.

Methodology in Plain English

The authors replace the single opaque similarity comparison of standard dense retrieval with three layered interventions.

First, MAR extracts explicit symbolic features from what a patient says — terms like "depressed mood," "anhedonia," or "ACE disclosure" — and accumulates them across conversation turns. These accumulated features are pushed into the embedding space through a learned projection matrix, shifting the query vector. Documents are shifted the same way using their own annotated features. Retrieval is still cosine similarity, but now the score reflects the symbolic features present, and a reader can inspect which features pushed which documents up. The paper also computes a complexity score from feature count, knowledge-graph connectivity, and clinical risk weighting, and uses a sigmoid to decide how strongly to lean on symbolic information.

Second, KG-Path RAG maps a user query onto nodes of a knowledge graph, walks one-hop or two-hop neighborhoods using breadth-first search to find conceptually related concepts, and has a language model generate an enriched query from the original question plus those graph-derived concepts. Rather than stopping there, the authors propose a joint objective: the retrieval score combines cosine similarity between query and document encodings with a PageRank-style importance term for the document in the graph, and training minimizes a combined retrieval loss plus a graph-ranking loss that separates relevant documents from graph-sampled negatives. This produces an explicit traversal path that can be shown as the justification for a retrieval decision.

Third, Proknow-RAG takes the documents retrieved by KG-Path RAG and reorders them using a standardized clinical questionnaire as a template. The knowledge graph identifies which instrument applies given the reported symptoms, and the instrument's item structure determines the sequence in which evidence is presented — mirroring how ReAct-style prompting and process reward models supervise the reasoning process rather than only the answer.

Evaluation combines automated metrics (accuracy, F1, precision, recall) on the IMHI dataset against a MentalLLAMA-33B baseline, plus a qualitative human study by three licensed clinical psychologists.

Why This Matters

Impact on research. The paper reframes RAG interpretability as an architectural property rather than a post-hoc add-on. It positions itself against ArgRAG and SymRAG, which it argues address post-hoc explainability or computational efficiency without transforming retrieval itself, and against RuleRAG (reported 89.2 percent improvement in Recall@10) and GraphRAG, which it says keep symbolic structures decoupled from neural retrieval. It also distinguishes itself from G-Retriever, which conditions retrieval on prize-collecting Steiner tree subgraphs but requires joint fine-tuning and offers limited explicit control over graph-level semantics.

Real-world applications:

• Clinical decision support for mental health, where a system must ask about depression before it asks about trauma history and must assess risk before recommending an intervention.
• Suicide and self-harm risk triage, the tasks where the procedural reordering layer produced its largest reported gains (89.1 percent and 83.2 percent accuracy respectively).
• Structured intake interviews and follow-up sessions, where the system retrieves a patient's history and generates targeted follow-up questions such as "You previously reported X. Are you still experiencing it?"
• Enterprise or legal knowledge management, where auditability of which source was used and why it was ranked where it was matters as much as answer quality.

Industry relevance. Systems that must justify their citations to regulators, clinicians, or auditors are a natural market for inherently interpretable retrieval. The paper's note that Proknow-RAG was also the most computationally efficient option in expert evaluation speaks directly to deployment cost, and the support from a UMBC Faculty Startup Award and a gift from NeuralNest LLC indicates commercial interest in the direction.

Future Directions

1. Systematic transparency evaluation: The authors call for rigorous assessment of transparency, evidentiary traceability, and alignment with clinical reasoning, specifically in comparison to emerging agentic and closed box retrieval methods that lack explicit auditability.

2. Multimodal extension: Extending these architectures to neuroimaging, physiological monitoring, and longitudinal patient records, and preserving interpretability as data complexity grows.

3. Clinician-centered evaluation frameworks: Retrieval outputs should be judged not just for correctness but for appropriateness within professional practice, consistency with diagnostic guidance, and safety in high-stakes decisions.

4. Closing the gap between symbolic and neural components: The paper itself identifies that KG traversal in its basic formulation is used only for query enrichment while the actual retrieval remains decoupled from the graph, which is what motivates the joint-optimization formulation. Whether that joint formulation fully resolves the issue, and whether MAR's preliminary results hold under full evaluation, remain open — the paper reports no numerical results for MAR.

Target Audience

This paper is most useful for researchers and practitioners working on retrieval-augmented generation who are dissatisfied with post-hoc explanation methods, and for applied machine learning engineers building clinical, legal, or enterprise systems where the provenance of a retrieved document must be defensible. It is also relevant to clinical informatics specialists interested in how validated instruments like PHQ-9 and GAD-7 can be encoded as retrieval constraints, and to knowledge-graph researchers looking for an application domain where graph structure carries genuine procedural meaning rather than only semantic association. Readers should be comfortable with embeddings, cosine similarity, and PageRank-style propagation, though the worked clinical dialogue examples make the core intuition accessible without that background.

Authors’ abstract

Retrieval Augmented Generation (RAG) has made significant strides in overcoming key limitations of large language models, such as hallucination, lack of contextual grounding, and issues with transparency. However, traditional RAG systems consist of three interconnected neural components - the retriever, re-ranker, and generator - whose internal reasoning processes remain opaque. This lack of transparency complicates interpretability, hinders debugging efforts, and erodes trust, especially in high-stakes domains where clear decision-making is essential. To address these challenges, we introduce the concept of Neurosymbolic RAG, which integrates symbolic reasoning using a knowledge graph with neural retrieval techniques. This new framework aims to answer two primary questions: (a) Can retrievers provide a clear and interpretable basis for document selection? (b) Can symbolic knowledge enhance the clarity of the retrieval process? We propose three methods to improve this integration. First is MAR (Knowledge Modulation Aligned Retrieval) that employs modulation networks to refine query embeddings using interpretable symbolic features, thereby making document matching more explicit. Second, KG-Path RAG enhances queries by traversing knowledge graphs to improve overall retrieval quality and interpretability. Lastly, Process Knowledge-infused RAG utilizes domain-specific tools to reorder retrieved content based on validated workflows. Preliminary results from mental health risk assessment tasks indicate that this neurosymbolic approach enhances both transparency and overall performance