Disco-RAG: Discourse-Aware Retrieval-Augmented Generation

AI summary

Overview

Research area: Natural Language Processing, specifically Retrieval-Augmented Generation (RAG) for large language models, combined with discourse theory (Rhetorical Structure Theory).

Technical level: Advanced. The paper assumes familiarity with RAG pipelines, retrieval metrics, discourse parsing, and benchmark evaluation protocols.

Scope: The paper proposes Disco-RAG, an inference-time framework that injects intra-chunk discourse trees, inter-chunk rhetorical graphs, and discourse-driven plans into RAG to improve generation, evaluated on Loong, ASQA, and SciNews.

What This Paper Is About

Standard RAG systems retrieve text chunks and concatenate them in a flat, unstructured way, so the generator sees what the authors call a "bag of facts" rather than a coherent line of reasoning. This creates two problems: the model misses the internal discourse structure within each chunk, and it misses the rhetorical connections between chunks, which can lead to overgeneralized or contradictory conclusions. The paper's goal is to explicitly model and supply discourse structure to the LLM so that generation becomes more correct, coherent, and faithful to the retrieved evidence.

Key Contributions

1. Disco-RAG, an inference-time strategy that explicitly injects discourse knowledge into the RAG pipeline to reduce the discrepancy between chunk-level evidence and discourse-level reasoning.

2. A three-part discourse modeling method combining intra-chunk RST trees (local hierarchies), inter-chunk rhetorical graphs (cross-passage coherence), and discourse-driven plans (argumentative flow).

3. Empirical validation on knowledge-intensive QA and summarization, with consistent gains over strong RAG baselines plus analysis studies on correctness, coherence, and factuality.

4. Extensive analysis experiments, including ablations, retrieval granularity and noise robustness tests, structural perturbation studies, mixed-model deployment, supervised fine-tuning interaction, and human evaluation.

Main Findings

• Loong benchmark gains: With Llama-3.3-70B as backbone, Disco-RAG achieves an LLM Score of 71.00 in Set 1 (10K–50K tokens), outperforming standard RAG by 8.22 points. In Set 4 (200K–250K tokens), Disco-RAG scores 54.62 versus 35.61 for standard RAG. Averaged across all four sets, it surpasses the previously reported training-based method StructRAG (62.07 vs. 60.38).

• ASQA benchmark gains: With Llama-3.1-8B, EM/RL/DR scores rise from 37.3/36.9/23.4 to 40.4/42.2/32.6. With Llama-3.3-70B, EM reaches 42.0 and DR 32.8, and the RL score of 42.3 exceeds MAIN-RAG (42.0 RL) and Tree of Clarifications (39.7 RL).

• SciNews benchmark gains: Using Llama-3.3-70B, Disco-RAG obtains 21.11 RL, 65.67 BERTScore, 44.37 SARI, and 69.48 SummaC, surpassing standard RAG and the previous best systems.

• Ablation results: The full model's overall LLM Score of 62.07 drops to 56.22 without the RST tree, 57.10 without the rhetorical graph, and 59.75 without planning. EM falls from 0.24 to 0.20–0.22 across ablated variants. In the long-document Set 4, removing the RST tree lowers the score from 54.62 to 47.63, removing the rhetorical graph gives 48.16, and removing the planner gives 50.34. Two generic planning baselines on standard RAG reach 50.64 (retrieve-and-plan) and 51.38 (plan-and-retrieve), both below the full Disco-RAG.

• Robustness to retrieval conditions: Standard RAG performs best at a chunk size of 256 tokens (49.33) and degrades with larger chunks, while Disco-RAG stays between 62.07 and 58.91 across chunk sizes. Both peak at Top-10 retrieval, but Disco-RAG remains robust up to Top-50. Under retrieval noise, standard RAG drops from 49.33 to 45.23, while Disco-RAG retains 56.17.

• Structure quality matters: Perturbing structure reduces performance. Shuffling rhetorical relation labels lowers the LLM Score from 62.07 to 55.48; altering nucleus–satellite roles gives 55.15; removing a subtree connection gives 56.77. For graphs: removing connections gives 57.60, flipping edge directions 55.82, replacing relation labels 55.50. For plans: omitting the plan gives 59.75, shuffling steps 57.50, removing steps 58.14. Even with corrupted signals, Disco-RAG still beats standard RAG and full context.

• Mixed-model deployment works: On Loong, standard RAG (70B/70B) scores 49.33 LLM Score and 0.17 EM. An all-8B Disco-RAG scores 58.94 and 0.20, an 8B structurer with a 70B generator scores 60.52 and 0.22, and the all-70B configuration scores 62.07 and 0.24. The gap between 8B+70B and all-70B is 1.55 LLM Score.

• Discourse structure complements fine-tuning: On SciNews, end-to-end SFT without discourse reaches 20.3 RL and 66.8 SummaC; training-free Disco-RAG reaches 21.1 and 69.5; SFT with test-time discourse reaches 22.8 and 72.3; SFT with discourse at both training and test time reaches 23.3 and 74.0.

• Human evaluation: On 15 sampled SciNews test articles rated by three graduate students on a three-point Likert scale, Disco-RAG scores 2.40 (Relevance), 2.43 (Simplicity), 2.27 (Conciseness), and 2.53 (Faithfulness), versus 1.87/2.12/1.60/1.67 for standard RAG, 1.65/1.98/1.52/1.45 for full context, and 2.89/2.63/2.48/2.88 for human references. Fleiss' kappa was 0.73, 0.65, 0.66, and 0.68 across the four dimensions.

Methodology in Plain English

The framework adds three stages on top of a standard RAG pipeline, all at inference time with no task-specific fine-tuning required.

1. Retrieval. The query is used to retrieve the top-k chunks (Top-10 in the main setup) from a corpus using cosine semantic similarity with Qwen3-Embedding-8B at a 256-token chunk size without sliding window.

2. Intra-chunk RST trees. An LLM-based parser splits each chunk into elementary discourse units (EDUs) and assigns nucleus/satellite roles and rhetorical relations such as Elaboration, Contrast, and Cause, forming a tree per chunk. These trees are constructed offline.

3. Inter-chunk rhetorical graph. All retrieved chunks are given to the parser in a single listwise pass, and the parser predicts a directed rhetorical relation or an UNRELATED label for every ordered chunk pair, producing a graph over chunks.

4. Discourse-driven planning. A planning module takes the query, chunks, RST trees, and rhetorical graph and generates a blueprint that selects salient content, organizes argumentative flow, and prioritizes supporting evidence.

5. Discourse-guided generation. The final answer is generated conditioned on the query, the raw chunks, the RST trees, the rhetorical graph, and the plan.

The intuition is that rhetorical modeling acts as a knowledge-level prior while planning offers reasoning-level guidance, jointly inducing stronger structural biases than flat concatenation. The structural modules and the generator are decoupled, so different models can be used for each.

Why This Matters

Impact on research: The work reframes RAG as a discourse-organization problem rather than just a retrieval and concatenation problem, and shows that rhetorical structure theory can be operationalized inside an LLM pipeline. It provides a modular design that can be combined with fine-tuning rather than replacing it, and it reports gains without in-domain fine-tuning.

Real-world applications:

• Long-document question answering over reports, filings, or technical documentation where evidence spans hundreds of thousands of tokens.
• Scientific lay summarization that must be accurate and accessible for general audiences, as tested on SciNews.
• Domain-specific assistants in medicine, law, or policy, where conflicting or conditional claims must be reconciled rather than merged.
• Robust enterprise search where retrieval noise, duplicate content, or mismatched chunk sizes are common.

Industry relevance: The mixed-model results show that an 8B model can serve as the structural backbone while a 70B model handles final generation, retaining most of the benefit (60.52 vs. 62.07 LLM Score on Loong) and offering a cost-reduction path for resource-constrained deployments. The robustness to chunk size, Top-k, and retrieval noise suggests the approach tolerates imperfect production retrieval.

Future Directions

• Reducing the remaining scaling gap: The paper reports a 1.55 LLM Score gap between the 8B+70B and all-70B configurations, leaving open how much structural quality could be recovered with cheaper structural models.

• Bias and label-distribution analysis: The limitations section states that the authors do not conduct a dedicated analysis of potential biases in the content or label distributions of Loong, ASQA, and SciNews. The provided content is truncated mid-sentence at this point, so additional limitations beyond the data scope are not reported.

• Parsing quality and shallow-cue comparisons: The authors point to appendix discussion of parsing evaluation, shallow discourse marker analysis, and significance testing, which are natural targets for further verification of whether deep structure beats surface cues.

• Human quality ceiling: Human references remain the strongest in the human evaluation on all four dimensions, indicating room for improvement in perceived relevance, simplicity, conciseness, and faithfulness.

Target Audience

Researchers and practitioners working on RAG systems, long-context LLM reasoning, discourse parsing, and knowledge-intensive generation. It is also relevant to engineers deploying retrieval-augmented assistants who care about robustness to chunk size, retrieval depth, and noisy retrieval, and to readers interested in how discourse theory from computational linguistics can be applied inside modern LLM pipelines.

Authors’ abstract

Retrieval-Augmented Generation (RAG) has emerged as an important means of enhancing the performance of large language models (LLMs) in knowledge-intensive tasks. However, most existing RAG strategies treat retrieved passages in a flat and unstructured way, which prevents the model from capturing structural cues and constrains its ability to synthesize knowledge from dispersed evidence across documents. To overcome these limitations, we propose Disco-RAG, a discourse-aware framework that explicitly injects discourse signals into the generation process. Our method constructs intra-chunk discourse trees to capture local hierarchies and builds inter-chunk rhetorical graphs to model cross-passage coherence. These structures are jointly integrated into a planning blueprint that conditions the generation. Experiments on question answering and long-document summarization benchmarks show the efficacy of our approach. Disco-RAG achieves state-of-the-art results on the benchmarks without fine-tuning. These findings underscore the important role of discourse structure in advancing RAG systems.