BRIEF-Pro: Universal Context Compression with Short-to-Long Synthesis for Fast and Accurate Multi-Hop Reasoning

AI summary

Overview

• Research area: Natural Language Processing — retrieval-augmented generation (RAG), context compression, and multi-hop question answering
• Technical level: Intermediate (requires familiarity with LLMs, RAG pipelines, and fine-tuning concepts)
• Scope: The paper introduces BRIEF-Pro, a lightweight 3B-parameter context compressor trained on synthetically expanded data to compress 10k+ word retrieval contexts into short, query-relevant summaries for multi-hop reasoning.

What This Paper Is About

RAG systems improve LLM answers by prepending retrieved documents, but as more documents are retrieved, contexts balloon, slowing inference and overwhelming the model — particularly for multi-hop questions that require reasoning across several sources. BRIEF-Pro addresses this by learning to distill long retrieved contexts into compact, query-focused summaries that smaller, larger, and proprietary reader models can process faster and more accurately.

Key Contributions

1. Long-context compression for multi-hop reasoning: The first context compressor explicitly designed to handle 10k+ word inputs across diverse RAG scenarios, rather than the shorter contexts addressed by prior work.
2. A short-to-long synthetic data pipeline: A method that starts from short seed documents (under 1k words) and expands them into coherent 10k+ word training examples by stitching in surrounding Wikipedia content and distractor documents.
3. User-controllable compression: An instruction-conditioned paradigm where users specify the exact number of sentences for the summary, alongside an "Auto" mode where the model decides the length.
4. A lightweight 3B compressor that generalizes: A Llama-3.2-3B-Instruct fine-tune that produces textual summaries transferable to any frozen reader model, with strong efficiency gains versus much larger baselines.

Main Findings

• Compression outperforms no compression: BRIEF-Pro-Auto achieves roughly 32x compression while improving average QA F1 by 6.70%, 0.60%, and 7.27% over non-compressed inputs with 8B, 70B, and GPT-4.1-nano readers, respectively.
• Beats LongLLMLingua at a fraction of the cost: With the 70B reader, BRIEF-Pro-Auto compresses 32x (vs. LongLLMLingua's 9x), improves QA by 4.67% on average, and uses only about 23% of LongLLMLingua's computational overhead.
• Large TFLOPs reductions: Total compute drops to 45% and 8% of the uncompressed baseline with 8B and 70B readers respectively; end-to-end latency with the 70B reader falls to 14%, 32%, and 7% of Non-Compression, Rerank Top-5, and LongLLMLingua respectively.
• Expanding both oracle and distractor documents matters: Training on contexts that mix helpful and irrelevant documents yields a 6.0k-word average context length and clearly better QA than expanding only oracle documents (3.6k words) — the noisier setup generalizes better.
• User-controllable length works reasonably well: Requested 5-sentence and 10-sentence summaries produce averages of 6.2 and 10.4 sentences respectively; precise control at higher verbosity (20 sentences) is harder (18.0 average).
• Transferability across reader scale: The same frozen compressor helps 8B, 70B, and proprietary GPT-4.1-nano readers, showing that compression quality, not reader size, drives the gain.

Methodology in Plain English

The researchers built a training data generator rather than collecting new labeled data. Starting from existing multi-hop QA datasets (MuSiQue, HotpotQA, LongAlign) with short oracle documents, they:

1. Expanded documents by locating each document's source Wikipedia page and pulling in a random number of surrounding sentences (sampled from a normal distribution centered on a 20x expansion ratio), producing genuinely long, coherent contexts.
2. Curated compact target summaries by iteratively pruning head and tail sentences from oracle documents whenever removing a sentence did not hurt an LM's likelihood of producing the correct answer — keeping only the central, helpful content.
3. Generated instruction-tuning pairs by counting the sentences in each pruned summary and pairing it with the instruction "Summarize the documents relevant to the question in K sentences" where K is that count.

They then fine-tuned Llama-3.2-3B-Instruct with LoRA for three epochs on 45.2k examples (about two days on 2× A100 GPUs). At inference, a frozen reader model (8B, 70B, or GPT-4.1-nano) answers the question using only the compressed summary in place of the full retrieved context. Evaluation used Exact Match and F1 on four multi-hop datasets (MuSiQue, HotpotQA, 2WikiMultiHopQA, LongSeal).

Why This Matters

• Research impact: This is the first work to demonstrate that a small, distilled compressor can handle contexts 10x longer than its seed data by leveraging synthetic expansion, opening a path toward compression as a standard RAG component rather than an afterthought. It also shows that compressing both signal and noise is crucial — a counterintuitive but practical insight for RAG training.
• Real-world applications:
  • Enterprise search and internal knowledge assistants where latency and per-query cost dominate
  • Multi-hop legal, medical, or scientific QA over large document corpora
  • Customer-support agents that must reason across multiple product manuals or tickets
  • On-device or resource-constrained RAG where the reader model has limited context windows
• Industry relevance: BRIEF-Pro reduces both inference cost and latency by large margins while improving accuracy, which directly maps to lower serving bills and better user experience for any production RAG deployment. Its textual (rather than soft-prompt) summaries also make it compatible with black-box APIs like GPT-4.1.

Future Directions

• Scaling to 20k+ word contexts: The authors acknowledge that performance may degrade on inputs far longer or more complex than the training data.
• Beyond RAG: The compressor is untested on few-shot learning, long-dialogue history, or code completion, where exact syntactic fidelity is required and abstractive compression may lose critical details.
• Tighter length control: Achieving precise adherence to user-specified sentence counts at higher verbosity (e.g., 20 sentences) remains an open challenge.
• Alternative evidence sources: The pipeline currently relies on Wikipedia; extending it to other authoritative web corpora could broaden coverage and reduce topic bias.

Target Audience

• NLP and IR researchers working on RAG efficiency, context compression, or multi-hop reasoning
• ML engineers deploying RAG systems who need to cut latency and inference cost without sacrificing answer quality
• Practitioners building LLM agents or assistants over large document collections
• Graduate students looking for a clean example of synthetic data curation combined with lightweight fine-tuning

Authors’ abstract

As retrieval-augmented generation (RAG) tackles complex tasks, increasingly expanded contexts offer richer information, but at the cost of higher latency and increased cognitive load on the model. To mitigate this bottleneck, especially for intricate multi-hop questions, we introduce BRIEF-Pro. It is a universal, lightweight compressor that distills relevant evidence for a given query from retrieved documents into a concise summary for seamless integration into in-context RAG. Using seed data consisting of relatively short contexts (fewer than 1k words), BRIEF-Pro is trained to perform abstractive compression of extended contexts exceeding 10k words across a wide range of scenarios. Furthermore, BRIEF-Pro offers flexible user control over summary length by allowing users to specify the desired number of sentences. Experiments on four open-domain multi-hop question-answering datasets show that BRIEF-Pro generates more concise and relevant summaries, enhancing performance across small, large, and proprietary language models. With the 70B reader model, 32x compression by BRIEF-Pro improves QA performance by 4.67% on average over LongLLMLingua's 9x, while requiring only 23% of its computational overhead.