N2N-GQA: Noise-to-Narrative for Graph-Based Table-Text Question Answering Using LLMs

AI summary

Overview

Research area: Natural Language Processing — open-domain multi-hop question answering over hybrid table-and-text data, combining Retrieval-Augmented Generation (RAG) with graph-based evidence organization.

Technical level: Intermediate. The paper assumes familiarity with RAG pipelines, retrieval models (ColBERTv2), TF-IDF, and graph centrality, but its core argument is conceptual rather than mathematically heavy.

Scope: A zero-shot, training-free framework called N2N-GQA that builds and prunes dynamic evidence graphs from noisy retrieval outputs to improve multi-hop table-text QA, evaluated on HybridQA and OTT-QA.

What This Paper Is About

Standard RAG pipelines hand retrieved documents to a language model as a flat ranked list, which means the relationships between evidence pieces are invisible and retrieval noise can bury the reasoning chain. This paper argues that multi-hop questions — which require finding one fact, using it to find the next, and so on — need an explicit structure showing how documents connect. N2N-GQA builds a query-specific graph from retrieved passages and serialized table rows, then prunes it to a coherent "narrative" of connected evidence before the LLM synthesizes an answer.

Key Contributions

1. Graph-based evidence curation as the core finding. The authors systematically show that organizing retrieved documents as connected knowledge structures — rather than ranked lists — produces large gains on multi-hop reasoning, establishing graph organization itself as the essential ingredient.
2. A zero-shot framework, N2N-GQA. The pipeline includes structured query planning, iterative hop-by-hop evidence gathering, dynamic graph construction, and final context curation, with no task-specific training. The authors state that, to their knowledge, this is the first zero-shot framework for open-domain hybrid table-text QA.
3. GraphRank. A re-ranking refinement that multiplies a document's normalized semantic score by a factor derived from its normalized weighted degree centrality in the graph, providing consistent but modest complementary gains.
4. Competitive empirical results without fine-tuning. N2N-GQA reaches 48.80 EM on OTT-QA, which the authors report as matching CORE (49.0 EM) and approaching COS (56.9 EM), both of which require extensive task-specific training.

Main Findings

• Graph curation drives a large improvement. On OTT-QA with GPT-4.1, adding graph-based context curation (the N2N-GQA w/o GraphRank configuration) raises EM from 28.60 to 48.50 — a 19.9-point absolute gain over RAG with query decomposition. The paper describes this as its core finding.
• Vanilla RAG performs poorly on multi-hop questions. With GPT-4o on OTT-QA, Vanilla RAG scores 8.00 EM and 16.09 F1; with GPT-4.1, 8.00 EM and 15.68 F1; with Llama3-70B, 7.60 EM and 12.50 F1.
• Query decomposition alone helps substantially but not enough. On OTT-QA, RAG with Query Decomposition scores 31.40 EM / 43.07 F1 (GPT-4o), 28.60 EM / 40.82 F1 (GPT-4.1), and 37.60 EM / 47.70 F1 (Llama3-70B).
• Full pipeline results on OTT-QA. N2N-GQA w/ GraphRank scores 47.60 EM / 54.63 F1 (GPT-4o), 48.80 EM / 57.26 F1 (GPT-4.1), and 40.80 EM / 48.08 F1 (Llama3-70B). Without GraphRank the corresponding EM values are 47.40, 48.50, and 39.80.
• GraphRank's contribution is consistent but modest. The paper reports gains of roughly +0.3 to +1.0 EM across readers, and concludes that graph structure itself ("what is connected") matters far more than fine-grained centrality scoring ("how central").
• Hybrid-QA results follow the same pattern. With GPT-4o in a retrieval-based setting, Vanilla RAG scores 9.50 EM / 14.05 F1, RAG with Query Decomposition 22.00 EM / 31.06 F1, N2N-GQA w/o GraphRank 41.00 EM / 48.38 F1, and N2N-GQA w/ GraphRank 41.50 EM / 48.17 F1.
• Fine-tuned baselines remain ahead. On OTT-QA, BM25-HYBRIDER scores 10.3 EM / 13.0 F1, Fusion+Cross-Reader 28.1 EM / 32.5 F1, CARP 33.2 EM / 38.6 F1, CORE 49.0 EM / 55.7 F1, and COS 56.9 EM / 63.2 F1. The authors describe an 8.1-point gap between their 48.80 EM and COS.
• The benefit is reader-independent. Consistent improvements across GPT-4o, GPT-4.1, and Llama3-70B suggest graph curation addresses a limitation of list-based retrieval rather than a quirk of any single model.
• Failure modes. The authors identify three: retrieval failures when evidence is not in the top-k, entity ambiguity during intermediate extraction (e.g., "John Smith" matching multiple people), and table serialization limits where complex table structures lose relationships when flattened to text.

Methodology in Plain English

The system runs a question through four stages.

Query planning. An LLM is prompted to classify the question as 1-hop, 2-hop, or 3-hop and produce a machine-readable plan: an initial query, the expected type of the first intermediate entity, and templates with placeholders (e.g., {entity1}) for subsequent hops. The prompt instructs the model to be conservative about labeling questions 3-hop.

Iterative evidence gathering. Each hop's query is sent to a pre-indexed ColBERTv2 retriever. Table rows are serialized into semi-structured strings such as "Table:Population Stats|Row:5|City|New York|Population|8.4 million" so they can be indexed alongside text. The retrieved items become nodes in a temporary graph, and edges are weighted by TF-IDF overlap of shared terms. The authors chose TF-IDF deliberately over dense embeddings: it requires explicit lexical overlap, which reduces semantic drift and keeps "bridge" documents anchored to concrete entity references. It also needs no neural inference, is interpretable, and requires no training. Nodes are then re-ranked with GraphRank, and the graph is pruned to a small focused context that an LLM reads to extract the intermediate entity for the next hop.

GraphRank scoring. Each node's semantic score and structural (weighted degree centrality) score are min-max normalized to [0, 1]. The final score is multiplicative: normalized semantic score times (1 + (1 − α) × normalized structural score). The multiplicative form acts as a gate — structurally central but semantically irrelevant nodes are suppressed, which the authors say an additive formula would not achieve. α is set to 0.85 so the structural boost stays conservative.

Final curation and answering. Evidence from all hops is pooled, then a Bridge-Aware Hybrid Selector splits items into passages and tables. If the top passage contains at least one key entity (cell value) from the top table row, a priority boost β is added to the table; otherwise the boost is added to the passage and then the table. Items are re-sorted by the boosted score, a final global graph is built and pruned the same way, and the LLM synthesizes the answer from the curated context, question, and reasoning path.

Settings. For final synthesis, top k = 100 documents are retrieved, the top 50 unique items form the candidate graph, which is pruned to between 12 and 25 nodes with at least 2 passages and 2 tables. For intermediate hops, top k = 20 is retrieved and pruned to 5–10 nodes.

Evaluation. The authors use zero-shot settings only, sampling 200 questions from the HybridQA development set and 500 from OTT-QA development set, following the ODYSSEY protocol. They report EM, token-level F1, precision, recall, and BERTScore-F1 computed with bert-base-uncased. They note the OTT-QA subset alone requires approximately 2,500 LLM calls and defend the sample size by the magnitude of the effect (+19.9 EM). HybridQA is evaluated in a retrieval-based setting rather than the usual gold-evidence setting, to test end-to-end behavior.

Why This Matters

Impact on research. The paper reframes the bottleneck in multi-hop RAG. Rather than attributing failures to weak semantic understanding or weak generation, it argues the missing piece is a structured intermediate representation. That shifts attention from better retrievers and bigger readers toward evidence organization — and suggests that graph curation could be layered on top of fine-tuned retrievers and readers as a complementary component.

Real-world applications.

• Enterprise document QA across mixed spreadsheets, tables, and prose where answers require chaining facts across sources.
• Financial or market analysis, where a figure in one table must be linked to a narrative passage to answer a question.
• Customer-support and internal knowledge-base assistants that retrieve from heterogeneous repositories without a curated knowledge graph.
• Any domain where labeled training data does not exist and cannot be affordably created, since the pipeline requires no task-specific training.

Industry relevance. The authors frame the value proposition as scalability and adaptability rather than maximum accuracy. COS requires pretraining on Wikipedia-scale data, task-specific fine-tuning on the OTT-QA training set, and specialized retriever training; N2N-GQA needs none of that. For organizations without the resources or the annotated data for large-scale fine-tuning, the paper argues this offers a practical alternative that generalizes immediately to new domains.

Future Directions

• Richer graph structures. Moving beyond TF-IDF edges to entity-based graphs, typed edges (causal, temporal), or integration with external knowledge graphs.
• Better structural scoring. The authors note GraphRank's modest impact and suggest a dynamic weighting scheme for α, or alternative algorithms such as PageRank or betweenness centrality, to capture document importance in reasoning chains.
• Reducing computational cost. The framework depends on large LLMs; the paper suggests knowledge distillation to smaller models or methods that reduce the number of LLM calls. Llama3-70B's performance gap against GPT-4.1 illustrates the capability-versus-accessibility trade-off.
• Generalization beyond QA. Testing the noise-to-narrative paradigm on fact verification, document summarization, and knowledge base construction.

Target Audience

Researchers and practitioners working on retrieval-augmented generation, open-domain question answering, and multi-hop reasoning will get the most from this paper. It is also relevant to engineers building QA systems over heterogeneous enterprise data who lack the resources for task-specific fine-tuning, and to readers interested in how graph structure and symbolic scaffolding can supplement LLM reasoning. Familiarity with retrieval pipelines and basic graph concepts is helpful but not required to follow the central argument.

Authors’ abstract

Multi-hop question answering over hybrid table-text data requires retrieving and reasoning across multiple evidence pieces from large corpora, but standard Retrieval-Augmented Generation (RAG) pipelines process documents as flat ranked lists, causing retrieval noise to obscure reasoning chains. We introduce N2N-GQA. To our knowledge, it is the first zeroshot framework for open-domain hybrid table-text QA that constructs dynamic evidence graphs from noisy retrieval outputs. Our key insight is that multi-hop reasoning requires understanding relationships between evidence pieces: by modeling documents as graph nodes with semantic relationships as edges, we identify bridge documents connecting reasoning steps, a capability absent in list-based retrieval. On OTT-QA, graph-based evidence curation provides a 19.9-point EM improvement over strong baselines, demonstrating that organizing retrieval results as structured graphs is critical for multihop reasoning. N2N-GQA achieves 48.80 EM, matching finetuned retrieval models (CORE: 49.0 EM) and approaching heavily optimized systems (COS: 56.9 EM) without any task specific training. This establishes graph-structured evidence organization as essential for scalable, zero-shot multi-hop QA systems and demonstrates that simple, interpretable graph construction can rival sophisticated fine-tuned approaches.