TabDSR: Decompose, Sanitize, and Reason for Complex Numerical Reasoning in Tabular Data

AI summary

Overview

Research area: Natural Language Processing / Table Question Answering (TQA), specifically numerical reasoning over tabular data with large language models.

Technical level: Intermediate — familiarity with LLM prompting paradigms (Chain-of-Thought, Program-of-Thought) and standard table reasoning benchmarks helps, but the paper's core ideas are conceptually straightforward.

Scope: This paper proposes TabDSR, a three-agent prompting framework that improves LLM numerical reasoning over complex, noisy tables, and introduces a new leakage-resistant benchmark dataset called CalTab151.

What This Paper Is About

Large language models frequently fail at answering numerical questions over tables because real-world queries require multi-step ("multi-hop") reasoning, tables are noisy or inconsistently formatted, and LLMs imitate arithmetic rather than genuinely compute it. The authors build TabDSR, a prompt-only pipeline that mimics how humans tackle such problems: breaking a complex question into smaller ones, cleaning up the table, and then writing and executing code to produce an exact numerical answer. Their goal is to raise accuracy on complex tabular numerical reasoning without fine-tuning or large-scale annotation.

Key Contributions

1. The TabDSR framework — a three-agent pipeline combining a Query Decomposer Agent (splits complex questions into sub-questions using only the question text), a Table Sanitizer Agent (repairs nested headers, segmented tables, and messy cell content), and a PoT-based Reasoner Agent (generates and executes Pandas code to compute answers).

2. Broad empirical validation across model families — the framework is tested on Qwen2.5-7B, Qwen2.5-Code-7B, Qwen2.5-72B, GPT-4o, and DeepSeek-V3, demonstrating consistent gains regardless of model size or vendor, which the authors frame as evidence of transferability.

3. The CalTab151 dataset — a 151-sample benchmark constructed to minimize data leakage by perturbing numeric values, injecting noise, shuffling structure, filling nulls, and generating multi-hop questions, with all answers manually verified by human annotators.

4. State-of-the-art results on three benchmarks — TabDSR achieves accuracy improvements of 8.79%, 6.08%, and 19.87% on TAT-QA, TableBench, and CalTab151 respectively over the second-best method.

Main Findings

• State-of-the-art on all three benchmarks: TabDSR reaches 60.41% accuracy on TAT-QA, 52.75% on TableBench, and 45.70% on CalTab151, beating prior prompt-based, fine-tuned, and pre-trained baselines. ROUGE-L scores improve correspondingly (+4.59, +5.98, +18.97).

• Small models can beat larger ones: The 7B-parameter TabDSR configuration surpasses TableLLM-13B in both accuracy and ROUGE-L, showing that the framework's structure matters more than raw parameter count.

• Sanitization delivers the most reliable gains: In ablations, adding the Table Sanitizer (TabDSR S+R) consistently improved results over the reasoner alone across every model and dataset. Adding the Decomposer on top (D+S+R) produced further benefits in most configurations.

• Decomposition is not universally helpful: In a few model–dataset combinations, the Decomposer slightly hurt performance. The authors attribute this to chaotic tables (e.g., complex multi-level headers) where splitting the question before cleaning the table dilutes its value.

• Category ranking: Across benchmark categories, prompt-based LLMs generally outperform fine-tuned models, which in turn outperform pre-trained models. Fine-tuned models degrade sharply on unseen datasets (e.g., TableGPT2-7B and TableLLM-13B@PoT excel on TableBench but collapse on TAT-QA and CalTab151).

• Transferability to frontier models: TabDSR lifts GPT-4o to 80.96% accuracy on TAT-QA and DeepSeek-V3 to 83.67%, indicating the method is not merely compensating for weaker models.

• Evidence of data leakage in public benchmarks: Nearly every evaluated method performed worst on CalTab151, which the authors interpret as a sign that public datasets contain leakage that inflates reported scores.

Methodology in Plain English

The pipeline mirrors human table reasoning in three steps, each handled by a separate LLM prompt:

1. Query Decomposer receives only the question — not the table — to avoid the table's length overshadowing question details. It splits the query at conjunctions ("and", "or") and punctuation, and outputs sub-questions in a list format with a worked example to constrain the granularity and reduce hallucination.

2. Table Sanitizer performs structural and content cleanup. Structurally, it merges split multi-level headers and removes or extracts segmentation rows that separated sections in the original visual layout. For content, it strips currency symbols, percentage signs, commas, emojis, and explanatory notes, converts numerics to int/float, and normalizes blanks ("N/A", "–", "???") to a consistent "null" label. A reflection loop validates output through a Python parser; if parsing fails, the error is fed back for one regeneration attempt. The full table is retained rather than truncated, since modern LLMs handle long contexts.

3. PoT-based Reasoner loads the sanitized table into a Pandas DataFrame, generates Python code for each sub-question (filtering, aggregation, arithmetic), restricts certain Pandas methods to avoid version inconsistencies, and validates data formats. Results are reassembled in logical order to yield the final answer.

CalTab151 construction proceeds in six steps: (1) random numerical perturbation of ±3–5%, (2) injection of realistic symbols like $ and %, (3) row/column shuffling and deletion, (4) null-value substitution, (5) LLM-generated multi-hop questions where each hop depends on the previous answer, and (6) manual human verification of all answers. Samples draw from TableBench (84), FinQA (27), TAT-QA (32), and AitQa (8).

Evaluation uses accuracy and ROUGE-L on TAT-QA (736 table-only examples), a 493-sample subset of TableBench, and CalTab151 (151 samples). All prompt-based comparisons use Qwen2.5-7B as backbone unless stated otherwise.

Why This Matters

Impact on research: The paper challenges the reliability of widely used TQA benchmarks by showing that performance drops dramatically on a leakage-controlled dataset. It also demonstrates that modular prompt engineering — rather than fine-tuning — can substantially close the gap with specialized models, and that decomposition and data cleaning are separable, measurable contributors to reasoning performance.

Real-world applications:

• Financial analysis — extracting and computing figures from earnings reports, balance sheets, and regulatory filings where tables contain mixed formats and nested headers.
• Business intelligence — answering aggregate and trend questions over internal sales, inventory, or operational spreadsheets without dedicated engineering pipelines.
• Healthcare and clinical research — querying patient statistics, trial results, or hospital metrics where null values and inconsistent units are common.
• E-commerce and retail — comparing product specifications, pricing, and inventory tables that frequently mix currencies, units, and notes.

Industry relevance: Because TabDSR is prompt-only, it requires no labeled training data, no fine-tuning infrastructure, and no model-specific adaptation. This lowers the cost of deploying reliable natural-language querying over enterprise spreadsheets and databases, and it works with whichever frontier or open-source LLM an organization already uses.

Future Directions

• Expanding CalTab151 — increasing dataset size, diversifying question types, and covering more domains, since the current 151 samples are small and require costly manual verification.

• Hybrid question decomposition — developing a Decomposer that can selectively incorporate table signals while preserving the stability benefits of question-only decomposition.

• Robust sanitizer fallbacks — implementing call monitoring and default resolution strategies for cases where the LLM fails to repair a table even after reflection iterations, causing redundant calls.

• Closing the human-performance gap — since the method is prompt-only, its ceiling is bounded by the base LLM's reasoning ability, leaving room for techniques that more fundamentally improve numerical reasoning.

Target Audience

This paper is most valuable for NLP and LLM researchers working on table reasoning, question answering, or program-of-thought prompting; benchmark and dataset designers concerned with data leakage; and applied AI engineers and data analysts seeking a practical, no-fine-tuning recipe for natural-language querying of complex spreadsheets. Readers interested in agentic or multi-step prompting architectures will also find the modular decomposition design directly reusable.

Authors’ abstract

Complex reasoning over tabular data is crucial in real-world data analysis, yet large language models (LLMs) often underperform due to complex queries, noisy data, and limited numerical capabilities. To address these issues, we propose TabDSR, a framework consisting of: (1) a query decomposer that breaks down complex questions, (2) a table sanitizer that cleans and filters noisy tables, and (3) a program-of-thoughts (PoT)-based reasoner that generates executable code to derive the final answer from the sanitized table. To ensure unbiased evaluation and mitigate data leakage, we introduce a new dataset, CalTab151, specifically designed for complex numerical reasoning over tables. Experimental results demonstrate that TabDSR consistently outperforms existing methods, achieving state-of-the-art (SOTA) performance with 8.79%, 6.08%, and 19.87% accuracy improvement on TAT-QA, TableBench, and TabDSR, respectively. Moreover, our framework integrates seamlessly with mainstream LLMs, providing a robust solution for complex tabular numerical reasoning. These findings highlight the effectiveness of our framework in enhancing LLM performance for complex tabular numerical reasoning. Data and code are available upon request.