Rethinking Table Pruning in TableQA: From Sequential Revisions to Gold Trajectory-Supervised Parallel Search

AI summary

Overview

Research area: Natural Language Processing, specifically Table Question Answering (TableQA) and table pruning for structured-data reasoning.

Technical level: Advanced. The paper combines supervised fine-tuning, Direct Preference Optimization, a regression-based verifier, and beam search over candidate sub-tables.

Scope: The paper introduces a table pruning framework, TabTrim, that replaces step-by-step sequential revision with gold-trajectory-supervised parallel search, and evaluates it on WikiTQ, TabFact, and TableBench.

What This Paper Is About

TableQA systems take a raw table plus a natural-language question and must reason over only the small portion of the table that actually matters. The paper argues that existing pruning methods refine their output step by step using unreliable "critique" signals — execution errors for program-based methods, or an LLM judging its own steps for LLM-based methods — and that this lets them silently delete answer-critical cells without noticing. TabTrim's goal is to supervise pruning against an objective reference and to explore many pruning paths at once instead of committing to a single one.

Key Contributions

1. New insight on the pruning problem. The authors reframe multi-step table pruning from a sequential revision process into a gold trajectory-supervised parallel search, arguing that the two root causes of failure are unreliable critique and the inability to backtrack.
2. Automatic construction of gold pruning trajectories. The framework derives step-level sub-table supervision from single-table Text-to-SQL data by decomposing gold SQL queries into clause-level operations and executing them in logical order, producing a trajectory without additional manual annotation. Off-trajectory negatives are produced by modifying those clauses (e.g., replacing predicate values, swapping nearby operators, dropping or adding conjuncts, dropping or adding projected columns).
3. A two-component framework (TabTrim). A trajectory-supervised pruner trained with supervised fine-tuning plus Direct Preference Optimization, and a loss-aware verifier trained to regress an α-weighted F-score that measures cell-set overlap with the gold final sub-table.
4. State-of-the-art results. TabTrim-8B reaches 73.5% average accuracy, outperforming the strongest baseline by 3.2%, with 79.4% on WikiTQ and 61.2% on TableBench.

Main Findings

• Best overall performance. TabTrim-8B achieves 73.5% average accuracy across WikiTQ, TabFact, and TableBench, beating the strongest non-TabTrim baseline (Table-Critic, 70.3%) by 3.2%. TabTrim-4B reaches 70.8% average and already surpasses all non-TabTrim methods.
• Large WikiTQ gain. TabTrim-8B scores 79.4% on WikiTQ, exceeding the best baseline by 6.8%.
• Strong TabFact result. TabTrim-8B reaches 91.2% on TabFact.
• Best on every TableBench subtask. Compared with the strongest baseline, TabTrim-8B improves TB-NR by 5.8%, TB-FC by 2.0%, and TB-DA by 0.9%. The authors attribute the large TB-NR gain to the sensitivity of multi-hop numerical reasoning to retaining the right operands.
• Scaling with pruner size. Upgrading the pruner from 4B to 8B yields a 2.5% average gain across datasets.
• Hardness stratification. WikiTQ questions were bucketed by GPT-4o success over 100 sampled answers: Easy (90–100 correct), Medium (60–89), Hard (10–59), Extra Hard (0–9). TabTrim-8B outperformed Table-Critic and TALON at every level, with the largest margin in the Extra Hard bucket.
• Ablations on pruner training (WikiTQ / TableBench, TabTrim baseline 79.4 / 61.2): removing DPO drops to 78.1 (-1.3) and 58.6 (-2.6); removing correction samples drops to 74.8 (-4.6) and 55.4 (-5.8); removing training entirely collapses to 54.7 (-24.7) and 49.6 (-11.6).
• Loss-aware scoring matters. Replacing the default loss-aware score (α = 1.5) with a balanced score (α = 1) lowers accuracy to 77.8 (-1.6) on WikiTQ and 58.3 (-2.9) on TableBench.
• Search and ranking signal matter. Ranking candidates by pruner likelihood instead of the verifier gives 74.2 (-5.2) on WikiTQ and 56.7 (-4.5) on TableBench; disabling search (sequential revisions, k = b = 1 with a matched call budget) gives 72.9 (-6.5) and 55.1 (-6.1).
• Positive compute scaling. With k = b = 2, increasing maximum depth D_max from 1 upward improved accuracy monotonically on both WikiTQ and TableBench; fixing D_max = 4 and b = 2 while increasing beam width k also improved performance. TabTrim also beat a Best-of-N baseline (N = k · b = 4) at matched compute across all depths.
• Plug-and-play gains for downstream reasoners. Using TabTrim(8B)-selected sub-tables: Qwen3 goes from 52.2 to 78.1 on WikiTQ (+25.9) and 48.4 to 59.2 on TableBench (+10.8); Table-R1 goes from 77.5 to 84.9 on WikiTQ (+7.4) and 34.3 to 43.1 on TableBench (+8.8).
• Lower token cost than Table-Critic. On WikiTQ, Table-Critic used 135.5M input and 3.8M output tokens versus TabTrim's 63.8M input and 14.5M output tokens, a 0.56× ratio in total. On TabFact, 62.1M input and 2.4M output tokens versus 32.9M and 9.7M, a 0.66× ratio.
• Hyperparameter sensitivity. For the correction weight λ, WikiTQ/TableBench scores were 76.5/58.1 at 0.3, 78.1/60.8 at 0.5, 78.8/61.7 at 0.8, and 79.4/61.2 at 1.0. For α, scores were 77.8/58.3 at 1.0, 78.6/59.7 at 1.2, 79.4/61.2 at 1.5, 78.9/61.5 at 1.7, and 78.7/60.4 at 2.0.

Methodology in Plain English

The core idea is that there is a hidden "correct path" through pruning, and the authors can recover it automatically. Starting from existing Text-to-SQL datasets, they parse each gold SQL query into an ordered list of clause-level operations — row filters from WHERE and column projections covering SELECT, GROUP BY, HAVING, and aggregates — then execute those operations one at a time. The intermediate tables produced along the way become the gold pruning trajectory, a sequence of increasingly trimmed sub-tables that is guaranteed to preserve answer-critical data because it is grounded in a verified query result.

From this trajectory they build two training sets: a progression set, where the model sees a correct intermediate sub-table and must produce the next correct one, and a correction set, where the model sees a deliberately wrong sub-table (produced by perturbing the SQL clause, e.g., dropping a needed column) and must still recover to the correct next state. The pruner is trained on both with a supervised loss, then refined with Direct Preference Optimization so that it prefers the gold next sub-table over the incorrect one under the same context.

A separate, smaller model acts as the verifier. To give it objective targets, each sub-table is canonicalized into a set of indexed cells (row index, column index, value), and precision and recall are computed against the gold final sub-table by cell-set overlap. These are combined into an α-weighted F-score, where α > 1 penalizes missing answer-critical data more than retaining redundant cells. The verifier is trained by regression to predict that score.

At inference, the system does not take a single path. It runs a beam search: at each depth the pruner proposes several candidate next sub-tables for every beam entry, the verifier scores each candidate, and only the top-k survive and expand further. Finally, the highest-scoring sub-table found anywhere in the search is passed, along with the question, to an LLM to produce the answer.

Training data: over 80K samples built from WikiSQL and SQUALL. Pruners: Qwen3-4B and Qwen3-8B, fully fine-tuned. Verifier: Qwen3-0.6B, with α = 1.5. Default search settings: beam width k = 2, branching factor b = 2, maximum depth D_max = 4, giving an upper bound of O(k · b · D_max) pruner and verifier calls per example. All experiments on 8 NVIDIA A100 40GB GPUs, 2 epochs with AdamW and a cosine learning rate schedule with 3% warmup.

Datasets and metrics: WikiTQ (4,344 samples from 421 Wikipedia tables), TabFact (2,024 test samples from 298 tables), and TableBench (886 questions covering numerical reasoning, fact checking, and data analysis). Exact match accuracy is used for WikiTQ and TableBench; binary classification accuracy for TabFact. All baseline methods were executed with GPT-4o-mini as the reasoning backbone for a fair comparison, and TabTrim also uses GPT-4o-mini to generate the final answer.

Why This Matters

This work reframes table pruning as a search problem with objective supervision rather than a self-critique loop, and shows that the intermediate states of a gold program can be repurposed as free process-level labels. If that idea generalizes, it reduces reliance on LLM-as-a-Judge signals, which the paper argues are prone to both rationalizing errors and over-criticizing valid steps.

Real-world applications:

• Enterprise analytics assistants that answer natural-language questions over large spreadsheets or database exports, where answering from the raw table is noisy and error-prone.
• Financial and business reporting, where the relevant rows and columns must be isolated precisely and losing a figure changes the answer.
• Scientific and clinical data exploration over wide tables of measurements, where multi-hop numerical reasoning is common (the TB-NR subtask is where the largest TableBench gain appears).
• Reducing inference cost in production LLM pipelines: the paper reports total token consumption of 0.56× and 0.66× of Table-Critic's on WikiTQ and TabFact respectively.

Industry relevance: TabTrim is described as an open-source, plug-and-play pruning module — the plug-and-play experiment shows it lifted two different downstream reasoners on both datasets, and the 4B and 8B pruner sizes are practical for deployment. The reported token savings matter for serving costs, though the paper reports token counts rather than latency or wall-clock time.

Future Directions

• Scaling to larger base models. The authors explicitly list this as a limitation: evaluation is confined to the 4B and 8B parameter regimes because of computational cost, so the scaling laws and performance ceiling with stronger base models remain uncharacterized.
• Relaxing the dependence on Text-to-SQL data. Gold trajectories are derived from single-table Text-to-SQL datasets (WikiSQL, SQUALL) with clause-level operations restricted to WHERE predicates and column projections. Extending to multi-table schemas, joins, and other query forms is not addressed.
• Search budget efficiency. Beam width, branching factor, and depth were explored, and TabTrim beat Best-of-N at matched compute, but no guidance is given on how to choose k, b, and D_max per task or adapt them per example.
• Combining with end-to-end reasoners. The plug-and-play results suggest a component pipeline; whether pruning supervision and reasoning could be trained jointly rather than as separate stages is left open.

Target Audience

Researchers and engineers working on table understanding, structured-data question answering, and retrieval-augmented or tool-augmented LLM pipelines. It will be especially useful to practitioners building production TableQA systems who need pruning that preserves answer-critical cells, and to researchers interested in deriving process-level supervision from executable programs. Readers should be comfortable with supervised fine-tuning, Direct Preference Optimization, and beam search; the paper's motivational framing is accessible, but the training and inference machinery is not introductory.

Authors’ abstract

Table Question Answering (TableQA) benefits significantly from table pruning, which extracts compact sub-tables by eliminating redundant cells to streamline downstream reasoning. However, existing pruning methods typically rely on sequential revisions driven by unreliable critique signals, often failing to detect the loss of answer-critical data. To address this limitation, we propose TabTrim, a novel table pruning framework which transforms table pruning from sequential revisions to gold trajectory-supervised parallel search. TabTrim derives a gold pruning trajectory using the intermediate sub-tables in the execution process of gold SQL queries, and trains a pruner and a verifier to make the step-wise pruning result align with the gold pruning trajectory. During inference, TabTrim performs parallel search to explore multiple candidate pruning trajectories and identify the optimal sub-table. Extensive experiments demonstrate that TabTrim achieves state-of-the-art performance across diverse tabular reasoning tasks: TabTrim-8B reaches 73.5% average accuracy, outperforming the strongest baseline by 3.2%, including 79.4% on WikiTQ and 61.2% on TableBench.