AGENTIQL: An Agent-Inspired Multi-Expert Framework for Text-to-SQL Generation

AI summary

Overview

• Research area: Natural Language Processing, specifically text-to-SQL (NL2SQL) semantic parsing and LLM-based multi-agent system design.
• Technical level: Intermediate. The paper assumes familiarity with large language model prompting, execution-accuracy evaluation, SQL, and the Spider benchmark, but the architecture is described conceptually and does not require deep mathematical background.
• Scope: The paper proposes and evaluates AgentiQL, a modular multi-expert pipeline that decomposes natural language questions into sub-questions, generates and merges sub-queries, refines column selection, and routes queries adaptively between a modular pipeline and a single-step baseline on the Spider benchmark.

What This Paper Is About

Monolithic LLM architectures for text-to-SQL often struggle with complex reasoning and diverse database schemas, while static ensemble methods add significant computational overhead and large models are costly to serve. Most existing systems also offer limited interpretability, making it hard to see why a particular SQL query was produced. AgentiQL addresses this by splitting query generation into specialized components — a reasoning agent that decomposes questions, a coding agent that writes sub-queries, a merge step, a column-selection refinement, and a learned router that decides when the expensive pipeline is actually worth invoking.

Key Contributions

1. A Divide-and-Merge module in which a reasoning agent decomposes a natural language question into sub-questions and a coding agent generates corresponding sub-queries that are then merged into a final SQL query, exposing intermediate reasoning steps for interpretability.
2. A Column Selection (CS) refinement step that adjusts the SELECT clause, including column choices and ordering, in the final SQL query to better align with user intent and raise execution accuracy.
3. An Adaptive Routing mechanism that directs each query either to the modular divide-and-merge pipeline or to a one-step baseline parser, using a learned router such as an XGBoost classifier or a reasoning-agent "judge," to balance accuracy against efficiency.
4. An empirical evaluation on Spider comparing divide-and-merge configurations, merging strategies, column selection, and routing against Qwen2.5 baselines of 7B, 14B, and 32B scale.

Main Findings

• Column Selection consistently helps. Adding the CS step improved execution accuracy across model sizes and both merging strategies, typically by 2-5% EX, with gains of up to 9% reported relative to models without refinement. For example, 7B models with the Last Sub-query merge rose from 72.26% to 74.44%.
• Planner&Executor needs CS to shine. Without CS, Planner&Executor sometimes underperformed the simpler Last Sub-query strategy (66.77% vs 72.26% at 7B), but with CS it surpassed it (75.85% vs 74.44% at 7B). The paper attributes this to the planner occasionally introducing extraneous or misordered columns that the column-selection step fixes.
• Full Table 1 results. With CS, Planner&Executor reached 75.85% (7B), 77.16% (14B), and 79.77% (32B); Last Sub-query with CS reached 74.44% (7B), 76.61% (14B), and 78.57% (32B). Corresponding no-CS Planner&Executor values were 66.77% (7B), 74.27% (14B), and 75.58% (32B); no-CS Last Sub-query values were 72.26% (7B), 69.33% (14B), and 73.84% (32B).
• Baselines were strong in the divide-and-merge-only comparison. In Table 3, the one-step baseline scored 76.4% (7B), 80.80% (14B), and 79.93% (32B) EX on the Spider test set, versus the divide-and-merge module's best values of 75.85%, 77.16%, and 79.77% respectively.
• The two approaches make different mistakes. Table 2 shows that with 7B models, AgentiQL succeeded where the baseline failed on 6.96% (Last Sub-query) and 7.34% (Planner&Executor) of instances, while the baseline succeeded where AgentiQL failed on 8.91% and 8.21%. For 14B the corresponding figures were 5.00%/4.78% (AgentiQL only) and 9.10%/8.91% (baseline only); for 32B, 4.94%/4.78% and 6.30%/5.54%. The paper concludes both methods capture distinct strengths and that Planner&Executor reduces the gap.
• Reported headline accuracy. The abstract states AgentiQL achieves up to 86.07% EX with 14B models using the Planner&Executor merging strategy. Note that the 86.07% figure appears in the abstract; the Tables 1-3 values reported for the 14B Planner&Executor configuration with CS are 77.16% for the divide-and-merge module alone, so the 86.07% number likely reflects the full routed system rather than the module in isolation (the paper's truncated content does not break this down explicitly).
• Routing narrows the gap to SOTA. The paper compares against XiYan-SQL with GPT-4o at 89.65% EX on Spider. Figure 8 shows AgentiQL's execution accuracy rising as the router's accuracy increases, approaching the SOTA system despite using much smaller open-source models.
• Schema complexity correlates with relative performance. Table 4 reports Pearson and Spearman coefficients between schema complexity (measured by number of tables) and AgentiQL's improvement over the baseline. For 7B models the correlations were negative (Pearson -0.61 and -0.39; Spearman -0.74 and -0.79), while for 14B and 32B models they were positive (14B: 0.46 and 0.37 Pearson, 0.70 and 0.52 Spearman; 32B: 0.35 and 0.12 Pearson, 0.67 and 0.64 Spearman).
• Accuracy-efficiency trade-off. Planner&Executor with CS yields the highest accuracy at each model size but incurs more latency than the Last Sub-query heuristic. The paper notes that several pipeline steps can be executed in parallel, making the framework scalable to larger workloads.
• Failure modes. Qualitative analysis presents two failure cases: an ambiguous unqualified agency_id column after a join between Agencies and Clients, and a case where the divide-and-merge output returned building names in addition to the requested number of stories.
• Cost of very large models. The paper does not fully explore models beyond 32B due to resource constraints, noting that in a preliminary trial a 235B-parameter model took nearly 60 minutes per question on four A100 GPUs with CPU offloading.

Methodology in Plain English

The researchers start from a standard setup: given a natural language question plus a database schema, produce an executable SQL query. Rather than asking one large model to do everything, they break the job into stages handled by different model roles.

First, a reasoning model acts as a filter, removing tables from the schema that are unlikely to be relevant, producing a reduced schema used by every later stage. Second, a second reasoning model decomposes the original question into a set of smaller natural language sub-questions, chosen so that answering each one and combining the answers yields the answer to the original question. Third, a coding-specialized model translates each sub-question into a SQL sub-query; if a generated query errors, the same model retries the correction for up to three iterations (R = 3).

The sub-queries then need to be combined. The paper tests two merging approaches. The simple one, "Last Sub-query," just takes the SQL produced for the final sub-question, assuming the reasoning agent ordered the sub-questions so the last one is the complete solution. The more general one, "Planner&Executor," uses a reasoning model as a planner to describe how the sub-queries should be combined and a coding model as an executor to turn that plan into the final SQL.

After merging, a refinement pass called Column Selection asks a reasoning model to rewrite the SELECT clause so the output columns and their order match exactly what the question asked for. Finally, a router — implemented as an XGBoost classifier or a reasoning-agent judge — decides per query whether to run this whole pipeline or to hand the query to a simple one-step few-shot baseline that maps text directly to SQL using a Qwen2.5-Coder model.

Evaluation uses execution accuracy (EX) on Spider, counting a predicted query correct if it produces the same result as the ground-truth query. All local experiments ran on an internal cluster with eight NVIDIA A100 GPUs of 80GB each, totaling approximately 1450 GPU-hours for the open-source models.

Why This Matters

Impact on research. The paper argues that monolithic LLM architectures are a bottleneck for complex text-to-SQL reasoning, and demonstrates that decomposition plus selective refinement can match much larger systems on execution accuracy. It also reframes the design space around explicit trade-offs — routing lets a system spend compute only where it is needed, and the exposure of sub-questions and sub-queries makes failures diagnosable in a way that a single opaque generation step is not. The authors state they will release code and prompt templates publicly for reproducibility.

Real-world applications:

• Business intelligence and analytics tools that let non-technical staff query warehouses in plain language.
• Customer-facing data assistants that translate questions into SQL over product, order, and customer databases.
• Internal data platforms where cost matters and a router can send simple queries down a cheap path while reserving the modular pipeline for complex ones.
• Auditing and compliance workflows, where the visible intermediate sub-questions and sub-queries provide a traceable record of how a query was derived.

Industry relevance. Serving very large LLMs is expensive; the paper's results suggest that a routed pipeline built on open-source models in the 7B-32B range can approach GPT-4-based state of the art while remaining deployable on modest hardware — a practical consideration for organizations that cannot or will not route every query through a closed-source API.

Future Directions

• Additional benchmarks. The authors evaluated primarily on Spider and explicitly identify testing on BIRD and SQL-Eval as necessary to assess generalizability.
• Larger and closed-source models. Experiments were limited to open-source LLMs up to 32B parameters. The paper proposes testing GPT-4o, Claude 4, Qwen3-235B-A22B-Instruct, and Qwen3-Coder-480B-A35B-Instruct to probe scalability.
• Reinforcement learning for query generation. The authors suggest incorporating RL, similar to what has been demonstrated in SkyRL-SQL, to provide adaptive feedback and improve generated query quality.
• Better merging and routing. The paper calls for investigating the effect of different merging strategies on accuracy and efficiency, and for exploring richer routing options such as XGBoost classifiers and retrieval-augmented generation to adapt to diverse question complexities and schema structures.

Target Audience

This paper is most useful to NLP and database researchers working on semantic parsing and text-to-SQL, and to machine learning engineers building LLM-powered data access products who need to weigh accuracy against inference cost. Practitioners designing multi-agent or prompt-chaining systems will find the divide-and-merge, column-selection, and routing components directly reusable. Readers looking for a dense theoretical treatment will not find one here; the contribution is architectural and empirical.

Authors’ abstract

LLMs have advanced text-to-SQL generation, yet monolithic architectures struggle with complex reasoning and schema diversity. We propose AGENTIQL, an agent-inspired multi-expert framework that combines a reasoning agent for question decomposition, a coding agent for sub-query generation, and a refinement step for column selection. An adaptive router further balances efficiency and accuracy by selecting between our modular pipeline and a baseline parser. Several steps in the pipeline can be executed in parallel, making the framework scalable to larger workloads. Evaluated on the Spider benchmark, AGENTIQL improves execution accuracy and interpretability and achieves up to 86.07% EX with 14B models using the Planner&Executor merging strategy. The attained performance is contingent upon the efficacy of the routing mechanism, thereby narrowing the gap to GPT-4-based SOTA (89.65% EX) while using much smaller open-source LLMs. Beyond accuracy, AGENTIQL enhances transparency by exposing intermediate reasoning steps, offering a robust, scalable, and interpretable approach to semantic parsing.