Reasoning about Intent for Ambiguous Requests

AI summary

Overview

Research area: Natural Language Processing — ambiguity handling, intent reasoning, and reinforcement learning for large language models.

Technical level: Intermediate (familiarity with LLMs and basic RL concepts helps, but the core ideas are explained plainly in the paper).

Scope: The paper introduces IntentRL, a reinforcement-learning method that trains an LLM to emit structured interpretation-answer pairs when a request is ambiguous, evaluated on conversational question answering and text-to-SQL parsing.

What This Paper Is About

Large language models usually answer an ambiguous request by silently picking one interpretation, which can frustrate users and create safety risks if the chosen reading is wrong. This paper's goal is to train a model to reason about the user's intent first, and then output several explicit interpretations of the request, each paired with its own answer, in a single response.

Key Contributions

1. Structured interpretation-answer output format. Instead of clarification questions (which need extra interaction rounds) or long-form answers (which lack explicit interpretations), the model produces up to five pairs of an explicit interpretation plus its corresponding answer, in one generation step. This works for both textual answers and other output forms such as SQL queries.

2. A dual reward objective trained only on answer annotations. For inputs with multiple valid answers the reward is recall (coverage of valid interpretations); for inputs with a single gold answer the reward is precision (suppressing spurious alternatives). Supervision requires only multiple valid answers per input — no clarification questions and no explicit interpretation labels.

3. IntentRL itself. A model trained with DAPO (Decoupled Clip and Dynamic Sampling Policy Optimization, itself based on GRPO) on top of Qwen3 4B Instruct, using the Hungarian algorithm to compute an optimal one-to-one match between predicted answers and gold answers.

4. Evaluation across verifiable and non-verifiable tasks plus a human study. Comparisons against prompting baselines (IntentPrompt), supervised fine-tuning, and state-of-the-art clarification/multi-stage systems (ACT, DisambigParse), on Abg-CoQA, Ambrosia, and the out-of-domain AmbiQT benchmark.

Main Findings

• Best ambiguous-question coverage on Abg-CoQA. IntentRL reaches 72.9% word-overlap F1, 78.1% LLM-judge recall, 53.5% judge precision, and 61.0% full coverage on ambiguous questions, versus 55.2% / 63.1% / 47.0% / 45.5% for the strongest IntentPrompt baseline (235B Instruct).

• Strong single-intent recognition. On unambiguous Abg-CoQA questions IntentRL achieves 84.4% word-overlap F1 and 90.9% judge recall, showing it recognises questions with a single intent. Its 40.7% precision there reflects a bias toward generating extra interpretations.

• Prompting baselines over-detect ambiguity. On ambiguous Abg-CoQA questions the 235B Thinking IntentPrompt model gets 66.2% recall but only 49.7% precision, generating spurious answers.

• Supervised fine-tuning collapses to one answer. SFT on Abg-CoQA reaches 70.3% precision on ambiguous questions but only 48.1% recall and 20.3% full coverage — it learns to commit to a single answer rather than enumerate.

• Large gains on Ambrosia text-to-SQL. IntentRL reaches 82.4% recall, 77.5% precision, and 74.1% full coverage on ambiguous questions (88.7% recall and 74.7% precision on unambiguous), against 68.5% / 75.6% / 58.8% for SFT and 22.5% / 30.8% / 3.8% for the 4B Instruct IntentPrompt baseline.

• Prompting fails badly on text-to-SQL, and thinking models do not help. On Ambrosia, the 4B Thinking IntentPrompt model gets 16.9% recall vs. 22.5% for 4B Instruct on ambiguous questions. The authors note preliminary experiments found reasoning models do not outperform instruction-tuned models on ambiguity resolution while costing much more.

• Beats a multi-turn clarification system. On Abg-CoQA, IntentRL scores 83.0% embedding-based similarity versus 75.1% for ACT, despite resolving ambiguity in a single turn rather than through clarification dialogue.

• Beats a two-stage parser on every metric. On Ambrosia ambiguous questions, IntentRL outperforms DisambigParse with 82.4% vs. 59.7% recall, 77.5% vs. 26.3% precision, 92.4% vs. 87.8% single coverage, and 74.1% vs. 30.5% full coverage — while learning from answers alone in one stage.

• Out-of-domain generalisation where fine-tuning fails. On AmbiQT (no training on that dataset), SFT achieves only 9.1% full coverage, while IntentRL achieves 66.9% recall, 58.2% precision, and 49.1% full coverage.

• Interpretations are judged meaningful. In a human study over 30 ambiguous examples per dataset (80 interpretation-answer pairs in Abg-CoQA, 92 in Ambrosia) with three annotators per pair, alignment accuracy was 90.0% for Abg-CoQA and 91.7% for Ambrosia, with agreement of 81.0% and 84.0%. Fleiss' kappa was low (29.7 and 27.2) due to the kappa paradox, so Gwet's AC1 is also reported (83.8 and 86.5).

• Efficiency vs. coverage. IntentRL balances reasoning length and coverage better than prompting baselines; thinking models often overthink on Abg-CoQA, getting stuck on the same interpretation.

• Ambiguity is not captured by the model's output distribution. The authors report that neither parallel sampling nor alternative prompting formats recovers the diverse interpretations, indicating gains come from learning to reason about intent rather than from the output format.

Methodology in Plain English

The researchers start from the observation that although interpretations and clarification questions are expensive to annotate, multiple valid answers per input arise naturally — for example, when annotators disagree. So they treat disagreement as the only supervision signal needed.

Given a context (a passage or a database schema) and a question, the model is instructed to first analyse potential ambiguities in a chain of thought, then output up to five interpretation-answer pairs. Interpretations are generated sequentially, so each new interpretation is conditioned on the ones already produced, which pushes them apart and keeps answers aligned with their interpretations.

Training uses DAPO for 5 epochs at a learning rate of 5e-7 with linear scheduling and 3% warmup, sampling 16 completions per prompt and 4 examples per batch. Because ambiguous examples are scarce, the training data is rebalanced by oversampling ambiguous examples to a 3:1 ambiguous-to-unambiguous ratio.

Rewards come from matching predicted answers to gold answers: the Hungarian algorithm finds the maximum weighted bipartite match, with similarities that are binary execution-match for text-to-SQL and word-overlap F1 for question answering. Ambiguous inputs are scored with recall, unambiguous ones with precision.

Baselines isolate each design choice: IntentPrompt (same prompt, no training, across Qwen3 4B and 235B with Instruct and Thinking variants), SFT (Qwen3 4B Instruct, no reasoning instructions, and answers-only on Abg-CoQA because gold interpretations are unavailable), and the state-of-the-art ACT and DisambigParse systems.

Evaluation uses the authors' overlap F1 plus an LLM-as-a-judge (Qwen3 4B Instruct) measuring recall, precision, and full coverage, complemented by execution-based metrics for SQL and a human alignment study.

Why This Matters

Impact on research. The paper reframes ambiguity handling as "reasoning about what to answer rather than how", connects it to pluralistic alignment and specifically to overton pluralism (predicting multiple interpretations within one answer, as opposed to distributional pluralism based on parallel samples), and shows empirically that targeted training — not prompting or sampling — is what produces diverse interpretations. It also shows human-verified interpretation quality above 90% despite no direct supervision on interpretations, and demonstrates transfer to an unseen dataset where supervised fine-tuning collapses.

Real-world applications:

• Database and analytics interfaces. Text-to-SQL systems can show users several possible queries (for example, whether a 20M budget applies to both horror films and thrillers or only to thrillers) so the user picks the intended result instead of getting a silently wrong query.
• Conversational assistants and chatbots. A single structured response resolves ambiguous follow-up questions, such as what a pronoun like "it" refers to, without an extra clarification round.
• Agentic systems. The paper explicitly notes that branching across interpretations can be beneficial in agentic settings, and that the structured format is easier to parse for downstream applications than verbose long-form answers.
• Safety-sensitive deployments. The authors argue that over-generating plausible interpretations is preferable to silently committing to a potentially wrong one, since users can quickly identify and reject an interpretable wrong answer.

Industry relevance. The base model is a relatively small open-weight one (Qwen3 4B Instruct, chosen for a balance of performance and efficiency), and training requires only the answer annotations that many organisations already have through annotator disagreement — not scarce interpretation or clarification labels. The comparisons against a multi-turn clarification pipeline (ACT) and a two-stage semantic parser (DisambigParse) are directly relevant to teams deciding whether to build extra interaction rounds or extra pipeline stages.

Future Directions

1. Using predicted interpretations to detect and correct errors without regenerating the full response.
2. Integrating reasoning about intent into broader post-training alongside domains such as mathematics and code.
3. Applying controlled decoding to improve efficiency and accuracy, for example by ensuring interpretations are mutually exclusive.
4. Adapting to individual user preferences to make interpretations more personalized and relevant.

(Open questions raised by the analysis: the model's lower precision on unambiguous Abg-CoQA questions, generic or vague interpretations, and occasional non-executable SQL queries noted in the error analysis.)

Target Audience

Researchers and practitioners working on LLM alignment, ambiguity resolution, and reinforcement learning post-training; NLP engineers building text-to-SQL, conversational QA, or agentic systems that must handle underspecified user requests; and anyone interested in pluralistic alignment and output diversity. Readers without an RL background can still follow the main argument, while the appendices supply the algorithm and prompt details needed for reproduction.

Authors’ abstract

Large language models often respond to ambiguous requests by implicitly committing to one interpretation, frustrating users and creating safety risks when that interpretation is wrong. We propose generating a single structured response that enumerates the different ways an ambiguous request can be interpreted, each coupled with a corresponding answer. Our models are trained with reinforcement learning using a dual reward objective: recall on ambiguous inputs to maximise coverage of valid interpretations, and precision on unambiguous ones to suppress spurious alternatives. Training requires only multiple valid answers per input as supervision, no clarification questions or explicit interpretations are needed. Experiments on conversational question answering and semantic parsing demonstrate that our method achieves higher coverage of valid answers than baseline approaches. Human evaluation confirms that predicted interpretations are meaningful and explain their corresponding answers. Our approach promotes transparency with explicit interpretations, avoids additional interaction rounds by producing all interpretations in a single generation, and supports downstream applications through its structured output format.