You Changed Your Mind, The Model Didn't: Demystifying Intent in Multi-Turn Dialogue

AI summary

Overview

Research area: Multi-turn dialogue and intent tracking in large language models, spanning benchmark design (tool actions, code, databases, mathematics) and a training method based on on-policy self-distillation.

Technical level: Advanced. The paper combines a controlled multi-turn benchmark with a distillation objective built on Jensen–Shannon divergence between Teacher and Student next-token distributions.

Scope: The paper shows that merely mentioning a change the user later rejects (or supersedes) degrades LLM task execution, and proposes a decision-conditioned distillation method that supervises the model from the single-turn task that remains in effect.

What This Paper Is About

When a user proposes a change during a multi-turn task and then rejects it, the model should keep executing the original task as if nothing changed. The authors find it often does not: the rejected proposal lingers and derails the answer. They build a controlled benchmark, Intent-Eval, to measure how models handle clarification, rejection, and revision across Actions, Code, Database, and Math, and then propose Intent-OPSD, a training method that teaches a model to follow only the requirements that remain active.

Key Contributions

1. Intent-Eval, a four-domain benchmark with four matched multi-turn conditions (Original, Neutral, Retained, Revised), single-turn controls that separate multi-turn effects from task difficulty, and extended dialogues that test whether interference accumulates or persists. It comprises 414 source tasks and yields 3,312 evaluation instances per model.

2. Empirical findings on accuracy degradation across conditions, showing that degradation can deepen or persist as interaction continues, with error patterns consistent with mentioned-as-in-effect confusion under Retained and Revised.

3. Intent-OPSD, an on-policy self-distillation framework that selects the Teacher's single-turn task based on the user's decision and trains the Student on the full dialogue, improving mean accuracy over the corresponding base models by 10.81 pp across four models and four domains.

4. A decision-conditioned formulation in which the frozen Teacher receives the complete single-turn task matching the user's decision (the revised task when the change is accepted, the original task otherwise) and the Student requires neither the Teacher nor the single-turn prompt at inference.

Main Findings

• Dispersed disclosure costs 36.30 pp. Splitting a task across turns lowers mean accuracy by 36.30 pp relative to the single-turn control, even with no interruptions (the Original condition).

• Clarification alone costs an extra 4.43 pp. Adding Neutral clarification, which changes neither the active task nor the reference answer, incurs an additional 4.43 pp drop. This pattern appears across all eight models in the main table.

• Decision-induced confusion. In multi-turn dialogue, Retained and Revised average 8.06 and 5.55 pp below Original, respectively. Relative to Neutral, the additional drops are 1.12 pp (Revised) and 3.63 pp (Retained).

• The effect appears even in a single turn. In single-turn controls, Revised and Retained reduce accuracy by 5.74 and 7.52 pp compared with stating the same active task directly — so the loss is not simply because the revised task is harder.

• Rejected content shows up in wrong answers. Inactive content appears in 59.63% of new Retained errors and 30.97% of new Revised errors across ten models. In Math, these rates reach 67.70% under Retained and 47.94% under Revised.

• Interference accumulates. In the accumulation protocol, the first inserted block produces the largest average single-step decline, and accuracy generally continues to decline as blocks are added. At the final depth, accuracy falls by 8.13 pp for Neutral and 20.77 pp for Retained relative to Original. This analysis uses the 372 of 414 sources that have four distinct proposals.

• Interference persists. In the persistence protocol over all 414 sources, adding four Neutral clarification blocks after a decision lowers accuracy by 2.72 pp under Retained and 3.50 pp under Revised relative to each condition's baseline.

• Frontier models are robust in Code specifically. GPT-5.6 and DeepSeek-V4 retain at least 85% accuracy across all multi-turn Code conditions, contrasting with larger degradation in Database, Actions, and Math. The advantage is described as highly domain-specific.

• Intent-OPSD improves accuracy. It improves mean accuracy over Base and SFT by 10.81 and 3.73 pp respectively. It outperforms Base in all 16 model–domain pairs and SFT in 13. Overall averages are 40.31 (Base), 47.39 (SFT), and 51.12 (Intent-OPSD).

• Gains hold across interaction depth. Depth gains versus Base are +9.61 (k=1), +11.74 (k=2), +11.67 (k=3), and +10.64 (k=4); versus SFT they are +2.91, +3.86, +2.49, and +2.16.

• The gain is largest where intent tracking is the bottleneck. Intent-OPSD improves Actions accuracy by 21.46 pp over Base. Code shows smaller gains, which the authors attribute to demands on program structure and logic that persist even when the active requirement is tracked correctly.

Methodology in Plain English

Building the benchmark. The authors take 414 source tasks from the LiC benchmark, drawn from BFCL (Actions, 104 revisions), HumanEval and LiveCodeBench (Code, 100 revisions), Spider (Database, 107 events — 88 SQL-condition changes and 19 output-field changes), and GSM8K (Math, 103 tasks). Each task is split into information shards revealed one user message at a time. One requirement per task is selected for revision, and candidates must be unambiguous, atomic, and verifiable. After human review, approved revisions are rendered as dialogue messages.

Four matched conditions. Original continues disclosure with no event. Neutral inserts two clarification questions about the same requirement. Retained and Revised share the identical proposal, followed by rejection or acceptance respectively. Only Revised changes the target answer — Original, Neutral, and Retained are scored against the original reference answer; Revised is scored against the revised one. Single-turn controls state the same tasks in one message, with S_Orig, S_Dir (the revision stated directly), and decision-bearing versions S_Rev and S_Ret that append the proposal plus acceptance or rejection.

Testing depth. Two protocols vary interaction depth using an added-block count k from 0 to 4: accumulation inserts k clarification blocks or k proposal-and-rejection blocks, and persistence adds k Neutral blocks after a decision.

The training method. Intent-OPSD uses a frozen Teacher and a Student initialized from the same model. The Student receives the full dialogue history; the Teacher receives the complete single-turn prompt corresponding to the user's decision (S_Dir for Revised, S_Orig otherwise). The Student samples its final answer on-policy, and at each answer position both policies score the same Student-generated prefix; the loss is the Jensen–Shannon divergence between their next-token distributions over the full vocabulary, averaged equally over the four conditions. Intermediate replies receive no loss. Sources are admitted only if the Teacher solves both the original and revised single-turn prompts and passes the relevant verifiers; reference answers and verifier results are used only for source selection and never given to either model. Only the Student is updated, and at inference the Student answers from the dialogue alone. For RQ4, the four models trained are Qwen3-8B, Gemma-4-26B-A4B, Qwen2.5-7B-Instruct, and Llama-3.1-8B, with 500 candidate training sources and 50 validation sources per domain, non-overlapping with evaluation sources.

Evaluation setup. Ten LLMs from six providers are evaluated: Qwen3.6-27B, Qwen3-8B, Qwen3-4B-Instruct-2507, Qwen2.5-7B-Instruct, Llama-3.1-8B, GPT-5.6-Luna, DeepSeek-V4-Flash-0731, Claude-Sonnet-5, Gemini-3.7-Flash, and Gemma-4-26B-A4B. RQ1 reports eight models in the main table with earlier Qwen models in the appendix. Multi-turn runs use a fixed Qwen3.6-27B user simulator. RQ2 pairs outputs by source and model across ten models.

Why This Matters

The paper reframes multi-turn failure: it is not only about spreading information across turns or about task difficulty, but about models failing to distinguish what has been said from what remains in effect. That distinction is central to any agent that must respect a user's final decision rather than the conversational path that led there.

Real-world applications:

• Tool-calling agents: a user who explores one argument value and then rejects it should not have that value silently applied to the final call.
• Coding assistants: a rejected function rename should not appear in generated code, and the paper notes Code is where frontier models already show the strongest robustness.
• Database query agents: a rejected change to an age cutoff or output field should not alter the SQL executed against production data.
• Mathematical and analytical reasoning: reused intermediate steps computed from a superseded numerical premise propagate errors, which the paper links to the highest inactive-content rates under Math.

Industry relevance. The paper reports that Intent-OPSD raises mean accuracy over base models by 10.81 pp, requires no Teacher at inference, and is trained with 500 candidate sources and 50 validation sources per domain — a relatively modest data requirement for a method aimed at a failure mode that appears across all evaluated providers and domains.

Future Directions

• Extending the study to longer and more open-ended interactions with richer forms of intent evolution, as the authors explicitly propose.
• Closing the domain gap: the strongest robustness appears only in Code, while Database, Actions, and Math continue to degrade, leaving open why frontier progress in agentic coding does not transfer.
• Understanding why clarification without any change in requirements still costs accuracy, and what property of a dialogue makes interference persist after a clear accept-or-reject decision.
• Addressing the residual Code gap, where structural or implementation errors can still cause failures even when the active requirement is tracked correctly.

Target Audience

Researchers and practitioners working on multi-turn dialogue, agentic tool use, and LLM alignment to user intent; benchmark designers interested in controlled evaluation of evolving requirements; and engineers building assistants that must honor a user's final decision over intermediate proposals. It is most useful to readers with some familiarity with language model training and distillation, given the technical level of the Intent-OPSD section.

Authors’ abstract

When a large language model handles a multi-turn task and a user proposes a change but ultimately rejects it, the model should continue as if nothing changed. We find a surprising failure: merely mentioning a rejected change can derail task execution, even when the user's final intent remains unchanged. To systematically study language model behavior under evolving user intent, we introduce Intent-Eval, a controlled benchmark spanning tool actions, code, databases, and mathematics. Across diverse tasks, models are vulnerable to both rejected proposals and superseded requirements, consistent with mentioned-as-in-effect confusion: conversational content is treated as active requirements even after it has been rejected or replaced. Accuracy degradation can deepen or persist as interaction continues, highlighting the need to distinguish what has been mentioned from what remains in effect. Building on this insight, we propose Intent-OPSD, a decision-conditioned on-policy self-distillation framework with Teacher and Student initialized from the same model. The frozen Teacher provides active-intent supervision from the complete task matching the user's decision, training the Student on the full dialogue to follow active requirements reflecting user intent.