Do Reasoning Models Ask Better Questions? A Formal Information-Theoretic Analysis on Multi-Turn LLM Games

AI summary

Overview

Research area: Multi-turn LLM agents, information-seeking dialogue, and question-asking evaluation using information theory.

Technical level: Intermediate. Familiarity with LLMs, prompting, and basic Shannon entropy helps, but the framework is described in accessible terms.

Scope: The paper introduces a three-agent, multi-turn evaluation framework that measures how much uncertainty each yes/no question removes in a hierarchical "Guess My City" knowledge graph, comparing LLMs with and without Chain-of-Thought (CoT) reasoning under fully observable (FO) and partially observable (PO) conditions.

What This Paper Is About

LLMs often fail to recognize ambiguity in user requests and rarely ask clarifying questions that efficiently narrow down what the user actually means. The authors build a controlled game in which a Seeker LLM must identify a hidden target city by asking yes/no questions, and they score every question by how much it reduces uncertainty, measured as Information Gain (IG) in bits. The goal is to test whether models with explicit reasoning capabilities ask better questions, how they behave when the underlying hypothesis space is hidden, and what their internal reasoning traces reveal about how they choose questions.

Key Contributions

1. A multi-turn evaluation framework for quantitatively assessing LLM information-seeking ability in structured hypothesis spaces, tested under both fully observable and partially observable conditions.

2. A systematic comparison of LLMs with and without Chain-of-Thought reasoning, spanning different model families and sizes (Llama-3.1-8B-Instruct, Qwen3-8B, Qwen3-30B-A3B-Thinking-2507, and the fine-tuned paprika-Meta-Llama-3.1-8B-Instruct).

3. A fine-grained analysis of reasoning traces that characterizes how models of different sizes explore the hypothesis space and select informative questions, using candidate questions extracted from CoT traces.

4. A turn-level and cumulative IG metric grounded in Shannon entropy, providing feedback at both the intermediate (per-turn) and final (cumulative) level, unlike prior word-game benchmarks that rely mainly on win rate.

Main Findings

• Reasoning models gain more information per turn in partially observable settings. In the PO condition, Qwen3-30B-A3B-Thinking-2507 reached a win rate of 0.93 ± 0.03 with 14.88 ± 0.78 average turns and 0.42 ± 0.02 IG per turn, while Qwen3-30B-A3B-Instruct-2507 (no CoT) reached only a 0.10 ± 0.04 win rate with 29.13 ± 0.49 average turns and 0.14 ± 0.01 IG per turn.

• CoT substantially helps the 8B Qwen model in the PO setting. Qwen3-8B with CoT achieved a win rate of 0.94 ± 0.03, 15.82 ± 0.74 average turns, and 0.41 ± 0.03 IG per turn; without CoT it fell to a 0.08 ± 0.03 win rate, 29.60 ± 0.20 average turns, and 0.17 ± 0.00 IG per turn.

• In fully observable conditions, most models solve the game. Llama-3.1-8B-Instruct reached a 1.00 ± 0.00 win rate in 9.33 ± 0.43 turns with 0.64 ± 0.03 IG per turn; paprika-Meta-Llama-3.1-8B-Instruct reached 1.00 ± 0.00 in 9.57 ± 0.52 turns with 0.63 ± 0.03 IG per turn; Qwen3-30B-A3B-Thinking-2507 reached 1.00 ± 0.00 in 7.12 ± 0.15 turns with 0.78 ± 0.02 IG per turn.

• Total IG saturates near 5.32 bits for successful FO runs. Models that solved the game in the FO condition generally report Total IG of 5.32 ± 0.00, reflecting the fixed entropy of the initial hypothesis space.

• The fine-tuned language-game model shows slightly higher PO IG per turn. paprika-Meta-Llama-3.1-8B-Instruct posted 0.40 ± 0.03 IG per turn in PO versus 0.35 ± 0.02 for Llama-3.1-8B-Instruct, suggesting the additional post-training on language games may encourage more inquisitive question-asking, though the authors describe the effect as small.

• Large and small reasoning models compensate differently in PO. Qwen3-8B considered more candidate questions per turn than Qwen3-30B-A3B-Thinking-2507 (9.74 ± 2.31 vs. 7.63 ± 2.15) but selected the optimal candidate less often (Avg Optimal Rate 0.19 ± 0.11 vs. 0.25 ± 0.13). The authors attribute the similar PO–CoT total IG and win rates to the 8B model partially offsetting its smaller capacity with more aggressive exploration.

• In FO, the smaller model picks the best candidate more often, but the larger model proposes stronger candidates. Qwen3-8B had a higher Avg Optimal Rate than Qwen3-30B-A3B-Thinking-2507 (0.36 ± 0.17 vs. 0.26 ± 0.16), while the 30B model had higher Avg Chosen IG and Avg Optimal IG (0.77 ± 0.21 vs. 0.66 ± 0.22, and 1.40 ± 0.41 vs. 1.11 ± 0.28).

• Performance declines as the hypothesis space narrows. The authors report that averaged IG per turn is highest early in the game and drops as fewer candidate cities remain.

Methodology in Plain English

The authors build a game around a hierarchical knowledge graph of geography, with five levels: region, subregion, country, state, and city. Leaf nodes are the possible target cities, and upper levels group them.

Three LLM agents play the game:

• The Seeker asks yes/no questions to identify the hidden target city. It is the model being evaluated.
• The Oracle knows the true target, answers yes or no, and detects when the Seeker has named the city, ending the game.
• The Pruner removes candidate cities that the Oracle's answer logically rules out.

Each turn, the Seeker asks a question, the Oracle answers, and the Pruner updates the candidate set. If any node was pruned, the IG for that turn is computed.

IG is defined as the reduction in Shannon entropy: uncertainty is H = log2(N) bits, where N is the number of active candidate cities, and IG = H_before − H_after. With a uniform prior over candidates, a question that halves the space yields roughly one bit.

The target set is the 40 most populous cities worldwide, drawn from the Countries-States-Cities database merged with 2025 population estimates from World Population Review. Using well-known cities keeps the Oracle (an 8B model) reliable and reduces confounds from obscure targets. Each of the 40 cities is used as a target, with three runs per city, giving 120 game instances per model. Games end when the Seeker identifies the target or after 30 turns.

The Oracle and Pruner are both Qwen3-8B in all experiments. The Seeker role is played by Llama-3.1-8B-Instruct, Qwen3-8B, Qwen3-30B-A3B-Thinking-2507, and paprika-Meta-Llama-3.1-8B-Instruct. For the Qwen variants, reasoning was enabled and disabled to isolate CoT effects. Models ran with their default hyperparameter configurations.

For the reasoning-trace analysis, the authors used Gemini 2.5 Flash to parse the Seeker's reasoning and extract every candidate question the model mentioned before committing to one. They then computed the IG each candidate would have produced, allowing comparison between chosen and unchosen questions.

Why This Matters

Impact on research. Most prior information-seeking benchmarks (LMRL-Gym, the 20-Questions work of Mazzaccara et al. and Bertolazzi et al.) either report only success-based rewards or evaluate a single model without varying CoT. This paper adds a per-turn, information-theoretic signal and a controlled comparison across model families, sizes, and reasoning modes, and it releases code at https://github.com/machadoDaniels/info-gainme.

Real-world applications:

• Clarifying ambiguous user requests: assistants that ask the one question that best narrows down what a user wants, rather than guessing or answering every interpretation.
• Diagnostic systems: medical, technical, and troubleshooting assistants that select the most informative test or question next.
• Search and recommendation: systems that maximize the value of each additional query or clarification prompt.
• Autonomous agents in partially observable environments: agents that must plan under uncertainty without full visibility into the world state.

Industry relevance. The IG metric is a candidate reward signal for reinforcement learning and for evaluating agentic products where turn count and user patience matter. The finding that CoT dramatically improves PO performance (for example, Qwen3-8B going from a 0.08 to a 0.94 win rate) directly informs when reasoning mode is worth its inference cost, and the result that FO settings are nearly saturated suggests graph access in the prompt can substitute for expensive reasoning.

Future Directions

• Test across other domains and language games. The authors explicitly call for evaluating the framework beyond geography, including "20 Questions" and other LMRL Gym tasks, and any domain with enumerable hypothesis spaces.

• Enrich the hypothesis space. Adding more detail to the graph could reduce the framework's reliance on the model's pre-existing world knowledge, which the authors identify as a limitation.

• Use IG as a reinforcement-learning reward. The turn-level and cumulative IG metrics could train agents to select questions that maximize expected information gain over multi-turn interactions.

• Remove the requirement for a complete, pre-structured hypothesis space. The framework currently requires the full hypothesis space to be known and organized as a hierarchical knowledge graph, and its reliability depends on the Oracle and Pruner having accurate knowledge and reasoning, so extending it to open-ended or dynamically discovered hypotheses remains open.

Target Audience

Researchers and practitioners working on LLM agents, dialogue systems, and multi-turn interaction evaluation will benefit most, particularly those interested in information-seeking behavior, ambiguity resolution, Chain-of-Thought reasoning, and information-theoretic evaluation metrics. The paper is also useful for engineers deciding whether to enable reasoning mode or expose structured context in agentic applications, and for anyone designing benchmarks that need intermediate signals rather than a single win-rate score.

Authors’ abstract

Large Language Models (LLMs) excel at many tasks but still struggle with a critical ability for LLM-based agents: asking good questions for resolving ambiguity in user requests. While prior work has explored information-seeking behavior through word games, existing benchmarks lack comprehensive evaluation frameworks that provide both final and intermediate signals based on Information Gain (IG). Moreover, they rarely provide systematic comparisons between models that use chain-of-thought reasoning and those that do not. We propose a multi-turn dialogue framework that quantitatively measures how effectively LLMs gather information through yes/no questions in a hierarchical knowledge graph environment. Our framework employs a triad of interacting LLM agents that ask questions, answer them, and update the hypothesis space. We adopt IG as the main metric, grounded in Shannon entropy, to assess query effectiveness at each turn and cumulatively. We instantiate our framework in a geographical Guess My City game setting organized in a five-level taxonomy and evaluate multiple LLM variants under fully and partially observable conditions, with and without Chain-of-Thought reasoning. Our experiments demonstrate that, among the evaluated models, the ones with explicit reasoning capabilities achieve higher IG per turn and reach solutions in fewer steps, particularly in partially observable settings. Analysis of reasoning traces reveals that smaller models compensate for limited capacity through more aggressive exploration of candidate questions, while larger models exhibit higher assertiveness in selecting optimal queries, generating candidates with greater potential IG.