RecToM: A Benchmark for Evaluating Machine Theory of Mind in LLM-based Conversational Recommender Systems

AI summary

Overview

Research area: Evaluation of large language models (LLMs) for Theory of Mind (ToM) reasoning within conversational recommender systems (CRS). Category listed on arXiv: Artificial Intelligence.

Technical level: Intermediate. The paper is readable without deep cognitive-science background, but it assumes familiarity with LLM prompting setups (zero-shot, chain-of-thought), multiple-choice QA evaluation, and standard classification metrics.

Scope: The paper introduces RecToM, a human-annotated question-answering benchmark built on movie-recommendation dialogues, and uses it to measure whether state-of-the-art LLMs can infer users' and recommenders' mental states and predict or judge dialogue strategies.

What This Paper Is About

Existing Theory of Mind benchmarks for LLMs mostly rely on simplified narrative setups inspired by the Sally-Anne test, which stress physical perception and do not resemble real recommendation conversations. They also focus on retrospective mental-state questions and generally omit behavioral prediction, meaning they do not test whether a model can use inferred mental states to choose what to do next. The paper builds RecToM to fill both gaps: a benchmark of realistic, dual-role recommendation dialogues that evaluates ToM along two dimensions, Cognitive Inference (what has been communicated) and Behavioral Prediction (what should be done next).

Key Contributions

1. A new benchmark, RecToM, described by the authors as the first human-annotated conversational recommendation benchmark to introduce ToM evaluation for LLMs in realistic recommendation scenarios, and the first and only benchmark to systematically evaluate ToM reasoning in the context of CRS. It contains 336 dialogues, 4,583 total turns, 10 question types, and 20,524 QA pairs.

2. A two-part ToM evaluation framework covering Cognitive Inference (desire, intention, and belief reasoning, following the Belief-Desire-Intention model) and Behavioral Prediction (prediction reasoning and judgment reasoning about dialogue strategy effectiveness).

3. Four distinctive design features for conversational recommendation: multi-choice strategy (one utterance can carry several intentions), multi-granular intention (coarse-grained plus fine-grained categories), multi-dimensional belief (e.g., who proposed a movie, whether the seeker has seen it, and whether they liked or accepted it), and multi-concurrent desire (a seeker's simultaneous inclinations toward each recommended item).

4. An empirical study of five state-of-the-art LLMs under zero-shot and chain-of-thought prompting, producing five headline findings about where ToM reasoning in CRS succeeds and where it breaks down.

Main Findings

• Multi-choice formats sharply reduce accuracy: On multiple-choice tasks, models averaged only 27.74% on fine-grained intention reasoning for the recommender role (random guess: 0.10%). Single-choice tasks were much higher, with model averages of 68.72% on belief reasoning and 86.35% on desire reasoning (random guess: 14.29% and 50.00% respectively).

• A granularity gap in intention inference: Models handled coarse-grained intention better than fine-grained intention. GPT-4o reached 64.22% on seeker coarse-grained intention but only 28.84% on seeker fine-grained intention. The paper's error analysis attributes this to difficulty discriminating among fine-grained options within the same coarse category, not to misidentifying the broad category.

• Fine2Coarse mapping supports that interpretation: Human-annotated mapping of fine-grained outputs to coarse categories gave higher scores than direct fine-grained accuracy (DeepSeek-v3: 29.71% vs. 45.40% for the recommender; GPT-4o: 28.84% vs. 50.25% for the seeker), and approached coarse-grained accuracy (DeepSeek-v3+CoT: 51.16% vs. 46.21% for the recommender).

• Limited but real multi-dimensional belief inference: The best configuration, DeepSeek-v3 with CoT, reached 79.46% on belief reasoning, above the random baseline of 14.29% and above GPT-4o-mini's 52.50%. Human performance on belief reasoning was 96.84%.

• Chain-of-thought prompting gives little and inconsistent benefit: CoT produced a +1.95 percentage point gain for DeepSeek-v3 in coarse-grained intention inference for the recommender and +0.46 percentage points for GPT-4o in belief reasoning, with degradation elsewhere (GPT-4o on seeker coarse-grained intention: 64.22% to 54.10%).

• A systematic affirmative or "sycophantic" bias in judgment reasoning: On the binary judgment task, models fell below random guess (50.00%). They predicted "Yes" about 93.37% of the time, had a false positive rate of about 93.28%, and recall for "No" of only about 7.22%. The paper links this to the "Answer Sycophancy" phenomenon.

• A wide human-model gap overall: Human accuracy ranged from 59.92% (seeker fine-grained intention) to 98.25% (desire), while the model averages ranged from 20.54% (recommender prediction) to 86.35% (desire), and no model matched human performance on any task.

Methodology in Plain English

The researchers started from ReDial, a public corpus of movie recommendation dialogues between a seeker and a recommender. Following the selection protocol from IARD, they kept 253 satisfactory dialogues, where the seeker first rejects a recommended movie but later accepts another suggestion, and 83 unsatisfactory dialogues, where no recommendation is accepted, for 336 dialogues in total.

They then refined the data manually in two steps: locating the utterance where the seeker states an opinion on a recommended movie (to establish belief about final acceptance), and re-annotating three dimensions for each movie mentioned, namely suggestion (recommender-proposed or seeker-initiated), seen (whether the seeker has watched it), and liked (whether the seeker liked the movie or the recommendation). Three PhD students trained in ToM and prior psychology projects did the annotation, with two labeling initially and a third resolving conflicts; the inter-annotator agreement was a Fleiss's Kappa of 0.79.

From these dialogues they generated 20,524 question-answer pairs across 10 question types, cast as multiple-choice or single-choice items. Cognitive Inference items ask about desire ("Is the seeker likely to watch the movie?"), intention ("What is the intention expressed by the recommender/seeker in the utterance, given the dialogue history?"), and belief ("How does the recommender believe the seeker's attitude about the movie?"). Behavioral Prediction items ask which strategy a participant will use next, or whether a randomly specified strategy will be effective.

They then evaluated five LLMs (DeepSeek-v3, GPT-4o, GPT-4o-mini, Gemini 2.5 Flash-Lite, and Claude 3.5 Haiku) using two prompting strategies: vanilla zero-shot, and chain-of-thought with the instruction "Let's think step by step." Temperature was set to 0.7, and answers were extracted by string matching against a fixed output format. For the judgment task, they additionally computed a confusion-matrix analysis using Prediction Bias, False Positive Rate, and Recall for "No."

Why This Matters

Impact on research. The paper argues that previous ToM benchmarks over-rely on simplified Sally-Anne-style narratives and on retrospective questions, leaving behavioral prediction untested. RecToM moves evaluation into realistic, asymmetric two-party recommendation dialogues and separates cognitive inference from strategic action, giving researchers a way to diagnose whether a model's failures come from poor mental-state inference or from poor use of inferred states. The reported difficulty — models well below human accuracy and below random guess on judgment — sets a clear target for future work.

Real-world applications:

• Conversational shopping and product-recommendation assistants that must infer a user's evolving preferences from indirect cues.
• Streaming and entertainment chat interfaces, the domain used here, where a recommender must track which titles a user has seen, likes, or has already rejected.
• Customer-support and service dialogue agents that need to decide whether to ask a clarifying question, explain an option, or make a direct suggestion.
• Any multi-turn assistant where an overly agreeable response style could mislead users, given the paper's finding of strong affirmative bias in strategy judgment.

Industry relevance. The failures reported are directly relevant to deployed CRS features: the granularity gap limits fine-grained personalization, the multiple-choice weakness limits reasoning when several plausible user intentions coexist, and the sycophancy bias is especially costly when a system must judge whether a proposed action is actually useful rather than merely agreeable.

Future Directions

• Improving fine-grained intention discrimination, since models often identify the correct coarse category but not the specific fine-grained intent within it.
• Developing prompting or training methods that work where chain-of-thought does not, given that CoT yielded only marginal and sometimes negative changes in this context-rich setting.
• Mitigating affirmative and sycophantic bias in strategy judgment, where the paper reports near-total "Yes" prediction rates (about 93.37%) and recall for "No" of about 7.22%.
• Extending the benchmark beyond movie recommendation, since the current data is derived entirely from ReDial and the IARD selection protocol, leaving open whether these ToM failures generalize to other recommendation domains.

Target Audience

Researchers and practitioners working on LLM evaluation, conversational recommender systems, and machine Theory of Mind will benefit most. The paper is also useful for product and engineering teams building dialogue-based recommendation or assistant features, and for cognitive-science-informed AI researchers interested in how BDI-style mental-state modeling translates into practical benchmarks. Readers need basic familiarity with LLM prompting and benchmark evaluation to follow the results tables.

Authors’ abstract

Large Language models are revolutionizing the conversational recommender systems through their impressive capabilities in instruction comprehension, reasoning, and human interaction. A core factor underlying effective recommendation dialogue is the ability to infer and reason about users' mental states (such as desire, intention, and belief), a cognitive capacity commonly referred to as Theory of Mind. Despite growing interest in evaluating ToM in LLMs, current benchmarks predominantly rely on synthetic narratives inspired by Sally-Anne test, which emphasize physical perception and fail to capture the complexity of mental state inference in realistic conversational settings. Moreover, existing benchmarks often overlook a critical component of human ToM: behavioral prediction, the ability to use inferred mental states to guide strategic decision-making and select appropriate conversational actions for future interactions. To better align LLM-based ToM evaluation with human-like social reasoning, we propose RecToM, a novel benchmark for evaluating ToM abilities in recommendation dialogues. RecToM focuses on two complementary dimensions: Cognitive Inference and Behavioral Prediction. The former focus on understanding what has been communicated by inferring the underlying mental states. The latter emphasizes what should be done next, evaluating whether LLMs can leverage these inferred mental states to predict, select, and assess appropriate dialogue strategies. Extensive experiments on state-of-the-art LLMs demonstrate that RecToM poses a significant challenge. While the models exhibit partial competence in recognizing mental states, they struggle to maintain coherent, strategic ToM reasoning throughout dynamic recommendation dialogues, particularly in tracking evolving intentions and aligning conversational strategies with inferred mental states.