Words Caught in the Heart: Simulating Person-Specific Expression for Self-Opaque Clients in Psychotherapy

AI summary

Overview

Research area: Human–Computer Interaction, specifically large language model (LLM) based "client simulators" for psychotherapy training and evaluation, with ties to computational linguistics (speech production, stylometry) and clinical psychology.

Technical level: Intermediate. The paper assumes some familiarity with LLM prompting pipelines, ablation studies, and text-similarity metrics, but its central argument is conceptual rather than mathematical.

Scope: The paper proposes Pandora, a modular framework that models how a psychotherapy client's psychological state becomes individual, situated speech, and evaluates it against five baselines and an expression-planning ablation on 115 cases plus a four-evaluator human study of 45 matched disclosure items.

What This Paper Is About

LLM-based psychotherapy client simulators usually generate responses from clinically organized case formulations — the therapist's coherent explanation of the client's difficulties. Real clients, however, often feel distress before they can articulate it, and their wording shifts with emotion, perceived safety, and trust. Pandora addresses this gap by explicitly separating what a client communicates from how that client says it, so simulated clients sound like a specific person with partial understanding rather than a textbook case summary.

Key Contributions

1. Problem framing. The authors identify the mismatch between clinically organized client representations and clients' own limited, situated expression, and reframe client simulation around the transformation of psychological experience into situated speech.

2. The Pandora framework. A multi-stage pipeline that (a) evaluates the therapist's intervention and updates a 12-dimensional psychological state, (b) constructs subjective appraisal and attribution, (c) plans content, stance, and response purpose, and (d) applies a separate expression planner combining an offline-distilled individual language profile with the current interactional conditions.

3. An evaluation suite. Automatic measures of disclosure timing/structure (LDDA, DS, BFE, BSE, GDS) and expression fidelity (content-reduced and raw stylometric similarity), plus a human study of perceived "AI-like" expression — with an ablation isolating the contribution of expression planning.

4. Empirical evidence that realization is a distinct bottleneck. Content planning alone organizes disclosure well (ablation GDS 0.828) but leaves individual expression under-specified; adding expression planning raises AI-like judgments from 35.6% down to 7.2%.

Main Findings

• Best expression similarity among compared systems. Pandora reaches content-reduced/raw similarity of 0.566/0.385, versus 0.512/0.352 for the strongest baseline (Consistent MI) — a margin of 0.053/0.032. The ordering holds on the 97 Chinese-only cases (0.489/0.422 for Pandora, 0.427/0.385 for Consistent MI, 0.362/0.337 for the ablation).

• Best disclosure alignment. Pandora records LDDA 0.852, DS 0.875, and GDS 0.871, compared with GDS 0.664 for Consistent MI and 0.828 for the ablation. Its timing advantage holds at every memory level: 0.843 (L1), 0.854 (L2), 0.866 (L3).

• Psychological context alone does not produce individual voice. MindVoyager's cognitive/relational conditioning yields content-reduced similarity of 0.408, below the simpler role-card baseline Interactive Agents at 0.501.

• The expression planner is doing separable work. Removing it drops content-reduced similarity from 0.566 to 0.458 (a 0.108 fall, below Consistent MI) and raw similarity from 0.385 to 0.311 (0.074 fall); LDDA, DS, and GDS improve by 0.042, 0.079, and 0.043 when it is restored.

• Large reduction in perceived AI-like expression. Across 180 judgments per source (45 items × 4 evaluators), Pandora receives 13 AI-like votes (7.2%), versus 43.3%–67.2% for baselines and 35.6% for the ablation — a 36.1-point reduction relative to Interactive Agents and a 28.3-point paired difference versus the ablation (exploratory case-bootstrap 95% CI: 20.6–36.1 points).

• The effect spans evaluator backgrounds. Pandora's AIR is 10.0% among counselors and 4.4% among synthetic-data researchers, versus 40.0% and 31.1% for the ablation (descriptive only, two evaluators per group). No Pandora candidate receives a strict majority of AI-like votes: 35 of 45 receive none, seven receive one, three receive two.

• Reference point is close but not equivalent. Reference utterances score 4.4% AIR, but the authors caution that reference provenance includes reconstructed or generated material, so proximity to Pandora does not establish equivalence to verified human speech.

• A bundling tradeoff remains. Interactive Agents has the lowest bundling frequency error (0.099 vs Pandora's 0.102), while Pandora has the lowest bundling size error (0.127 vs Consistent MI's 0.128) — brevity or low bundling error alone captures only part of realistic simulation.

• Inter-annotator agreement is moderate. Mean pairwise agreement across all 360 candidates is 62.9%, with positive agreement for AI-like judgments of 53.7% (95% CI: 49.4–58.1%).

Methodology in Plain English

The researchers start from a distinction in speech-production research between deciding what to say and how to phrase it, and build a pipeline with those two halves kept separate.

On the "what" side, a rule-based component reads each therapist turn and tags which techniques were used — clarification, questioning, emotional reflection — including opportunities the therapist missed. That signal updates a numerical client state with twelve dimensions (distress, trust, perceived safety, disclosure readiness, insight, mentalization, and so on), where a client can become clearer about a problem while still being reluctant to discuss it. From the updated state and available case material, the system builds a subjective appraisal (what the experience means to this client) and an attribution (how the client currently explains it, ranging from concrete circumstances to tentative relational guesses). A content plan then picks memories and a stance (hesitant, defensive, confused, overwhelmed, open), and a response-act plan sets the conversational purpose — answer first, be brief, confirm a reflection, or push back.

On the "how" side, the client's own past utterances are distilled offline into a language profile: habitual discourse markers, hedging, emotional explicitness, utterance-length distributions, and sentence counts. An online expression planner combines that profile with the current state and plans to specify speech style, pronoun focus, certainty, hedging, and length/sentence targets. The LLM then writes the utterance from these plans, with internal identifiers and numerical fields kept out of the output.

The cases come from four counseling and motivational-interviewing datasets. From 5,754 standardized records, structural filters (at least 24 counselor–client exchanges, 500 client characters, 0.70 speaker-alternation ratio) left 2,084 dialogues, and semantic review narrowed this to 509 candidates and finally 115 cases (97 PsyDial-D4, 11 AnnoMI, 5 Eeyore Full, 2 ConsistentMI; 97 Chinese, 18 English).

Automatic evaluation uses char 4-gram TF-IDF with up to 4,000 features and cosine similarity against the source dialogue, reported both raw and content-reduced. Disclosure is scored against a shared memory-card codebook (L1–L3) via claim verification. For the human study, 45 disclosure items (15 per level) were each shown as eight randomly ordered candidates to two counselors and two synthetic-data researchers, who gave binary AI-like/non-AI-like judgments. All LLM work used GPT-5.4 mini, at an estimated total API cost of $600.

Why This Matters

For research, the paper makes a case that content fidelity and expression fidelity are separable targets, and that a simulator can get disclosure structure nearly right while still sounding wrong. It supplies a reusable evaluation design — paired stylometric comparison plus AI-like judgment rates plus a realization ablation — for a field that has leaned on clinical plausibility as a proxy for realism.

Real-world applications:

• Therapist training. Trainees practicing on an unrealistically articulate client may find it easier to identify and explore a difficult concern than they would in practice; less polished simulated clients better preserve that difficulty.

• Simulated-client evaluation of therapists and therapy models. If simulated clients over-explain, automated therapist evaluations may reward responses to clarity that real clients cannot yet offer.

• Supervision and deliberate-practice platforms that need clients with stable, recognizable individual voices across a session rather than a single fixed persona prompt.

• Broader user simulation — tutoring, interviewing, and health intake — where the same "the user does not yet understand their own problem" dynamic applies.

Industry relevance is direct for developers of conversational training software and clinical-education tools: the ablation result suggests that investing in a separate linguistic realization stage, rather than only richer psychological profiles, is where perceived realism is won.

Future Directions

• Language and culture coverage. The collection is predominantly Chinese (97 of 115 cases) and includes reconstructed or generated material; the authors call for broader coverage of languages, cultures, and independently collected client speech.

• Finer-grained ablations. The current ablation removes expression planning as a whole; the authors note that finer-grained comparisons could isolate the contribution of individual conditioning signals and the numerical state settings, which are design choices.

• Full conversations with larger panels. The human study uses a small four-person panel judging isolated utterances, and source-reveal usage was not logged, so complete blinding cannot be confirmed. Extending to whole conversations and larger panels would test whether the benefit persists across interaction contexts.

• Downstream validation. Whether reduced AI-like expression actually improves therapist training outcomes or therapist evaluation remains untested, and the authors flag this as the key application question — alongside the fact that their measures do not establish clinical safety or treatment efficacy.

Target Audience

Researchers and graduate students working on LLM-based user simulation and role-play agents; HCI and NLP practitioners building conversational training systems; clinical-education and psychotherapy-training developers who commission or evaluate simulated clients; and computational-linguistics readers interested in style control and stylometric evaluation of generated dialogue. Readers focused on clinical efficacy or on non-LLM dialogue systems will find less directly applicable material.

Authors’ abstract

Psychotherapy clients may experience distress before they can coherently describe it, and their expression changes with emotion, perceived safety, and trust. Large language model-based client simulators often rely on clinically organized case descriptions, which can encourage responses explaining the client's difficulties more clearly than their current understanding warrants. We propose Pandora, a psychotherapy client simulation framework that models how psychological experience becomes situated, individual expression. Pandora updates psychological and relational states in response to therapist interventions, constructs subjective appraisals and attributions, and plans response content and strategy. A separate expression planner combines these decisions with language patterns distilled from the client's utterances to guide linguistic realization. We evaluate Pandora against five baselines on 115 cases from four counseling and motivational interviewing datasets. It achieves the highest expression similarity and overall disclosure-alignment score among the compared systems. In a four-evaluator study of 45 matched disclosure items, Pandora receives AI-like judgments in 7.2% of ratings, compared with 43.3--67.2% for the baselines and 35.6% without expression planning. The ablation also reduces expression similarity. These results support explicit expression planning in client simulation to improve alignment with individual language patterns and reduce perceived AI-like expression in the evaluated setting.