SPASM: Stable Persona-driven Agent Simulation for Multi-turn Dialogue Generation

AI summary

Overview

Research area: Natural Language Processing — multi-agent LLM dialogue simulation and synthetic data generation.

Technical level: Intermediate. The core idea is conceptually simple and can be understood without deep math, but the paper assumes familiarity with LLM prompting, multi-turn dialogue, embeddings, and basic clustering metrics.

Scope: This paper introduces SPASM, a stability-first framework for generating long, controllable, persona-driven multi-turn dialogues between two LLM agents, plus Egocentric Context Projection (ECP), a lightweight history-representation change that sharply reduces persona drift and eliminates role-mirroring ("echoing").

What This Paper Is About

Large language models are increasingly used to generate synthetic multi-turn dialogues for training and evaluation, often by letting two LLM agents talk to each other (a "Client" enacting a persona and a "Responder" playing a role). Over long conversations, these agents drift from their assigned identities, confuse roles, or begin mirroring each other's language — failures that contaminate the resulting data. The goal of this paper is to build a modular simulation framework that keeps each agent's persona, role, and goals stable across long horizons, and to show empirically that a small change in how dialogue history is represented solves most of the problem.

Key Contributions

1. SPASM framework. A modular, stability-first pipeline for persona-driven dialogue generation, composed of persona creation (schema sampling, plausibility validation, natural-language crafting), Client–Responder dialogue simulation, and termination detection for coherent stopping.

2. Egocentric Context Projection (ECP). A novel history-construction mechanism that stores dialogue turns in a perspective-agnostic form (who said what) and deterministically projects them into each agent's relative view (SELF vs. PARTNER) before generation. Crucially, ECP requires no model weight changes.

3. A large-scale dataset and behavioral analysis. 4,500 personas and 45,000 conversations across nine Client–Responder backbone combinations (GPT-4o-mini, DeepSeek-V3.2, Qwen-Plus), with quantitative analysis of embedding geometry, persona retrieval, drift, and echoing.

4. Empirical evidence that ECP works. Across all tested backbones, ECP reduces persona drift and, under full-coverage human validation, eliminates echoing entirely — while the standard history-concatenation baseline shows echoing rates up to ~41% under some pairings.

Main Findings

• ECP reduces persona drift consistently. Compared with naive history concatenation, ECP lowers drift on concerns, emotion, and motivation probes across all three backbones. The largest effect is on emotion under GPT-4o-mini (Cohen's d = −0.75). Motivation improvements are more backbone-dependent (no reliable change for DeepSeek).

• ECP eliminates echoing. Under human validation, zero echoing cases were observed in ECP conversations across all nine pairings. In the concatenation baseline, judge-flagged echoing rates reached up to 41% (DeepSeek/DeepSeek), with human-confirmed rates up to 32%.

• Same-backbone pairings produce tighter persona clusters. When Client and Responder share a backbone, silhouette scores are higher (0.39–0.68 vs. 0.10 for GPT–DeepSeek) and Davies–Bouldin indices lower, with much smaller within-persona distances (~0.09 vs. 0.34).

• The Responder model dominates interaction geometry. Fixing GPT-4o-mini as the Responder keeps clustering quality high (silhouette > 0.60) regardless of which model is the Client, whereas DeepSeek-V3.2 as the Responder degrades clustering sharply. This suggests the Responder shapes the emergent embedding space, while the Client mainly modulates variance.

• Cross-model misalignment inflates variance, not collapse. PCA variance explained stays stable (68–77%) across all settings; cross-backbone degradation manifests as higher within-cluster dispersion while between-cluster distances remain near 0.5.

• Persona identity is recoverable from embeddings. Nearest-neighbor retrieval (Acc@K) is far above the random baseline for all pairings. Top-1 accuracy ranges from 0.50 (GPT/DeepSeek) to 0.99 (DeepSeek/GPT), showing a real but pairing-sensitive persona signal.

Methodology in Plain English

The researchers built a pipeline with five moving parts:

1. Persona schema sampling. Randomly draw one value for each of several attribute categories: demographics (age, occupation, location), interaction context, emotional state (emotion and intensity), and behavioral traits (expressiveness, self-disclosure, politeness, assertiveness).

2. Persona validation. Because random combinations can be absurd (e.g., an 18-year-old student planning retirement pensions), an LLM validator checks plausibility. Invalid combinations are resampled until a coherent profile is found.

3. Persona crafting. An LLM converts the validated field set into a fluent natural-language persona description, optionally enriched beyond the raw fields based on instructions.

4. Client–Responder simulation. The Client enacts the persona; the Responder follows a user-supplied role prompt (listener, expert, advisor, etc.). Turns alternate.

5. Termination detection. After each turn, an LLM checks the most recent window of dialogue for natural closure cues (e.g., gratitude, goodbyes) and stops the conversation if found. A hard cap of 25 turns per agent prevents runaway dialogues.

The key technical trick is ECP. Instead of pasting the raw transcript back into each agent's prompt, the system stores history as pairs of (absolute speaker identity, utterance content). Before each generation call, it relabels each historical turn relative to the current speaker: turns by this agent become "SELF," turns by the other become "PARTNER." This keeps the same content and order while removing role ambiguity.

To test ECP, the team ran an ablation: the same setup with either ECP or the usual concatenated transcript. They measured drift by asking the agent persona-probing questions before the conversation and again at various turns, then computing embedding cosine distance between the answers. They measured echoing by having an LLM judge scan completed conversations for any message that sounds more like the partner's role than the speaker's own, then had two trained human annotators validate those judgments through a custom GUI. They also analyzed all 45,000 conversations with PCA, UMAP, silhouette scores, Davies–Bouldin indices, and ANOVA on within- vs. between-persona distances.

Why This Matters

The paper targets a practical bottleneck in synthetic data generation: fluent-looking LLM–LLM dialogues often quietly violate the specification, so downstream models trained on this data inherit the drift. SPASM shows that a minimal change in how history is represented — not in model weights or scale — can substantially improve role fidelity.

Real-world applications:

• Tutoring and educational dialogue. Stable personas let systems simulate diverse student profiles reliably over long sessions, improving training data for adaptive tutors.
• Customer support and service simulation. Generating controllable, persona-consistent conversation corpora for training support agents without collecting real customer data.
• Mental health and emotional support research. Simulating clients with specific emotional states and intensity levels for evaluation and training, where preserving the persona over many turns is critical.
• Safety, bias, and robustness auditing. Producing realistic, long-horizon dialogues to stress-test conversational models for role confusion, drift, and unsafe behaviors.

Industry relevance: The framework is directly useful for teams building LLM post-training pipelines, evaluation benchmarks, and persona-based conversational agents. Because ECP is a prompt-level, weight-free change, it can be adopted by any organization already generating synthetic dialogues at scale with minimal engineering effort.

Future Directions

• Broadening the model and language scope. Extending evaluation to non-English languages, smaller open-weight models, and architectures beyond the three instruction-tuned backbones tested here.

• Moving beyond two-agent, fixed-role setups. Extending ECP to multi-agent group interactions and dynamically changing roles, where the projection operator must handle more than two relative roles.

• Richer persona representations. Current personas come from structured schemas and natural-language descriptions that may not capture real human variability or nuance. Integrating richer behavioral or longitudinal data is an open direction.

• Scaling and mechanistically explaining echoing. Human validation of echoing is subjective and does not scale easily. The paper sketches three mechanistic hypotheses (role-label ambiguity, post-training alignment priors, closed-loop feedback amplification) that would benefit from deeper causal investigation.

Target Audience

NLP and dialogue-systems researchers, especially those working on multi-agent LLM simulation, synthetic data construction, and instruction-following stability. It is also valuable for alignment and evaluation researchers who need controllable, long-horizon conversational data, and for industry practitioners building conversational training pipelines or persona-driven agents who want a low-cost way to prevent role drift and echoing in their generated corpora.

Authors’ abstract

Large language models are increasingly deployed in multi-turn settings such as tutoring, support, and counseling, where reliability depends on preserving consistent roles, personas, and goals across long horizons. This requirement becomes critical when LLMs are used to generate synthetic dialogues for training and evaluation, since LLM--LLM conversations can accumulate identity-related failures such as persona drift, role confusion, and "echoing", where one agent gradually mirrors its partner. We introduce SPASM (Stable Persona-driven Agent Simulation for Multi-turn dialogue generation), a modular, stability-first framework that decomposes simulation into (i) persona creation via schema sampling, plausibility validation, and natural-language persona crafting, (ii) Client--Responder dialogue generation, and (iii) termination detection for coherent stopping. To improve long-horizon stability without changing model weights, we propose Egocentric Context Projection (ECP): dialogue history is stored in a perspective-agnostic representation and deterministically projected into each agent's egocentric view before generation. Across three LLM backbones (GPT-4o-mini, DeepSeek-V3.2, Qwen-Plus) and nine Client--Responder pairings, we construct a dataset of 4,500 personas and 45,000 conversations (500 personas X 10 conversations per pairing). Ablations show ECP substantially reduces persona drift and, under human validation, eliminates echoing; embedding analyses recover persona structure and reveal strong responder-driven interaction geometry. Our code is available at https://github.com/lhannnn/SPASM.