Agent Sociodrama: Social-Dynamics-Guided Persona Recasting for LLM Agent Teams

AI summary

Overview

• Research area: Multi-agent LLM systems, social simulation, and automated team/persona composition (artificial intelligence; multi-agent collaboration).
• Technical level: Intermediate — assumes familiarity with LLM agents and prompting, but the social-state formalism is explained from first principles.
• Scope (one sentence): The paper builds a role-play framework in which LLM agents carry fixed personas but evolving emotions and directed relationships, validates those dynamics against classical social theory and human raters, and then uses the resulting "social trajectories" to re-cast team personas for better task performance.

What This Paper Is About

LLM agents are increasingly deployed as teams, but they fail in distinctly social ways: the agent holding critical information stays silent, a dominant speaker anchors the group on a wrong answer, and norm violations go unsanctioned. Existing systems encode social context through static role instructions or unstructured memory, with no explicit account of how relationships evolve over time. The paper's goal is to model a team as a dynamic social system — stable personas plus evolving emotions and directed, asymmetric relationships — verify that those dynamics behave like real social behavior, and then turn them into an actionable signal for repairing team composition.

Key Contributions

1. A dynamic multi-agent role-playing framework (Agent Sociodrama). Each agent has a time-invariant persona (identity, traits, boundaries), a scalar emotional state, and directed relationships toward every other agent (favorability, categorical status, and an event memory), all updated every round by LLM calls conditioned on persona and visible events.
2. Cross-model validation of social fidelity. Across six LLM backbones (GPT-4o, GPT-5-mini, Claude Sonnet 4.6, Gemini 3.1 Flash-Lite, Grok 4.20, DeepSeek V4-Flash) and five social scenarios, the trajectories are checked against four classical social-theory regularities (structural balance, reciprocity, conflict-driven polarization, repair-versus-escalation) and against judgments from thirteen human raters.
3. Trajectory-Guided Persona Recasting (TPR). A casting controller that runs a task, diagnoses group-dynamic failures from the transcript plus the emotional and relational trajectories, and recasts selected agents' personas before re-running — treating "team chemistry" as a searchable design variable.
4. Empirical gains on four collaborative benchmarks. TPR improves performance from both a heterogeneous (hand-assigned) and a random initial casting, and outperforms SOTOPIA and AgentVerse in three of four settings, by up to 37.5 percentage points in Pareto rate.

Main Findings

• Coherent response to social pressure: Averaged over six backbones, as scenarios move from a hospital ethics review to a demolition standoff, mean terminal favorability falls from 0.75 to 0.23, the E-I index shifts from −0.10 to −2.25, structural balance declines from 100.0% to 71.7%, and the repair rate falls from 58.5% to 18.9%.
• Reciprocity does not simply track conflict: Final reciprocity is 0.35 in Hospital, 0.74 in Tavern, 0.78 in Quest, 0.91 in Negotiation, and 0.68 in Demolition — conflictual settings can show stronger bilateral alignment even as favorability declines.
• Model-specific social styles: Final favorability differs significantly across backbones in every scenario (Kruskal–Wallis, all p < 0.01). In the Tavern scenario, GPT-5-mini stays broadly positive and weakly polarized (Fav^T = 0.75, E-I = −0.22), Claude Sonnet 4.6 shows high reciprocity (0.93) and strong repair (80.0%), while GPT-4o shows only 10.0% repair despite positive affect.
• Moderate but meaningful human agreement: Inter-human agreement is itself low (Fleiss' κ = 0.288 for relationships, 0.184 for emotions). On items where at least seven of thirteen raters agreed on a direction, the GPT-4o judge matched the human majority on 76.2% of relationship updates and 88.2% of emotional updates.
• Recasting produces large, uneven gains: Model-averaged Student Council top-1 accuracy rises from 16.7% to 54.2% (Hetero) and from 27.8% to 58.3% (Random); Negotiation Pareto rate rises from 38.9% to 48.6% (Hetero) and 40.3% to 44.4% (Random). The largest single jump is GPT-5-mini on Student Council, from 16.7% to 100.0%.
• Error reductions on ranking tasks: Model-average L1 error drops on NASA Moon Survival from 13.94 to 12.03 (Hetero) and 15.11 to 12.92 (Random), and on Lost at Sea from 23.78 to 17.22 (Hetero) and 23.64 to 18.19 (Random). Gemini 3.1 FL reduces its NASA error from 17.17 to 12.00 under Hetero.
• The social state itself carries the signal: Removing the emotional and relational updates while keeping the rest of the pipeline hurts final model-average results, most visibly on Negotiation, where the Hetero Pareto rate falls to 27.8% versus 48.6% with full TPR.
• Beats existing frameworks: Against SOTOPIA and AgentVerse, the framework has the best final mean in three of four settings (Pareto rates of 48.6% and 44.4% versus 18.1%/6.9% and 22.2%/40.3%); AgentVerse narrowly leads only on Random NASA (12.89 vs. 12.92).
• Recasting is interpretable, not a single template: In a Lost at Sea case study with DeepSeek V4-Flash (Random init), the diagnosis identifies that Blake's argument about the fuel can was undervalued; successive recasts lower mean L1 from 21.83 to 18.00, then 14.33, and finally 11.33, ending with four distinct personas.

Methodology in Plain English

The researchers build a simulated "stage play" for a team of agents. Before the episode, each agent receives a fixed persona card describing who it is, how it tends to behave, and what it will not do. During the episode — nine rounds in the fidelity study, four rounds in the task study — scenario events arrive, agents speak, and then two things update: each agent's emotional state (a number from 0 to 1, with 0.5 neutral, adjusted by at most ±0.3 per round) and, for each ordered pair of agents, a directed relationship record. That record holds a favorability score from −1 to 1, a status label (neutral, ally, rival, or enemy, set by favorability thresholds with override for consequential events like betrayal or alliance), and a short memory of salient events, capped at K entries with low-retention items evicted first. One unified external LLM judge performs these updates from what each agent could actually see.

Speaking is decoupled into two phases so that turn order does not bias who gets heard: first every agent independently reports an urgency score from 0 to 10, a mode (public, private, or silence), a target, and a one-sentence intent; then agents speak in descending urgency order, with silent agents skipped at no inference cost. Private messages and public messages produce different observer sets, so information genuinely stays distributed.

For validation, the authors check whether these trajectories reproduce four well-known social regularities, and they have thirteen human raters label whether each update increased, stayed flat, or decreased, comparing those labels to the judge's.

For the task experiments, the controller starts from an initial casting drawn from a five-archetype library (balanced, dominant, analytical, agreeable, reserved) across four agents (Avery, Blake, Casey, Drew). It runs the task twelve times, asks an external LLM to diagnose whose information was suppressed or who dominated unhelpfully using the transcript plus the S and R trajectories, and takes a majority vote over the twelve runs to decide who gets re-cast. The loop stops when the task target is hit, no change is proposed, or after three recasts. Task metrics are computed by rule — Borda count aggregation for the ranking tasks and exact Pareto-frontier enumeration over 3^5 = 243 possible agreements for the negotiation task — rather than by an LLM judge.

Why This Matters

The paper reframes a practical engineering problem — "why did my agent team fail?" — as a measurement problem. Instead of optimizing prompts, topologies, or workflows and treating the team as a black box, it gives a diagnostic instrument that points at a specific agent or dyad and says what went wrong. That makes team composition a controllable design variable rather than an accident of initial role assignment.

Real-world applications:

• Enterprise decision support: teams of agents preparing recommendations where one department's evidence must surface rather than being buried by a louder voice.
• Multi-party negotiation and procurement: mixed-motive bargaining where the goal is a Pareto-efficient settlement rather than a win for one side (the paper's Solar-farm Negotiation task is exactly this shape).
• Clinical or safety review boards: consensus settings with divergent risk preferences, where silent dissent is the failure mode that matters most.
• Automated research and analysis pipelines: distributed-information tasks like the NASA Moon Survival and Lost at Sea ranking exercises, where the team holds all the pieces but must pool them correctly.

Industry relevance: the method is model-agnostic (demonstrated across six backbones) and additive to existing frameworks — it is compared directly against SOTOPIA and AgentVerse using the same recasting procedure and initial casts — so it is a plausible layer on top of orchestration stacks rather than a replacement for them.

Future Directions

• Broaden the validation base: the fidelity study relies on one unified LLM judge (GPT-4o) and five designed scenarios; additional judges and structurally different scenarios would test external validity.
• More statistical power: the optimization experiments use twelve episodes per casting, which the authors acknowledge is small relative to the variance of LLM interaction; they report consistent directions across tasks, backbones, and initializations rather than per-cell significance claims.
• Finer-grained interventions: casting is an episode-level, coarse adjustment. The same measured trajectories could plausibly drive runtime mediation, targeted information routing, or norm enforcement mid-episode.
• Understanding model-specific social styles: the paper documents that backbones differ sharply (for example GPT-4o's 10.0% repair rate versus Claude Sonnet 4.6's 80.0%) but does not explain why, leaving open what to do when a team must mix backbones.

Target Audience

Researchers and engineers working on multi-agent LLM systems, agent orchestration, and LLM-based social simulation will get the most from this paper, particularly those frustrated by team failures that cannot be traced to any single model's capability. It also suits social scientists interested in whether LLM populations reproduce classical group-dynamics regularities, and practitioners who need a principled alternative to hand-tuning role prompts for agent teams.

Authors’ abstract

Large language model (LLM) agents are increasingly deployed as teams, yet they fail in characteristically social ways: information holders stay silent, dominant speakers anchor groups on wrong answers, and norm violations go unsanctioned. We present Agent Sociodrama, a framework for modeling and validating evolving social dynamics in LLM agent teams. Under fixed personas, agents' emotional states and directed, asymmetric relationships are updated every round by a unified external LLM judge and shape their subsequent interactions. Across six LLM backbones and five social scenarios, we assess social fidelity through classical social-theory patterns and agreement with human judgments. The resulting trajectories exhibit structured social patterns and model-specific differences. We then introduce Trajectory-Guided Persona Recasting (TPR), which diagnoses group-dynamic failures from interaction transcripts and social trajectories and recasts agent personas before rerunning the task. Across four collaborative benchmarks, TPR improves performance from both heterogeneous and randomly initialized agent persona compositions, yielding up to 83.3%p of accuracy improvement over the initial casting. On representative distributed-information cooperation and mixed-motive bargaining tasks, we outperform SOTOPIA and AgentVerse in three of four comparisons, by up to 37.5%p in Pareto rate. These results show that validated social trajectories provide actionable signals for repairing LLM agent teams.