Identifying and Mitigating Bottlenecks in Role-Playing Agents: A Systematic Study of Disentangling Character Profile Axes

AI summary

Overview

Research area: Natural Language Processing, specifically Large Language Model (LLM) role-playing agents (RPAs) and post-training alignment behavior.

Technical level: Advanced — the paper assumes familiarity with LLM training pipelines (pre-training, instruction tuning, post-SFT alignment such as DPO), decoding strategies, and automated LLM-as-Judge evaluation.

Scope (one sentence): A controlled diagnostic study of how three character-profile axes (Familiarity, Structure, Disposition) affect LLM role-playing fidelity across 211 personas and five open-source LLMs, plus a training-free decoding method (Field-Aware Contrastive Decoding) that closes the resulting Moral–Immoral performance gap.

What This Paper Is About

Role-playing agents are typically steered by a "character profile" — a written specification of who the agent should portray — but it has been unclear which parts of that profile actually determine role-playing quality, because prior work used ad-hoc templates and studied known or unknown characters in isolation rather than under controlled conditions. The authors build a standardized profile schema and a matched dataset of 211 personas to disentangle three profile axes, then test five LLMs on single-turn and multi-turn benchmarks. They find that how well-known a character is and how the profile is formatted barely matter, while whether the character is moral or immoral causes large, consistent performance drops — and they propose a decoding-time fix for that specific bottleneck.

Key Contributions

1. A three-axis diagnostic framework for character profiles. The paper organizes profile effects along Familiarity (Known vs. Unknown), Structure (Structured vs. Unstructured), and Disposition (Moral vs. Immoral), each conceptually anchored to a stage of standard LLM training: pre-training, instruction tuning, and post-SFT alignment.

2. A unified hierarchical schema and controlled dataset. Profiles are organized into 5 top-level dimensions (Personal Attributes, Personality Traits, Interpersonal Relationships, Motivations, Abilities), expanding to 15 mid-level dimensions and 28 leaf fields. From this schema the authors build 211 personas (109 Known, 102 Unknown; 106 Moral, 105 Immoral), with paired Structured/Unstructured variants of every profile.

3. Identification of Disposition as the dominant bottleneck, with mechanism evidence. Counterfactual rewrite experiments show performance follows dispositional content rather than fixed character identity, and an alignment-stage ablation on OLMo-3.1-32B-Instruct-SFT vs. -DPO shows that preference alignment widens the Moral–Immoral gap. Field-level analysis localizes the damage to value-laden fields, especially Morality and Goal within the Motivations dimension.

4. Field-Aware Contrastive Decoding (FACD). A training-free decoding strategy that contrasts the full profile against a sanitized profile with immoral, non-Personal-Attributes fields removed (via MoralBERT), amplifying suppressed profile signals at inference. It narrows the Moral–Immoral gap without sacrificing moral-character performance.

Main Findings

• Disposition is the dominant driver. Immoral characters score lower than Moral characters on every model and both benchmarks. On PersonaGym the gaps range from 0.20 to 0.35 (all p < 0.01); on CoSER the gaps range from 5.89 to 9.22 (all p < 0.001), with DeepSeek-v3.2 showing the largest drop of 9.22 points.

• Familiarity has negligible, benchmark-dependent impact. On PersonaGym, |Δ| ≤ 0.07. On CoSER, Unknown characters tend to score higher, but only DeepSeek-v3.2 reaches statistical significance (p < 0.05). A counterfactual swap of the Personal Attributes field changes scores within ±0.2 on PersonaGym and ±1.1 on CoSER.

• Structure has virtually no impact. PersonaGym gaps between Structured and Unstructured are at most 0.03 and never significant; CoSER effect sizes are Δ ≤ 2.49 and inconsistent in direction. The authors note these null results hold when profiles are rich, schema-based, and semantically matched.

• Counterfactual rewrites confirm content, not identity, drives the gap. Rewriting toward Immoral degrades performance by up to −0.82 (PersonaGym) and −22.17 (CoSER), while rewriting toward Moral improves it by up to +0.34 and +3.47 respectively. A single-field flip of only the Motivations dimension (or its Morality/Goal leaves) reproduces the directional pattern and recovers roughly half of the full-rewrite effect.

• Post-SFT alignment amplifies the gap. Comparing OLMo-3.1-32B-Instruct-SFT and -DPO on CoSER, the average Moral–Immoral gap widens from −4.53 to −7.63. Character Fidelity moves from a non-significant −2.44 under SFT to a significant −9.66 after DPO; while DPO slightly improves Moral Character Fidelity, it substantially decreases it for Immoral characters. A subsequent RLVR-aligned checkpoint partially improves overall scores but does not close the DPO-amplified gap.

• The bottleneck is field-localized. Personality Traits remain relatively stable across increasing numbers of Immoral characters, whereas Motivations fields decline steepest and most consistently, especially Morality and Goal. Abilities and Interpersonal Relationships show intermediate degradation.

• FACD narrows the gap without hurting moral performance. Default CoSER gaps of −6.00 (Qwen3-8B) and −5.89 (Mistral-Small) shrink to −4.50 and −2.22. Moral average rises for Qwen3-8B (24.88 → 27.07) and stays comparable for Mistral-Small (35.04 → 34.31). Gains concentrate in Character Fidelity: Qwen3-8B Immoral Character Fidelity more than doubles (6.39 → 14.38), flipping the gap from −5.00 to +0.47; Mistral-Small rises from 15.78 to 24.38 (−11.34 → −1.84). FACD helps most when all three characters in a session are Immoral (Qwen3-8B roughly 16 to 20; Mistral-Small 25 to 29).

• The negative prompt must be both character-grounded and polarity-aware. Ablations show that restricting the negative prompt to Personal Attributes only yields low scores for both groups (Avg. 20.21), prompt-level Stay-In-Character Prompting attains a smaller gap only by degrading Moral performance (Avg. 21.52), and randomly selected fields give limited gains. FACD achieves the highest average (24.82).

• Safety cost is marginal. Using Llama-Guard-4-12B on per-character utterances, flagged ratios stay below 0.7% under FACD, with all increases under 0.4 percentage points (e.g., Qwen3-8B Immoral 0.304% → 0.687%; Mistral-Small Immoral 0.221% → 0.495%).

Methodology in Plain English

The authors start from a hypothesis: because LLMs are built in three stages (pre-training, instruction tuning, and post-training alignment), the influence of a character profile should also fall into three separable categories. Familiarity tests what the model already knows about a character. Structure tests how the profile is written — a filled-in schema versus flowing prose. Disposition tests the moral polarity of the character's described nature.

To compare these fairly, they build one shared template with 5 dimensions and 28 leaf fields, informed by psychological theories and by 50 popular user-made profiles from Risu AI. They fill it with 109 known characters pulled from Fandom pages for 34 famous works and summarized by Claude-4.5-Sonnet, and 102 synthetic "unknown" characters generated by sampling variables, having GPT-oss-120B write stories around them, and summarizing each episode. To create unstructured versions, they have Claude-4.5-haiku rewrite each structured profile as free-form prose while preserving meaning. An audit confirms the variants are comparable: about 550–620 words and 135–160 atomic facts per profile, with near-identical average semantic similarity (0.75–0.76).

Each profile is labeled Moral or Immoral by GPT-4o on a 10-point scale (Immoral at ≤5, Moral at ≥6), yielding 106 Moral and 105 Immoral characters. The five backbone LLMs are then run on PersonaGym (single-turn interviews, five rubrics) and CoSER (multi-turn narratives, three rubrics, 18 turns, three seed scenarios generated by gemini-2.5-pro), with GPT-4o as judge.

To test causation rather than correlation, they edit profiles directly: swapping Personal Attributes between known/unknown pairs, and rewriting fields to the opposite moral polarity. To explain the mechanism, they compare an SFT checkpoint to a DPO checkpoint of the same model. To fix the problem without retraining, they run two forward passes — one with the full profile, one with a sanitized profile that has immoral non-Personal-Attributes fields stripped out — and amplify the difference in token probabilities, with α set to 1.

Why This Matters

Impact on research. The paper reframes RPA evaluation from "which prompt template works" to "which profile attributes matter and why." Its central claim — that alignment procedures systematically suppress fidelity to immoral personas while leaving other profile axes untouched — connects alignment research to role-play fidelity, a link previously studied only indirectly. The finding that a single value-laden field flip reproduces roughly half the effect gives future work a precise target.

Real-world applications:

• Character-based chat services (the paper cites Character.AI as an industry example) that must host large and diverse casts, including antagonists, without degrading role fidelity.
• Game and interactive-fiction systems where non-player characters need consistent villainous or morally grey motivations rather than drifting toward polite assistant behavior.
• Evaluation and red-teaming pipelines that need a controlled way to stress-test how a model handles immoral personas before deployment.
• Deployment decisions around the fidelity–safety trade-off, since the paper explicitly frames this as an operational consideration rather than a solved problem.

Industry relevance. The mitigation method is training-free and requires no parameter updates, so it can be layered onto an existing deployed model. The paper provides an open repository at https://github.com/yonghyun010102/RPA_Bottleneck.git, and its diagnostic axes offer a practical checklist for anyone writing character profiles at scale.

Future Directions

• Beyond binary disposition. The authors acknowledge their Moral/Immoral split may not capture anti-heroes or morally ambiguous personas, leaving finer-grained polarity scales as a clear next step.
• Scaling FACD evaluation. Because contrastive decoding requires parallel forward passes, FACD was evaluated on only a subset of frontier LLMs under computational constraints; testing on larger architectures remains open.
• Long-horizon safety evaluation. The current safety assessment is utterance-level and cannot measure persuasive or normalizing effects that accumulate over long multi-turn conversations.
• Separating alignment effects from content difficulty. The authors note that residual gaps (even under Instruct-SFT and after FACD) may stem not only from alignment but also from immoral profiles encoding more aversive content such as conflict, manipulation, and harm, which could independently raise modeling difficulty.

Target Audience

This paper is most useful for NLP researchers working on LLM role-playing, persona conditioning, or alignment behavior; practitioners building character-based chat or game systems who need to deploy antagonistic or morally complex personas reliably; and safety teams evaluating the fidelity–safety trade-off of inference-time steering methods. Readers should be comfortable with terms like post-SFT alignment, DPO, contrastive decoding, and LLM-as-Judge evaluation, though the experimental design and conclusions are presented clearly enough for a technically literate non-specialist to follow with some effort.

Authors’ abstract

While Large Language Model (LLM) role-playing agents have advanced rapidly, it remains unclear which profile elements genuinely drive role-playing quality. To bridge this gap, we introduce a systematic diagnostic framework that disentangles the impact of character profiles along three axes: Familiarity (Known vs. Unknown), Structure (Structured vs. Unstructured), and Disposition (Moral vs. Immoral). Utilizing a unified hierarchical schema (5 dimensions, 28 fields), we construct a controlled dataset of 211 personas and evaluate five LLMs on both single- and multi-turn interactions. Our results reveal a striking asymmetry: \textbf{Familiarity} and \textbf{Structure} show negligible impact, while \textbf{Disposition} produces large, consistent performance degradation for immoral characters across all conditions. Further analyses suggest that the Moral--Immoral gap is amplified by post-SFT alignment, and that this degradation varies substantially across profile attributes. To mitigate this bottleneck, we propose Field-Aware Contrastive Decoding (FACD), a training-free strategy that amplifies suppressed disposition-sensitive signals, significantly closing the performance gap without sacrificing moral-character performance.