Your LLM, Your Style: Behavioral Mode Axes for LLM Behavioral Control

AI summary

Overview

Research area: LLM behavioral evaluation and activation-space steering (AI psychometrics, interpretability, and model control).

Technical level: Advanced. The paper combines psychometric instrument design (BFI-2, DOSPERT, HEXACO, GCS, UPPS), activation steering in the residual stream, layerwise sweep analysis, and sparse-autoencoder feature inspection.

Scope: The paper introduces a situated behavioral-data framework, 3,200 contrastive behavioral probes over 20 behavioral patterns and four prompt registers, and Behavioral Mode Axes (BMAs) that steer those behavioral patterns in open-weight LLMs.

What This Paper Is About

Existing studies of LLM "personality" mostly administer human questionnaires to models in a first-person self-report frame, producing profiles that are sensitive to wording and option order and that do not reliably track what models actually do in concrete situations. This paper instead measures how models choose, advise, and act inside concrete scenarios, then asks whether those situated behavioral tendencies can be causally controlled by adding a direction to the model's internal activations. Its central claim is that LLM personality-like tendencies are better described as measurable, controllable behavioral modes than as abstract self-report traits.

Key Contributions

1. A situated behavioral-data (B-data) framework that operationalizes LLM behavioral personality beyond self-report, grounding personality-related constructs in concrete choices, advice, and task behavior across 3,200 contrastive probes, 20 behavioral patterns, and four prompt registers.

2. Behavioral Mode Axes (BMAs), activation-space directions derived from contrastive behavioral traces, which steer behavioral modes. The authors report that BMA control generalizes across representative open-weight models and concentrates in Behavioral Control Layer (BCL) bands.

3. Evidence that BMA construction matters: axes derived from intermediate behavioral rationales (thought-derived BMA-T) give cleaner control than axes derived from final responses (BMA-R), which are more prone to "trait drift."

4. A trait-drift diagnostic, including a sparse-autoencoder readout at the BCL showing that BMA-T aligns with the intended behavioral motive while BMA-R entangles that motive with an output-style mechanism such as dismissiveness.

Main Findings

• Self-report diverges from behavior: Across all model–subdomain pairs, the average gap between questionnaire self-reports and behavioral profiles is 22.7 percentage points; 34.4% of pairs differ by at least 25 points and 20.0% differ by at least 40 points. The largest subdomain-level gap is Negative Urgency at 47.5 points, where models self-report stronger emotion-driven impulsivity than they enact.

• Profiles are stable within a register: Split-half analyses of independently sampled probe halves recover highly similar profiles within registers, with a mean split-half correlation of 0.933 (0.963 after Spearman–Brown correction).

• Profiles are register-dependent: Across the nine profile models, cross-register profile correlations average 0.76 and range from 0.37 to 0.97. The two advice registers are most similar (mean r = 0.89), while first-person and task profiles are less aligned (mean r = 0.63). The mean four-register range is 23.4 percentage points across model–subdomain pairs.

• Steering works, but only in some layers: In Llama-3.1-8B, mean clean directional range peaks around L08–L12 (0.82–0.89) and drops below 0.30 after L14. Effective control is not distributed uniformly across depth.

• BCL bands recur but move with the model: Llama models peak early in normalized depth (0.26–0.28), Qwen models shift progressively deeper with scale (0.41, 0.45, and 0.51 for 7B, 14B, and 32B), and Gemma models occupy a middle-depth region (0.44–0.49).

• Control transfers across registers: A first-person thought-derived BMA applied at each model's BCL layer moves the target-pole choice rate from a 21.3% baseline to 82.7% and 9.6% at the two endpoints (mean unknown rate 0.10%). Averaged over seven models and 20 subdomains, mean ΔA is 79.6 in-register and 67.4 on task.

• Thought- and response-derived axes encode different things: Across all 20 subdomains, the thought- and response-derived axes at the BCL are only weakly aligned, with a mean cosine of 0.37.

• Trait drift is mechanism substitution, not failure to move: In an Organization scenario, both BMA-T and BMA-R shift Llama-3.1-8B away from the high-Organization choice, but BMA-T frames it as an engaged preference for flexible, just-in-time handling, while BMA-R frames it through low effort and consequence dismissal. The sparse autoencoder shows BMA-T aligning with Feature 17024 (improvising/adapting on the fly, marked on-construct) and BMA-R aligning with Feature 103914 (dismissive, low-commitment stance, marked off-construct). Drift is especially pronounced in Organization, Recreational Risk, GCS Yielding, HEXACO Sincerity, and UPPS Lack of Perseverance.

Methodology in Plain English

The researchers treat model behavior, not self-description, as the unit of measurement. For each of 20 behavioral patterns derived from validated psychometric facets, they write scenarios in which two opposing courses of action are both plausible and competent, so that a model cannot simply pick the socially desirable answer. Each pattern is instantiated across four interaction registers: first-person choice, daily advice, task advice, and task execution. This yields 3,200 probes, balanced across subdomain-register cells.

A parser converts each model response into a low-pole or high-pole choice, and high-pole rates are aggregated by model, subdomain, and register to produce a behavioral profile. Probe quality control checks structure, pole assignment, label leakage, and whether each option actually reflects its intended behavioral mode, and profile stability is tested by splitting probes within each subdomain.

For control, the authors extract two kinds of direction. BMA-R is built from the activations of final responses instantiating the low- and high-pole modes; BMA-T is built from intermediate behavioral rationales that state why each mode is attractive. Each mean-difference vector is normalized, and at inference the scaled vector is added to the residual stream at a chosen layer during both prefill and decoding.

Steering is evaluated on held-out probes with scenario sets separate from those used for axis construction. The authors measure the pole rate A and the "unknown" rate U (unparseable generations), and search over coefficient intervals around zero, retaining only intervals where both the mean and maximum unknown rates stay below 1%; the largest resulting directional range is the "clean directional range." Contiguous layer regions with consistently high clean directional range across subdomains are called Behavioral Control Layer (BCL) bands. Open-ended generations are scored by an LLM judge for target-mode expression and off-target drift, and a sparse autoencoder trained at the BCL is used to identify which features each axis projects onto.

Why This Matters

• Impact on research: The paper challenges the dominant self-report paradigm in AI psychometrics, showing that questionnaire scores and situated behavioral profiles diverge substantially, and that profile shape shifts with interaction register. It also reframes activation steering for personality-like traits by separating what a direction encodes (a behavioral motive) from how it surfaces in text (an output style).

• Real-world applications:

  • Tuning assistant behavioral style for different interaction roles, such as advisory versus task-execution settings, using a single controlled direction.
  • Auditing whether a model's stated stance matches its action tendencies in concrete situations.
  • Characterizing behavioral shifts introduced by fine-tuning, as tested with LoRA adapters fine-tuned on good- and bad-medical data.
  • Building steering interventions that are faithful to intent rather than achieving a target label through an unintended mechanism such as dismissiveness.

• Industry relevance: For teams deploying open-weight models, the results suggest that behavioral style is both measurable and editable post hoc, that the effective intervention depth is model-specific, and that deriving steering vectors from final responses can silently trade the intended style for a different one.

Future Directions

• Extending the BCL analysis beyond the model families studied to determine how BCL location is determined by architecture, scale, and training.

• Clarifying what fixes BCL location, given that Qwen models shift progressively deeper with scale while Gemma models cluster at middle depth.

• Developing faithful-control criteria that detect trait drift automatically, since the current audit combines qualitative comparison, an LLM judge, and sparse-autoencoder feature inspection.

• Testing whether the situated B-data framework transfers to closed-weight models, and whether the 20 behavioral patterns and four registers cover the interaction contexts that matter in deployment. The paper's truncated content does not report results for closed-weight models.

Target Audience

Researchers and engineers working on LLM interpretability, activation steering, evaluation, and safety alignment; AI psychometrics researchers interested in alternatives to questionnaire-based personality measurement; and practitioners who need reliable, faithful control over how a deployed open-weight model behaves across different interaction roles.

Authors’ abstract

Large language models (LLMs) increasingly act in interactive settings where their behavioral styles affect user experience, safety, and downstream decision making. Existing LLM personality studies largely rely on self-report questionnaires administered in first-person settings, making the resulting profiles sensitive to surface elicitation choices and poorly grounded in concrete model behavior. In this work, we introduce a situated behavioral-data (B-data) framework for studying and controlling LLM behavioral personality. We construct 3,200 contrastive behavioral scenarios spanning 20 behavioral patterns and four prompt registers, grounded in validated psychometric facets such as BFI-2, DOSPERT, and HEXACO. Using this framework, we find that LLMs exhibit stable and model-specific behavioral profiles, while also revealing register-dependent shifts across first-person decisions, advice-giving, and task execution. We then show that these behavioral patterns can be controlled through Behavioral Mode Axes (BMAs), activation-space directions derived from contrastive behavioral traces. Compared with response-derived BMAs, which are more prone to trait drift, thought-derived BMAs more faithfully capture the intended behavioral mechanism and provide cleaner control over situated behavioral styles. Our results suggest that LLM personality-like tendencies are better understood not as abstract self-report traits, but as measurable and controllable behavioral modes grounded in concrete interaction contexts. Our code and data are available at https://github.com/lhz191/LLM-Behavioral-Personality.