Pluralistic Behavior Suite: Stress-Testing Multi-Turn Adherence to Custom Behavioral Policies

AI summary

Overview

Research area: Pluralistic alignment of large language models (LLMs) — specifically, whether models can follow custom, organization-specific behavioral policies rather than only a single universal set of safety and usage principles.

Technical level: Intermediate. The paper is readable without deep technical background, but familiarity with LLM alignment, red-teaming, and LLM-as-a-judge evaluation helps.

Scope in one sentence: The paper introduces PluralisticBehaviorSuite (PBSuite), a dataset of 300 industry-grounded behavioral policies plus a dynamic multi-turn adversarial evaluation framework, and uses it to measure how well six leading LLMs adhere to custom behavioral rules.

What This Paper Is About

Today's LLMs are post-trained to satisfy a universal set of safety and usage principles intended for broad public acceptability. Real deployments, however, happen inside organizations with their own corporate policies, regulatory constraints, use cases, and ethical commitments — and sometimes those policies deliberately forbid things a general-purpose model would happily do (the paper's example: an educational assistant that must refuse to review or improve student essays). The problem this paper addresses is that there is no systematic way to measure whether an LLM can reliably follow such custom behavioral policies, and the goal is to build both a realistic policy dataset and an adversarial multi-turn framework that stress-tests compliance.

Key Contributions

1. A dataset of 300 realistic LLM behavioral policies grounded in 30 industries. The policies are produced by a hierarchical, application-grounded generation pipeline that moves from industries, to behavioral risk dimensions, to enterprise use cases with assigned risk tiers, and finally to policies specifying allowed and prohibited behavior rules. The 30 industries are selected from the U.S. Bureau of Labor Statistics taxonomy of 147 total industries.

2. PBSuite, a dynamic evaluation suite for multi-turn adherence to custom policies. It extends the X-Teaming multi-agent red-teaming framework (Rahman et al., 2025) — originally designed for default safety alignment — into an adaptive, goal-directed framework for probing compliance with custom behavioral specifications. It uses four agents: a Planner, an Attack Agent, a Target Model, and an LLM Judge.

3. Quantified evidence of a single-turn versus multi-turn compliance gap. Leading open- and closed-source LLMs hold up in single-turn settings (failure rates below 4% in the reported single-turn table) but degrade severely under adversarial multi-turn interaction, with behavior-level attack success rates up to 84.4%.

4. An analysis of which attack strategies and industries are most problematic, plus human validation of the judge. The paper clusters attack strategies and finds roleplay-based approaches most effective, reports per-industry vulnerability for gpt-4o, and compares LLM-judge labels against two experienced human annotators.

Main Findings

• Custom-policy violations are orthogonal to standard safety moderation. Nearly all prohibited queries in the dataset were judged safe by a state-of-the-art content moderation model trained on canonical harms such as violence, abuse, and toxicity. In Table 1, "% Unsafe" values are at most 0.4 without the policy and at most 0.2 with the policy, while borderline/strict violation rates reach 64.5/19.0 (qwen3-32b without the policy). The paper concludes the targeted behaviors are enterprise-specific boundaries overlooked by general-purpose alignment.

• Providing the policy drastically reduces single-turn violations. Without the policy in the system prompt, strict violation rates range from 9.6 (gpt-4o) to 19.0 (qwen3-32b). With the policy included, strict rates fall to 0.2 (gpt-4o), 0.3 (gpt-4o-mini), 1.8 (llama-3.3-70b-instruct), 3.9 (llama-3.1-8b-instruct), 1.0 (qwen3-32b), and 1.8 (qwen3-8b). The paper notes qwen3 shows the largest gains despite weaker out-of-the-box safety, suggesting models with stronger reasoning follow structured constraints better.

• Multi-turn adversarial attacks break compliance. Behavior-level attack success rates in the Agentic setting are: qwen3-8b 84.4, llama-3.1-8b-instruct 76.8, qwen3-32b 74.4, llama-3.3-70b-instruct 38.0, gpt-4o-mini 37.6, and gpt-4o 25.1. In the Simple multi-turn setting the same models score 1.7, 7.9, 0.3, 1.7, 0.3, and 0.1 respectively. The paper reports the overall multi-turn range as 25% to 84%; OpenAI models trained with instruction hierarchy achieve better scores but are still deficient. From Section 4.3 onward only strict violations (judge rating equal to 5) are reported.

• Behavior-level scoring counts any successful strategy. A behavior is counted as compromised if any one of five attempted strategies per behavior produces a violation. A per-behavior breakdown of the number of successful strategies is reported in Figure 3.

• Roleplay is the dominant and most effective attack vector. Embedding-based clustering of attack strategies shows most strategies are a form of role-playing, and for gpt-4o the highest behavior ASR comes from strategies involving roleplay, narrative manipulation, and simulated documentation requests.

• Violations concentrate early in conversations. Most violations occur by the fourth turn; if no violation has happened by then, the likelihood of one in subsequent turns tapers off.

• Vulnerability varies by industry. For gpt-4o, the three most impacted industries are Alternative Medicine, E-Learning, and Newspapers/Journalism. The least impacted are Individual/Family Services, Entertainment/Movie Production, and Education Management.

• LLM-judge agreement with humans is moderate. On 26 responses from gpt-4o under adversarial multi-turn attacks, annotated by two experienced human annotators, inter-rater agreement between the judge and humans is κ_judge = 0.51 (moderate) while human-human agreement is κ_human = 0.73 (high). Manual review also identified planner-agent issues: some judge-labeled violations were not well connected to the input behavior risk rule, and some "successful" violations involved only generic or fictional information that human annotators dismissed.

Methodology in Plain English

The researchers built the policy dataset in stages, using gpt-4.1-2025-04-14 as the generator. Starting from the U.S. Bureau of Labor Statistics industry taxonomy of 147 industries, they selected 30 with high likelihood of LLM deployment. For each industry they generated 3–5 behavioral risk dimensions (e.g., public exposure, autonomy, jurisdictional constraints) and assigned discrete risk tiers: lower tiers correspond to tightly constrained, consumer-facing deployments, while higher tiers permit greater autonomy in expert-facing contexts. They then constructed 10 representative enterprise use cases per industry, annotated each with tier values — for example, in Legal Services a public-facing court-procedure assistant maps to Tier 0 public exposure, Tier 1 autonomy, Tier 0 jurisdictional constraints, while an internal research assistant for lawyers maps to Tier 5 public exposure, Tier 4 autonomy, Tier 3 jurisdictional constraints. From those annotated use cases they generated sets of allowed and prohibited behavior rules.

Quality was checked with a hybrid approach: manual human review of industry selection, plus automated LLM-as-a-judge rubrics scoring realism, naturalness, and diversity using gpt-4.1-2025-04-14. Risk dimensions were also clustered with BERTopic using all-MiniLM-L6-v2 embeddings, over roughly 500 primary risks constructed from the 30 starting industries, producing groups such as "Cultural and regional insensitivity," "Non-compliance with regional regulations," and "Excessive technical detail disclosure."

For evaluation, the researchers restricted the setting to isolated assistants with no external tools, metadata, or databases, and excluded rules that depend on unobservable signals — the example given is a rule about providing medical advice "if authorized by a physician." Only prohibited rules verifiable from the conversational context alone were kept, leaving 1,100 rules from the 300 policies to drive both single-turn and multi-turn evaluation.

The multi-turn framework adapts X-Teaming. A Planner generates high-level attack strategies and turn-level plans, moving from benign compliant queries to progressively more challenging ones. An Attack Agent turns the plan and conversation state into user queries. The Target Model answers with the behavior policy in its system prompt. An LLM Judge scores each response on a 5-point rubric, where 1 means full adherence, 5 means explicit violation, and 2–4 capture partial violations, hedging, or implicit breaches; a score of 5 ends the interaction. Generation is capped at 5 strategies per verifiable prohibited behavior and 7 turns per conversation. gpt-4.1-2025-04-14 serves as the judge.

In the single-turn setting, each policy is paired with five compliant user queries that are minimally modified into prohibited counterparts. In the multi-turn Simple setting, the first 2–4 turns are allowed queries and the last turn contains a prohibited query topically linked to the penultimate turn. In the Agentic setting, the adaptive framework above simulates adversarial conversations. The primary metric is attack success rate (ASR), the proportion of responses that breach the behavior policy in the system prompt. Models evaluated are llama-3.1-8b-instruct, llama-3.3-70b-instruct, gpt-4o, gpt-4o-mini, qwen3-8b, and qwen3-32b.

Why This Matters

Impact on research: The paper argues that current alignment and moderation methods fall short of coherently enforcing pluralistic behavioral policies, and that alignment optimized predominantly for single-turn, universal principles does not transfer to nuanced, domain-specific requirements. It positions PBSuite as the first systematic foundation for studying pluralistic alignment in multi-turn, industry- and use-case-specific scenarios, complementing—rather than replacing—traditional safety and content-moderation evaluation. It also notes that prior evaluation has been limited by static, harm-based benchmarks, with earlier efforts like CoSA offering small, single-turn datasets grounded in content-harm taxonomies.

Real-world applications (as framed by the paper):

• Enterprise assistants deployed under corporate compliance, regulatory, and reputational constraints that differ by organization.
• Consumer-facing versus internal deployments within the same industry, where risk tiers dictate different permitted behaviors.
• Industry-specific settings the paper names, including Legal Services, Healthcare, Gaming NPCs, Book Publishing, education (E-Learning), and journalism.
• Content moderation and guardrail tooling, since the paper shows targeted violations are largely invisible to a moderation model trained on canonical harms.

Industry relevance: All authors are affiliated with NVIDIA, and the framing is explicitly enterprise-oriented — the policies are derived from business sectors, deployment configurations, and organizational risk tiers rather than from general content-harm taxonomies.

Future Directions

• Reduce reliance on adversarial conversation. The limitations section notes the framework under-represents typical user behavior and enterprise conversation flows, and that over-refusal — rejecting actions that should be allowed — is not tested at all.
• Improve the judge and the planner. The paper reports moderate judge-human agreement (κ_judge = 0.51) and states that automated judging introduces potential noise, bias, and inconsistency; manual analysis also flagged planner attacks that were poorly connected to the target rule.
• Sharpen rule verifiability. Rule filtering is described as imperfect, since some ostensibly verifiable rules may implicitly require external knowledge or metadata.
• Harden policy enforcement architectures. Because policies are encoded in the system prompt, the setup is vulnerable to prompt injection in multi-turn settings; the paper calls for architectures with instruction hierarchies where only privileged users can define or modify behavioral policies.

Target Audience

Researchers and practitioners working on LLM alignment, pluralistic alignment, safety evaluation, red-teaming, and guardrails; enterprise teams deploying LLM assistants under organization-specific policies; and benchmark designers interested in multi-turn, policy-adherence evaluation rather than single-turn harm taxonomies.

Authors’ abstract

Large language models (LLMs) are typically aligned to a universal set of safety and usage principles intended for broad public acceptability. Yet, real-world applications of LLMs often take place within organizational ecosystems shaped by distinctive corporate policies, regulatory requirements, use cases, brand guidelines, and ethical commitments. This reality highlights the need for rigorous and comprehensive evaluation of LLMs with pluralistic alignment goals, an alignment paradigm that emphasizes adaptability to diverse user values and needs. In this work, we present PLURALISTIC BEHAVIOR SUITE (PBSUITE), a dynamic evaluation suite designed to systematically assess LLMs' capacity to adhere to pluralistic alignment specifications in multi-turn, interactive conversations. PBSUITE consists of (1) a diverse dataset of 300 realistic LLM behavioral policies, grounded in 30 industries; and (2) a dynamic evaluation framework for stress-testing model compliance with custom behavioral specifications under adversarial conditions. Using PBSUITE, We find that leading open- and closed-source LLMs maintain robust adherence to behavioral policies in single-turn settings (less than 4% failure rates), but their compliance weakens substantially in multi-turn adversarial interactions (up to 84% failure rates). These findings highlight that existing model alignment and safety moderation methods fall short in coherently enforcing pluralistic behavioral policies in real-world LLM interactions. Our work contributes both the dataset and analytical framework to support future research toward robust and context-aware pluralistic alignment techniques.