Learning to Make Friends: Coaching LLM Agents toward Emergent Social Ties

AI summary

Overview

Research area: Multi-agent large language model (LLM) simulation of online social behavior, bridging LLM agent-based modeling and computational social science.

Technical level: Advanced — the paper defines formal reward functions, a gated dynamic-network update rule, and proceeds through a mathematical formulation of tie formation.

Scope in one sentence: The paper builds a multi-agent LLM "conversation room" in which 30 agents with psychologically grounded personas interact over 15 rounds on climate change, receive behavioral rewards drawn from gratifications theory, optionally receive a coaching tip, and are evaluated on whether realistic social ties and network structures emerge endogenously.

What This Paper Is About

Individual LLM agents have demonstrated linguistic, psychological, and cognitive capabilities, but how they behave collectively inside networked environments is far less understood. The authors ask whether LLM agents can reproduce the complex social dynamics of human online behavior — shaped by homophily, reciprocity, and social validation — and which memory and learning mechanisms allow those dynamics to emerge. Their goal is a controlled testbed in which links form on their own instead of being pre-defined on a fixed graph.

Key Contributions

1. Conversation Room. An interactive environment integrating both public channels (posts and comments) and private channels (direct messages), letting agents deploy distinct strategies across contexts and allowing systematic analysis of how channel choice shapes conversational dynamics.
2. Reward Structures. Formally defined reward functions that translate empirically grounded human motivations — social interaction (SOC), information seeking (INF), self-presentation (PRE), coordination (COORD), and emotional support (EMO) — into agent objectives, creating a bridge between gratifications theory and multi-agent LLM learning. A compositional reward combines these components with weights λ_r that sum to 1.
3. Tie Formation. Mechanisms by which social ties emerge endogenously from conversational interactions without pre-defined network structure, so that support, alignment, and homophily can be studied as drivers of group formation. Ties are initialized at zero and strengthened or decayed over time.
4. Emergent network evaluation. Using the evaluation protocol and real-network baselines of Chang et al. (2025), the authors compare two tie-reweighting methods — a predefined-signal heuristic versus an LLM text-based scoring approach — against statistics from real social graphs.

Main Findings

• Learning improves over rounds. Performance generally increases over simulation rounds both with and without the coach (Figure 2a versus Figure 2b).
• Information seeking is the easiest policy to learn. The INF reward is typically the easiest to learn, because it is primarily driven by exposure to new content.
• Emotional support reaches comparatively high values, reflecting the generally positive behavior of most agents.
• Self-presentation is partially controllable through personal posts.
• Coordination-dependent policies are hardest. Social interaction (SOC) and coordination (COORD) are more difficult to learn.
• Coaching helps early, not uniformly late. The coach accelerates early learning for some policies but does not yield a uniform late-round improvement; COORD and EMO are the exceptions, likely benefiting from improved targeting. Final performance levels are broadly similar within the variability bands, and aggregate gains remain modest given task complexity.
• Text-based tie formation is more stable than the heuristic. The heuristic approach shows greater variability across thresholds — particularly in density, average clustering, and average shortest path length — whereas the LLM-based text approach produces more stable results that more often fall within or near real-network ranges.
• Connectivity saturates, modularity lags. Differences between methods are less pronounced for the largest connected component, where both yield near-complete networks, and more modest for modularity, where both remain below real-network levels on average.
• Some objectives are unattainable in principle, because achieving them would require control over other agents.
• Limits of scale. The study is explicitly conservative in scale — |V| = 30 agents and T = 15 rounds, with limited replications and simulation starting from empty networks.

Methodology in Plain English

The framework has two parts: creating personas and running the simulation.

Persona creation. Rather than sampling attributes from a user population, a planner agent draws on corpora of real online discussions on a chosen topic (the experiments use data from Kong et al. 2022). Because many users on such platforms are anonymous, the authors focus on features that can be inferred from the content itself: role, persona description, topic stance, and communication style. Each persona has three layers — personality (Big Five traits assessed with the Mini-IPIP, plus finer facets such as sociability and stubbornness, and constructs such as risk preference), task (the motive for engaging, such as social interaction, information seeking, or entertainment), and memory (conversation memory, relationship memory, and opinion memory).

Simulation. At t = 0 the social network has no edges. In round t = 1, agents are unaware of one another and must make a public POST. In later rounds they choose among POST, COM (comment), DM (direct message), or NOT (no action), with N = 3 actions per agent per round. Each round has three phases: Plan-Execute-Reflect (agents plan a pre-specified number of actions from the previous round's content batch, their reward, and side information; executed actions are stored in memory), Vote (agents like, dislike, or stay neutral on publicly visible content), and Reweighting (agents update tie strengths).

Rewards. Each round, every agent's behavior is scored by bounded reward functions in [0, 1]: SOC (sending and receiving direct exchanges, weighted by β^SOC), INF (topic novelty and entropy-based topic diversity, weighted by β^INF), PRE (posting volume plus tanh-normalized net likes/dislikes, weighted by β^PRE), COORD (mentions received plus reciprocity of DM replies, weighted by β^COORD), and EMO (average sentiment of received direct exchanges, on a scale from hostile to supportive). These combine into a single weighted reward, which agents learn from in-context, with a bandit-style plan-execute-reflect loop.

Coaching. An optional "coach" prompt supplies a tip so agents can focus on constructing a well-structured response instead of solving the whole planning problem. If a candidate plan fails validation checks, the agent is re-prompted up to three times.

Tie formation. The interaction structure is a weighted, directed network where entry [A_t]_uv in [0, 1] is the strength of the tie from u to v. A binary activation ζ_t(u→v) fires when u interacts with v through an Address channel (direct message or public mention) or an Engage channel (comment or vote). Active rounds raise tie strength based on an evidence score e_t(u→v) — aggregating novelty, reciprocity, approval (likes), and affective tone — capped by Δ_max and gated by a minimum-evidence threshold ξ; passive rounds multiply the tie by (1 − δ), where δ can be parameterized by a half-life h as δ = 1 − 2^(−1/h). Evidence scores can either be computed from predefined signals or produced by an LLM prompt scoring the round's text. For reporting only, the final adjacency matrix is symmetrized and thresholded at θ to yield a binary undirected graph.

Experimental setup. |V| = 30 agents, T = 15 rounds, N = 3 actions per agent per round, topic: climate change. Experiments were run with OpenAI's GPT-4o mini because of API rate limits.

Why This Matters

Impact on research. The platform offers a principled, controlled testbed for studying collective dynamics in LLM populations — echo-chamber formation, niche community dynamics, and the design of mitigating interventions — while revealing where artificial agents approximate versus diverge from human-like social behavior. It positions agents as a possible privacy-preserving alternative to experiments on real users, an explicit response to controversy over a recent Reddit study (O'Grady, 2025) that used AI-generated content on users without consent.

Real-world applications:

• Policy testing: simulating moderation strategies, such as content removal, before deploying them at scale (building on Schneider and Rizoiu, 2023).
• Opinion and market dynamics: simulating the spread of opinions, the formation of online communities, or the contagion of consumer behavior.
• Security analysis: exploring defenses against manipulation and detecting coordinated influence operations (Alizadeh et al., 2020).
• Digital twins of platforms: constructing a model of social media users that captures real behavior without compromising individual privacy (Bail, 2024).

Industry relevance. Social media platforms, moderation teams, and policy bodies could use such simulations to stress-test interventions; the framework's modular reward design lets a platform emphasize particular engagement forms through the λ weights, making it adaptable to different platform incentives.

Future Directions

• Scaling up. The authors name scaling to larger cohorts and longer horizons beyond |V| = 30 and T = 15, plus more replications, as the clear next step for external validity.
• Seeding pre-existing ties. Starting from empty networks is a limitation; initializing with real or synthetic prior structure would test how emergence differs from reconfiguration.
• Intervention stress tests. Running intervention experiments to strengthen policy relevance.
• Dynamic structure characterization. Further work is needed to characterize how emergent structures evolve over time, given the dynamic interplay between agent relationships, tie-strength updates, and topical alignment.
• Better coaching and richer action spaces. With a small number of actions per round, coaching produced only modest gains; whether richer action budgets make coaching more effective remains open.

Target Audience

Researchers and graduate students in computational social science, multi-agent LLM systems, and agent-based modeling; AI safety and trust-and-safety practitioners interested in simulating manipulation and moderation; and platform or policy analysts who need a controlled environment for testing interventions. Readers will benefit most from familiarity with reinforcement learning basics and network science concepts such as clustering, modularity, and assortative mixing, since the paper is mathematically dense and reports its evaluation as a first step rather than a finished benchmark.

Authors’ abstract

Can large language model (LLM) agents reproduce the complex social dynamics that characterize human online behavior -- shaped by homophily, reciprocity, and social validation -- and what memory and learning mechanisms enable such dynamics to emerge? We present a multi-agent LLM simulation framework in which agents repeatedly interact, evaluate one another, and adapt their behavior through in-context learning accelerated by a coaching signal. To model human social behavior, we design behavioral reward functions that capture core drivers of online engagement, including social interaction, information seeking, self-presentation, coordination, and emotional support. These rewards align agent objectives with empirically observed user motivations, enabling the study of how network structures and group formations emerge from individual decision-making. Our experiments show that coached LLM agents develop stable interaction patterns and form emergent social ties, yielding network structures that mirror properties of real online communities. By combining behavioral rewards with in-context adaptation, our framework establishes a principled testbed for investigating collective dynamics in LLM populations and reveals how artificial agents may approximate or diverge from human-like social behavior.