LLM Rationalis? Measuring Bargaining Capabilities of AI Negotiators

AI summary

Overview

• Research area: Natural language processing — specifically the evaluation of large language model (LLM) agents on strategic, multi-turn economic interaction (bilateral negotiation), drawing on behavioral economics and agent-based modeling.
• Technical level: Intermediate. The paper combines a readable experimental design with a mathematical curve-fitting framework that requires some comfort with nonlinear models and summary statistics.
• Scope: The paper introduces a hyperbolic-tangent model of negotiation concession dynamics plus two derived metrics (burstiness τ and the Concession-Rigidity Index), then uses them to compare human negotiators against four OpenAI LLMs across natural-language and numeric-only protocols, with and without market context, and under six controlled power-asymmetry scenarios.

What This Paper Is About

Bilateral bargaining requires negotiators to adjust anchors, pacing and flexibility in response to an opponent's behavior, context and leverage. This paper asks whether current LLMs do anything comparable, or whether they simply return fixed offers regardless of the situation. The authors build a quantitative framework for tracking how offers move over rounds and apply it to a matched human-versus-LLM comparison in a single house-sale negotiation scenario, extended with controlled variations in power and information.

Key Contributions

1. A hyperbolic tangent model of concession dynamics. The authors fit y(x) = d + b · tanh(a·x − c) separately for buyers and sellers, where a is concession pace, b is concession span, d is the anchor point, and c is the horizontal shift. They argue this captures S-shaped profiles (early rigidity → mid-stage flexibility → late rigidity) that the classical single-exponent power law of Faratin et al. (1998) cannot represent.

2. Two new negotiation metrics. Burstiness τ = |a_scaled| × b_scaled (using min-max normalized parameters) measures peak concession rate, and the Concession-Rigidity Index CRI = 1 − 1.32/(|a|·T) measures the fraction of negotiation time spent conceding, bounded in [0, 1]. A data-driven variant, CRI*, measures rigidity as the fraction of time spent above a normalized speed threshold θ (e.g. θ = 0.1).

3. A matched human-versus-LLM empirical comparison. Human data came from the dataset of Heddaya et al. (2023); LLM data came from 100 self-play negotiations per model using GPT-o4-mini, GPT-4.1-mini, GPT-4o-mini and GPT-4.1-nano, in natural-language and numeric-offers-only formats.

4. Controlled power-asymmetry experiments and a qualitative strategy pipeline. Six scenarios varied BATNA strength and time pressure (+1 / 0 / −1) with and without rich market context, alongside a multi-stage annotation pipeline (Solo Performance Prompting with four expert personas, embedding-based clustering from 72 strategy labels down to 12 categories, validated at Fleiss's κ = 0.67 and average LLM-human agreement of 0.62).

Main Findings

• The negotiation setup. The house had a $240,000 asking price, a buyer private valuation of $235,000 and a seller private valuation of $225,000, making the Zone of Possible Agreement $225,000–$235,000 with a $230,000 midpoint.

• LLM anchors are stuck at ZOPA extremes. In natural language, all LLM buyers anchored at the seller's floor ($225k), whereas human buyers and sellers anchored near the midpoint ($229.5k–$230.5k). Several LLM sellers (GPT-4.1-nano, GPT-4.1-mini, GPT-o4-mini) disclosed reservation prices early, narrowing the effective ZOPA.

• Concession timing diverges from humans. Humans showed bursty concessions with sustained rigidity (τ ≈ 0.39–0.51, CRI ≈ 0.64–0.72). GPT-4.1-mini's buyer showed the flattest curve and near-zero rigidity (CRI = 0.008, τ = 0.18), while GPT-4o-mini's seller was more rigid than humans (CRI = 0.74) and GPT-o4-mini was the most flexible (CRI = 0.56).

• Settlement prices reveal role-agnostic optimization. Humans consistently settled near $230k. GPT-4.1-mini and GPT-4o-mini gravitated to $225k regardless of role; GPT-4.1-nano reached higher settlements ($228.5k) but still lacked bidirectional strategy.

• Numeric-only negotiation does not fix the problem. Under alternating-offers-only, humans still used bursty concessions (τ ≈ 0.36–0.49) and moderate rigidity (CRI ≈ 0.60–0.65), while GPT-4.1-mini showed minimal rigidity (CRI = 0.04, τ = 0.19) and GPT-4o-mini's seller stayed highly rigid (CRI = 0.71). GPT-4.1-nano again produced the closest-to-human pacing.

• Power asymmetries are largely ignored. In the six power scenarios, weak sellers still opened at extreme highs (GPT-4.1-series at $235k) and strong buyers at extreme lows (GPT-4o-mini below $225k). Only GPT-4.1-nano showed leverage-sensitive rigidity with context (CRI rising to 0.71 when strong, falling to 0.54 when weak).

• Outcomes cluster at the ZOPA edges. With context, GPT-4.1-nano hit the seller-maximum $235k in four of six scenarios, while GPT-4o-mini secured the buyer-maximum $225k in three.

• Without context, leverage sensitivity mostly disappears. GPT-4.1-mini buyers stayed rigid (τ ≈ 0.2 in every scenario) and full flexibility (CRI = 0), GPT-4.1-nano adapted (τ up to 0.73 when weak, down to 0.26 when strong; CRI 0.74 when weak, 0.24 when strong), and GPT-4o-mini swung from near-zero seller concessions (τ = 1.5 × 10⁻⁴) to generous buyer peaks (τ = 0.78).

• Limited strategic diversity. Anchoring & Gradual Concession was the top strategy for GPT-4.1-nano (50%), GPT-4o-mini (34%) and humans (18%). Humans favored Active Listening & Empathetic Probing (30%), a strategy used by all LLMs in under 5% of turns. GPT-o4-mini also engaged in Logrolling 6% of the time as a buyer.

• Deceptive tactics appeared. GPT-o4-mini fabricated BATNA claims 7% of the time, followed by GPT-4.1-nano (5%) and humans (3%).

• Capability does not scale with model quality. The authors conclude that LLM negotiation ability does not improve with better models, and that behavior shifts with new information are mostly model-specific rather than systematically more human-like.

Methodology in Plain English

The authors used a single, well-defined scenario: a house sale where the buyer can pay at most $235,000 and the seller will accept no less than $225,000, giving a $225,000–$235,000 agreement range. Human dialogues came from an existing dataset; LLM dialogues were generated by having each model play both sides of 100 negotiations.

To measure how offers moved, they fit a smooth S-shaped curve to each negotiation's sequence of offers. Four numbers describe each curve: how fast offers move, how far they move in total, where the midpoint of the movement sits, and when the steepest change happens. From these they derived two summary statistics — burstiness, which captures how sharply offers shift, and the Concession-Rigidity Index, which captures how much of the negotiation is spent actively conceding versus holding firm.

They then ran the same scenario in two formats (free text versus bare numbers), under six power configurations that varied how much each side needed a deal, and with or without detailed facts about the house and nearby sales. Finally, they used an LLM-driven annotation pipeline with four expert personas to label each turn with a negotiation strategy, clustered the labels into 12 final categories, and had human annotators validate the labels.

Why This Matters

The paper argues that LLM negotiation behavior is not merely suboptimal but structurally different from human behavior: models optimize toward fixed price targets rather than reasoning about an opponent's position, leverage or context. If that holds, negotiation benchmarks that only score final prices would miss the underlying failure.

Impact on research. The tanh framework and burstiness/CRI metrics give researchers interpretable tools for describing concession trajectories, replacing a single-parameter power law that cannot express changing curvature. The matched human-versus-LLM design and the qualitative strategy taxonomy also provide templates for evaluating strategic behavior rather than just outcomes.

Real-world applications:

• Automated negotiation agents used in procurement, sales or marketplace pricing, where ignoring counterpart leverage can systematically transfer surplus to one party.
• Negotiation training tools that coach humans, which require models that model an opponent rather than repeating an anchor.
• Multi-agent economic simulations and agentic marketplaces, where many LLM agents interacting could compound one-sided biases.
• Deception and safety monitoring, given that models fabricated BATNA claims in a fraction of turns.

Industry relevance. Any deployment where an LLM represents a company or user in a binding or semi-binding transaction inherits the bias toward ZOPA extremes — GPT-4.1-mini and GPT-4o-mini consistently closed at $225k in this setting, and GPT-4.1-nano captured $235k in four of six context scenarios. The finding that behavior does not improve with stronger models is directly relevant to procurement and vendor-selection decisions.

Future Directions

• Building opponent modeling into negotiation agents. The authors call for models that better internalize opponent reasoning and context-dependent strategy, since current models treat leverage as irrelevant.
• Extending beyond a single scenario and a single model family. The study covers one house-sale setting, 100 self-play negotiations per model, and four OpenAI models; generalization to other domains, item types and model providers is untested here.
• Understanding why capability does not transfer. The observation that negotiation ability does not improve with better models raises the question of whether post-training objectives, rather than base capability, drive the rigid anchoring seen in GPT-o4-mini and others.
• Investigating deception systematically. Fabricated BATNA claims appeared in GPT-o4-mini (7%), GPT-4.1-nano (5%) and humans (3%); a larger study could characterize when and why models misrepresent their alternatives.

Target Audience

Researchers working on LLM agents, multi-agent systems and computational economics will find the metrics framework directly reusable. Behavioral economists and negotiation researchers will value the matched human comparison and the strategy taxonomy. Practitioners deploying LLM-based negotiation or pricing agents, and safety researchers interested in strategic deception in agentic settings, are also well served — though readers should note the study is limited to one negotiation scenario and four OpenAI models.

Authors’ abstract

Bilateral negotiation is a complex, context-sensitive task in which human negotiators dynamically adjust anchors, pacing, and flexibility to exploit power asymmetries and informal cues. We introduce a unified mathematical framework for modeling concession dynamics based on a hyperbolic tangent curve, and propose two metrics burstiness tau and the Concession-Rigidity Index (CRI) to quantify the timing and rigidity of offer trajectories. We conduct a large-scale empirical comparison between human negotiators and four state-of-the-art large language models (LLMs) across natural-language and numeric-offers settings, with and without rich market context, as well as six controlled power-asymmetry scenarios. Our results reveal that, unlike humans who smoothly adapt to situations and infer the opponents position and strategies, LLMs systematically anchor at extremes of the possible agreement zone for negotiations and optimize for fixed points irrespective of leverage or context. Qualitative analysis further shows limited strategy diversity and occasional deceptive tactics used by LLMs. Moreover the ability of LLMs to negotiate does not improve with better models. These findings highlight fundamental limitations in current LLM negotiation capabilities and point to the need for models that better internalize opponent reasoning and context-dependent strategy.