But How Would AI Agents Run a Town's Economy?

AI summary

Overview

Research area: Multi-agent systems / LLM-agent-based economic simulation, sitting at the intersection of computational social science and agent evaluation.

Technical level: Intermediate. The setup and conclusions are easy to grasp, but the paper leans on matched-seed nonparametric statistics (Wilcoxon signed-rank, Cliff's delta, Holm-Bonferroni correction) and a ledger-audit methodology that assumes some familiarity with experimental design.

Scope: The paper instruments a closed, money-conserving town economy run by 100 memory-equipped LLM agents for up to 26 simulated weeks, and traces exactly where monetary shocks stop propagating.

What This Paper Is About

Most work that puts LLM agents in a simulated society runs for a couple of simulated days or weeks, notes that the agents behave recognizably human, and stops. That leaves an unexamined question for anyone who wants to deploy agentic AI in economic roles: when money enters such a system, does it actually move — from demand to businesses, from businesses to wages, from windfalls to spending, and from one wealth bracket to another over time? This paper builds a fully audited economy of 100 LLM agents on real Pokhara Lakeside, Nepal geography and runs it long enough to find out whether the observed outcome is a stable equilibrium or just an artifact of stopping early.

Key Contributions

1. A validated, released dataset of 91 runs (2.44M agent decisions, 21.5B tokens, 41,328 recorded pulses), gated by two independent pipelines — a live TypeScript validator and an offline Python recomputation that reconciles every agent's balance against its own signed transaction history.
2. An identified mechanism for monetary transmission failure, decomposed exactly into a 1.50× extensive margin (more firms trading) and a 3.07× intensive margin (more revenue per firm), corroborated from the opposite direction by a within-run randomized cash transfer.
3. Direct evidence that study horizon changes the qualitative conclusion about wealth mobility: the distribution looks frozen at the 2-week horizon this literature conventionally uses, but measurably relaxes over 12 and 26 weeks.
4. A causal ranking of the knobs that matter: swapping the decision LLM moves every measured outcome at the significance floor; deleting agents' memory moves none detectably; and social-coordination tooling fails 94–97% across two independent model families.

Main Findings

• Demand reaches the firm and stops there. A 12× tourist arrival shock (delivered cleanly, δ = 1.00, p_adj < 0.001) raises business revenue 4.62×. Wages move 1.03× (p = 0.42) and only 0.3% of 3,981 menu items are ever repriced (p = 0.47). The wage share of revenue collapses monotonically across the tourism sweep: 0.90 (low) → 0.45 (baseline) → 0.20 (high). The chain a textbook tourism economy predicts breaks at the second link.

• The multiplier decomposes exactly, then dies. The 4.62× revenue response factors into 1.50× more firms transacting (458 vs. 304 of 762) and 3.07× more revenue per active firm; 1.504 × 3.067 = 4.61, matching the observed figure to rounding. Neither channel has any mechanical link to wages or prices, and empirically neither moves one.

• Windfalls are saved, not circulated. A cash transfer of NPR 5,000 to 20 of 100 agents, randomized within a single run so treated and control share the same world, leaves 99.8% of the grant held at pulse 84 and 96.7% at pulse 335. Two independent MPC estimates — 3.3% from the wealth trajectory, 4.0% from excess spending — agree and are indistinguishable from zero, against the 0.2–0.5 range typical of human households.

• "Frozen" wealth is a horizon artifact. Wealth-rank persistence is ρ = 0.964 at 2 weeks (9 runs), 0.832 at 12 weeks (4 runs), and 0.752 at 26 weeks (1 run). Quintile mobility rises 21% → 45% → 46% against an 80% chance level. In the 26-week run the Gini rises briefly, falls sharply to ≈0.641 by week 10, and plateaus near 0.643 — a relaxation timescale of roughly ten weeks that no short study can observe.

• Model choice dominates; memory does not. Swapping Qwen3.8-27B for gpt-oss-20b moves every outcome at the p = 0.0039 floor (Gini +5.8%, median wealth −22.9%, quintile mobility +62.4%), effects 3–10× larger than any other ablation. Removing memory tools entirely (verified by zero recorded calls) moves no economic outcome detectably — though with only 6 matched seeds, this is "no detected effect," not equivalence.

• The social layer fails systematically and model-agnostically. invite_to_talk fails 97.2% under Qwen and 94.8% under gpt-oss; accept_invite fails 100% of 1,413 calls; leave_conversation fails 97.7%. These three account for 79.6% of all tool failures, while economic tools succeed about 96% of the time under the same agents, context, and memory. The dominant logged reason is a platform concurrency cap — but agents receive this human-readable error thousands of times with no measurable adaptation away from the tool.

• Money conservation holds exactly. totalMoneyInSystem equals the independently computed sum of every agent's wallet and bank balance at every one of 41,328 pulses, with zero deviation across all 91 runs. 17 of 19 offline checks pass on all 91 runs.

Methodology in Plain English

The researchers built a spatial economy of 100 LLM-controlled agents on the actual street grid of Lakeside, Pokhara, using real building footprints and business locations. The registry holds 762 businesses across seven categories; 735 carry priced menus totaling 3,981 items. Each agent has a cash wallet, a bank balance, an 80-meter visibility radius, persistent memory, and a fixed 19-action toolset covering navigation, economic activity (buying food, setting prices, working shifts, paying other agents), memory operations, and social coordination.

Two clocks drive the simulation. A world clock advances in fixed 60-simulated-minute pulses during which the platform resolves scheduled dynamics and snapshots state. An agent clock is event-driven — agents wake, reason, and act asynchronously within a pulse rather than in lockstep — so a two-week run is a continuous staggered stream of reasoning episodes rather than 336 rounds of simultaneous decisions.

The design is a comparative-statics sweep with matched controls. Five conditions vary tourist arrival rates and scheduled interventions (baseline, tourism-low, tourism-high, tourism-shock stepping low→high→low across pulses 84–143, and wealth-grant). Three additional arms hold the condition fixed and change exactly one factor: a duplicate world layout as a placebo, memory tools removed, and the decision LLM replaced. Two arms extend baseline to 12 and 26 simulated weeks. All seeds are drawn from a single pre-registered pool, which makes matched-seed pairing possible throughout.

Validation is deliberately redundant. A live TypeScript validator checks run integrity at completion, and a separate offline Python validator recomputes every headline number in a second language from raw CSV exports. Its most demanding test is a per-agent ledger reconciliation: the change in wallet-plus-bank balance must equal the signed sum of that agent's own transaction history, which catches export truncation and accounting bugs that aggregate checks would miss. This strict conservation lets the authors decompose the revenue response into extensive and intensive margins exactly rather than approximately, and trace individual transfer rupees over 311 pulses.

The authors disclose several deviations from pre-registration honestly: the planned sweep was 5 conditions × 30 seeds with a pre-committed fallback to 15, but a mid-project shift to self-hosted inference left them at 9 seeds/condition and a longer 336-pulse horizon. They state plainly that 9 seeds is too few to rely on between-seed asymptotics and that every headline comparison is therefore a matched-seed or within-run contrast.

Why This Matters

Impact on research. The paper makes a methodological argument that applies well beyond its own setting: any distributional claim from an LLM-agent society is only as trustworthy as the horizon it was measured over, and a short-horizon "stable" result may be a pure artifact of stopping early. It also shows that an aggregate tool-success rate can conceal near-total failure of an entire action class — here, social coordination — behind a healthy overall number. The released run corpus plus ledger audit is an unusually direct answer to the validation weakness that generative agent-based modeling has been criticized for.

Real-world applications:

• Agentic economic policy simulation. If AI agents are ever used to stress-test tax, transfer, or stimulus designs, a 3–4% marginal propensity to consume rather than the human 20–50% would systematically understate second-round demand effects.
• LLM tool-reliability testing. The finding that a reduced three-tool probe reported 100% success on a failure mode that production traffic shows at 94–97% is a concrete warning for anyone benchmarking agent tool use. Any reliability probe must replay recorded production requests.
• Multi-agent system design. The result that the backing LLM, not memory architecture, governs aggregate outcomes suggests where engineering effort actually pays off in agent deployments.
• Agentic marketplaces and platforms. If agent-run businesses absorb demand shocks as owner profit rather than passing them to workers or customers, that concentration is a structural property of the policy class, not of the underlying model's capability.

Industry relevance. For companies building agentic systems that transact — autonomous procurement, agent-run storefronts, multi-agent negotiation layers — the paper's central warning is that agents may hold resources rather than recirculate them, and that this behavior is robust across model families when it comes to social tooling. The model-swap result also implies that swapping a vendor's foundation model can move economic aggregates by double-digit percentages, which matters for reproducibility and for anyone planning to re-platform.

Future Directions

• Test whether prompting can restore wage pass-through. The authors state explicitly that the transmission-failure claim is falsifiable: if a prompt reminding owners that raising wages is a live option when a shift sells out restored pass-through with nothing else changed, the central claim would fall. They did not test this.
• Replicate the horizon ordering at scale. The 12- and 26-week results rest on 4 runs and a single trajectory respectively. Independent 12- and 26-week replicates that fail to reproduce the persistence ordering would falsify the horizon claim.
• Separate mechanical price stickiness from agent reluctance. set_price is called roughly 18 times per run out of ~28,000 total calls, against a menu that is sticky by construction. The paper does not disentangle rare use of an available lever from structural stickiness.
• Broaden the environment. One town, one seed pool, one serving stack, no governance layer. Mistral-Small-3.2-24B could not complete a single pulse due to duplicate tool-call identifiers — a stack-specific result that raises the question of how much of the cross-model comparison generalizes beyond self-hosted vLLM.

Target Audience

Researchers in multi-agent systems and generative agent-based modeling will find the most value here, particularly those who build agent societies and report emergent behavior. It is equally relevant to computational economists and social scientists evaluating LLM agents as substitutes for human subjects in market and policy experiments, and to AI evaluation researchers interested in tool-reliability and horizon effects. Practitioners designing agentic systems that transact or coordinate — autonomous commerce, negotiation, multi-agent orchestration — should read the tool-failure and model-swap sections closely; the finding that a healthy aggregate success rate masks near-total social-tool failure is directly actionable for anyone benchmarking agent platforms.

Authors’ abstract

We placed 100 memory-equipped large language model (LLM) agents in charge of a closed, money-conserving spatial economy on real Pokhara Lakeside geography (earning wages, running businesses, setting prices) and ran this multi-agent simulation for up to 26 simulated weeks, well past the 1-2 weeks typical of agent-society studies. Across 91 validated runs (2.44M agent decisions, 21.5B tokens), the money stops moving, in a specific and measurable way. A 12x tourist demand shock raises business revenue 4.62x ($p&lt;0.001$), which we decompose exactly into a 1.50x extensive margin (more businesses trading) and a 3.07x intensive margin (more revenue each). Monetary transmission stops there. Wages move 1.03x ($p=0.42$); 0.3% of 3,981 menu items are ever repriced ($p=0.47$). A randomized cash transfer (NPR 5,000 to 20 of 100 agents) shows the same pattern from the opposite direction: 96.7% is still held 311 pulses later, marginal propensity to consume 3-4% by two independent measures, indistinguishable from zero. The wealth distribution is consequently near-frozen at the horizon this literature uses ($ρ=0.964$ over 2 simulated weeks), but not frozen. $ρ$ falls to 0.832 at 12 weeks and 0.752 at 26, a horizon-dependence no short study can see. Matched ablations show which knob actually matters. Swapping the backing LLM moves every outcome we measure ($p=0.0039$); deleting agents' memory moves none of them detectably. A purely social tool fails 94-97% of the time across two model families, compared with ~96% success on economic tools, with no measurable shift away from it. Every headline number is verified twice, by a live validator and by an offline recomputation that reconciles each agent's wealth against its own signed transaction history, and we release the full run corpus for reanalysis.