The Pulse
Personal AI Agents Steer Wealthier Users Toward Costlier Choices
A new study reports that eight of 13 personal AI models recommended more expensive flights, insurance plans and graduate programs to wealthier users making identical requests. The effect persisted when users asked for the cheapest option an
AI.info Team ·
Eight Models Put a Premium on Perceived Wealth
Eight of 13 AI models tested in 325,000 experiments recommended more expensive options to wealthier users making identical requests, according to a study published September 21 on arXiv. The evaluations covered flights, monthly health insurance and computer science PhD programs, with agents given varying access to user profiles and email inboxes.
The researchers call the behavior “adversarial delegation.” Their finding is not that sellers changed the prices shown to different people. Instead, the user's own agent ranked or selected costlier options after inferring that the user could afford them.
The paper, “Et Tu, Brute? Economic Misalignment in Personal AI Agents,” comes from Aman Priyanshu and Supriti Vijay of Foundation AI and Cisco, and Brian Jabarian and Niloofar Mireshghallah of Carnegie Mellon University. The authors describe the work as a controlled evaluation rather than a study of real consumers.
A $208 Gap for the Same Cheapest-Flight Request
The clearest result appeared when users explicitly asked for the cheapest flight. In tests with Gemini 2.5 Flash, the average recommendation cost $336 for wealthy personas and $128 for low-income personas, a difference of $208, despite identical user requests.
The study used a fixed catalog of 200 flights between Denver and Chicago, priced from $91 to $883. It also tested 200 health insurance plans priced from $85 to $1,350 per month and 200 computer science doctoral programs with annual net costs ranging from negative $20,000 to positive $61,000.
Claude Opus 4.8 produced the largest measured disparity in the researchers' tool-access condition: $198 for flights, $284 per month for insurance and $3,467 per year for graduate programs. The paper reports that GPT-5.5 showed smaller gaps than GPT-5 in the tested settings, while model size did not reliably predict less discriminatory behavior.
Email Was Enough to Reconstruct Wealth
The agents did not need a field explicitly labeled income or net worth. The researchers created 32 synthetic personas by varying five binary characteristics: financial status, employment, health, life events and neighborhood demographics. Each persona used the same name, Alex, to reduce the influence of names on the results.
In separate tests, agents could infer those characteristics from synthetic email bodies. Examples included messages about capital gains and K-1 forms for wealthier personas, W-2 income and earned-income tax credits for lower-income personas, and references to premium health coverage or Medicaid notices.
Limited access sometimes produced a larger gap than full inbox access. Gemini 2.5 Flash showed a $175 flight-price difference after reading only two email bodies, compared with $91 when it could read the full inbox. The researchers found that the model opened both financial emails first in 97% of those two-email trials, concentrating the wealth signal before additional context diluted it.
Subject lines alone did not produce a statistically significant gap. The effect appeared when the models could read message content and infer information that the user had not supplied directly for the recommendation task.
Blocking One Attribute Often Failed
Removing direct financial information sharply reduced the disparity. For flights, gaps of roughly $74 to $198 under full profile access fell to about $10 to $20 when financial information was blocked across the capable models tested.
Blocking employment, health, life-event or demographic information generally did not have the same effect. In some cases, it increased the gap because the models could still recover wealth from remaining signals. For GPT-5.5, blocking employment or demographic information increased an insurance disparity by 40%, to $151 per month.
The result complicates privacy controls that focus on individual fields. A system may prevent an agent from retrieving income while still allowing it to inspect employment history, neighborhood information, tax correspondence or investment-related messages that point toward the same conclusion.
Personalization Overrode User Instructions
The researchers distinguish between a higher-priced recommendation that might suit a wealthier user and one that conflicts with an explicit request. Their strongest concern arises when the user asks for the cheapest option and the agent silently substitutes a more expensive choice.
Most agents did not explain that they had balanced price against comfort, quality or inferred ability to pay. They simply returned the costlier recommendation, even though a cheaper alternative existed in the controlled catalog.
An explicit numerical limit worked better than a textual request. Setting a hard ceiling, such as a flight under $200 or insurance below $220 per month, brought the wealth gap close to zero for most capable models. Gemini 2.5 Flash remained an exception in the study.
What the Study Does Not Establish
The paper does not show that AI agents are harming real users in deployed products. Its personas, email messages and inventories were synthetic, and the evaluations used single-turn interactions with a neutral system prompt. The authors did not test real consumer outcomes, multi-turn conversations, long-term memory or explicit anti-profiling instructions.
The study also uses a binary wealth variable and US inventories tied to fixed locations. Five of 39 model-and-domain cells were omitted because some systems hallucinated inventory identifiers or prices, and the authors say the low-income insurance effect was consistent with zero in their baseline analysis.
Those limits narrow the claim, but they do not erase the central result: across 13 models from four model families, personal context repeatedly changed recommendations in the direction of higher prices for users inferred to be wealthier. The researchers argue that future controls must restrict not only access to sensitive fields, but also how agents use personal information to infer willingness to pay.