Who Gets Which Message? Auditing Demographic Bias in LLM-Generated Targeted Text

AI summary

Overview

Research area: Natural Language Processing — fairness, bias auditing, and controlled text generation with large language models.

Technical level: Intermediate. The paper combines descriptive statistics (odds ratios, t-tests, ANOVA), a composite linguistic metric, and off-the-shelf classifiers, but no model training or architectural work is involved.

Scope: A single-author study (Tunazzina Islam, Purdue University) that introduces a controlled framework for measuring how gender- and age-conditioned prompts shift the content, style, and persuasive framing of LLM-generated climate messages.

What This Paper Is About

When LLMs are told to write a persuasive message "for men" versus "for women," or "for a young adult" versus "for a senior," do they say different things — and do they say them with different force? The paper argues that demographic-conditioned persuasive generation is a distinct evaluation problem that existing bias audits (which mostly rely on sentiment, toxicity, or lexical polarity) do not capture, because persuasion depends on agency, certainty, and directive intent rather than emotional tone alone. The goal is to build a model-agnostic auditing framework that separates a model's intrinsic demographic associations from the biases that get amplified when thematic and regional context is added.

Key Contributions

1. A controlled two-setting evaluation framework for demographic-conditioned persuasive text generation, explicitly distinguishing Standalone Generation (SG) — demographic cues only — from Context-Rich Generation (CRG) — demographic cues plus theme and U.S. region. This design is intended to disentangle intrinsic demographic effects from context-amplified behavior.
2. The Persuasion Bias Index (PBI), a theoretically grounded composite metric that combines agency framing (high- versus low-agency verbs from the Connotation Frames lexicon), modal certainty (certainty markers versus hedges), and imperative usage, grounded in prior work on agency and connotation (Abele and Wojciszke, 2018; Sap et al., 2017).
3. An empirical audit across three LLMs — GPT-4o, Llama-3.3, and Mistral-Large-2.1 — on climate-related targeted messaging, measuring lexical content bias, language style bias, and persuasion bias across gender and four age groups.
4. Released materials, available at a public GitHub repository (llms-bias-audit-microtarget-climate).

Main Findings

• Male-targeted SG messages lean agentic, masculine, and leadership-oriented. In the Standalone Generation setting, odds ratios for the "Agentic" lexicon category were 4.03 for GPT-4o, 1.44 for Llama-3.3, and 4.20 for Mistral-2.1; "Masculine" was 2.01 / 1.39 / 1.14 and "Leadership" was 1.70 / 1.39 / 1.14. "Personal" (0.44 / 0.40 / 0.43) and "Feminine" (1.00 / 0.82 / 0.44) were generally more salient in female-targeted outputs.
• Older targets are strongly associated with warmth, and this is the most consistent age effect. Warmth-related terms were markedly overrepresented for older groups, with Seniors showing the highest odds ratios: 6.27 for GPT-4o, 6.31 for Llama-3.3, and 3.41 for Mistral. The paper states that competence representations were less stable and varied by model.
• CRG surfaces gender-stereotypical word choice and measurable embedding associations. Top salient words included "smart," "positive," "iconic," "effective," and "adventure" for males and "warm," "amazing," "beautiful," and "friendly" for females. WEAT scores with career/family words and power/support words were positive for all models: GPT-4o 0.077 (CF) and 0.062 (PS) on nouns, 0.004 and 0.079 on adjectives; Llama-3.3 0.184 and 0.048 on nouns, 0.004 and 0.020 on adjectives; Mistral-2.1 0.218 and 0.070 on nouns, 0.165 and 0.037 on adjectives.
• Formality differences by gender reach significance in only one model. Female-targeted texts were scored as more formal than male-targeted texts, with t = −1.787 (p = 0.075) for GPT-4o, t = −3.234 (p = 0.001) for Llama-3.3, and t = −0.819 (p = 0.413) for Mistral-2.1.
• Male-targeted messages show significantly higher modal certainty in every model. Agency (A), modal certainty (M), imperatives (I), and PBI for female versus male were: GPT-4o — A 0.4228 vs 0.6002 (t = −2.02, p = 0.0440), M −0.1864 vs −0.0636 (t = −3.65, p = 0.0003), I 2.3455 vs 2.1955 (t = 1.69, p = 0.0920), PBI 0.3345 vs 0.5624 (t = −3.15, p = 0.0017); Llama-3.3 — A 0.6562 vs 0.5000 (t = 0.83, p = 0.4098), M −0.3182 vs −0.1773 (t = −3.04, p = 0.0025), I 2.1682 vs 2.1000 (t = 0.82, p = 0.4151), PBI −0.0059 vs 0.1191 (t = −2.12, p = 0.0342); Mistral-2.1 — A 0.3360 vs 0.2982 (t = 0.33, p = 0.7409), M −0.2591 vs −0.1364 (t = −2.94, p = 0.0035), I 2.3045 vs 2.1045 (t = 2.13, p = 0.0334), PBI 0.1653 vs 0.2286 (t = −0.83, p = 0.4092).
• Context amplifies the disparities. The paper reports that contextual prompts systematically amplify gender and age differences, with persuasion scores higher for male-targeted messages, while age-related differences vary across models.
• Agency and imperatives are less consistent than certainty. The authors report that demographic conditioning primarily affects certainty-based persuasion rather than uniformly shifting all persuasive mechanisms.
• PBI is not just sentiment. Correlations between PBI and VADER sentiment scores were consistently small across all models, which the authors present as evidence that PBI captures rhetorical and agentic framing beyond affective polarity.
• Theme-specific emotional framing differs by gender. Using the GoEmotions classifier (27 fine-grained emotion classes plus neutral), male-targeted messages more often emphasized policy approval and female-targeted messages more often expressed emotional or caring language, in the Future Generation and Support Climate Policy themes.

Methodology in Plain English

The study fixes a topic (climate change) and a stance (pro-energy or clean-energy) and then varies only the demographic target of the message, so that any differences in the output can be attributed to the demographic cue rather than the subject matter.

Two prompting conditions are used. In Standalone Generation, the prompt supplies only gender and age descriptors, which isolates the model's latent associations. In Context-Rich Generation, the prompt adds a U.S. region and a thematic frame — pro-energy themes are Economy, Climate Solution, Pragmatism, Patriotism, and Against climate policy; clean-energy themes are Economy, Future Generation, Environmental, Human health, Animals, and Support climate policy. Iterating all combinations produced 48 messages for SG (16 per model) and 1,320 messages for CRG (440 per model).

Because the SG sample is small, the authors explicitly restrict SG analysis to lexical content bias only, stating that 16 messages per model per demographic group makes t-tests and odds ratios unstable. For lexical content they compute smoothed odds ratios (with an additive smoothing constant s = 1) over curated lexicon categories rather than single words: 9 gender-stereotypical categories (Ability, Standout, Leadership, Masculine, Feminine, Agentic, Communal, Professional, Personal) and 12 age-related categories (Competence, Incompetence, Warmth, Coldness, Independence, Dependence, Progressive, Traditional, Energy, Frailty, Opportunity, Risk). For age, which is four-way rather than binary, each focal group is contrasted against all remaining groups pooled together, so Young Adult is 18–24, Early Working 25–44, Late Working 45–64, and Senior 65+.

For style, an off-the-shelf formality classifier fine-tuned on Grammarly's Yahoo Answers Formality Corpus scores each message; gender differences are tested with Welch's t-test and age differences with one-way ANOVA plus Tukey HSD post-hoc comparisons. For persuasion, each message gets an agency score (high- minus low-agency verbs, divided by their sum), a modal certainty score (certainty markers such as "will" and "must" versus hedges such as "might" and "could"), and an imperative count scaled by λ = 0.1. These three are summed equally into the PBI, and group-level PBI values are compared with t-tests and ANOVA.

Why This Matters

Impact on research. The paper reframes demographic-conditioned persuasive generation as an auditable behavior rather than a generic generation setting, and argues it is the first empirical study to jointly measure demographic bias and persuasive bias in LLM-generated targeted messages. It offers a reusable metric (PBI) and a two-condition design that separates intrinsic bias from context-amplified bias — a distinction prior bias audits largely do not make.

Real-world applications:

• Political and campaign microtargeting, where fine-grained tailoring to age, gender, region, and theme could produce systematically different persuasive force for different segments.
• Public-interest and policy outreach on climate, where messages intended to inform may instead reproduce warmth-versus-agency stereotypes about older and female audiences.
• Marketing and customer personalization pipelines that condition generated copy on demographic attributes.
• Platform and regulatory auditing, where the framework offers a way to check whether a deployed system amplifies stereotypes under personalization.

Industry relevance. The paper argues directly for bias-aware generation pipelines and transparent auditing frameworks that explicitly account for demographic conditioning, and warns that personalization can magnify latent demographic stereotypes rather than merely reflecting them. The finding that contextual prompts amplify disparities is directly relevant to any production system that combines demographic targeting with thematic context.

Future Directions

• Extend beyond the climate case study. The authors state their framework is domain-agnostic; testing it in political, health, or commercial communication would show whether the same asymmetry patterns hold.
• Overcome the SG sample-size limitation. The 16-messages-per-model-per-group SG setup forced the authors to skip style and persuasion analysis there; larger SG sampling would allow the PBI to be tested under minimal demographic conditioning.
• Explain the variation in age-related effects. The paper notes that age-related differences vary across models and that competence representations are less stable than warmth ones, but does not identify why models diverge.
• Move from auditing to mitigation. The results motivate bias-aware generation pipelines, but the paper does not report or evaluate any debiasing or intervention method.

Target Audience

NLP and AI fairness researchers studying bias in generation; computational social scientists working on microtargeting and political communication; practitioners building personalization or targeted-outreach systems with LLMs; and policy or trust-and-safety audiences who need a concrete, statistically grounded protocol for auditing demographic conditioning.

Authors’ abstract

Large language models (LLMs) are increasingly capable of generating personalized, persuasive text at scale, raising new questions about bias and fairness in automated communication. This paper presents the first systematic analysis of how LLMs behave when tasked with demographic-conditioned targeted messaging. We introduce a controlled evaluation framework using three leading models: GPT-4o, Llama-3.3, and Mistral-Large-2.1, across two generation settings: Standalone Generation, which isolates intrinsic demographic effects, and Context-Rich Generation, which incorporates thematic and regional context to emulate realistic targeting. We evaluate generated messages along three dimensions: lexical content, language style, and persuasive framing. We instantiate this framework on climate communication and find consistent age- and gender-based asymmetries across models: male- and youth-targeted messages tend to emphasize more assertive and progressive framing, while female- and senior-targeted messages more often reflect warmth, care, and traditional themes. Contextual prompts systematically amplify these disparities, with persuasion scores being higher for male-targeted messages, while age-related differences vary across models. Our findings demonstrate how demographic stereotypes can surface and intensify in LLM-generated targeted communication, underscoring the need for bias-aware generation pipelines and transparent auditing frameworks that explicitly account for demographic conditioning in socially sensitive applications.