Linguistic Triggers of Gender and Racial Bias in Open-Weight LLMs Applied to Recruitment

AI summary

Overview

Research area: AI fairness auditing and natural language processing, specifically bias measurement in open-weight large language models deployed in hiring workflows.

Technical level: Intermediate. The statistical machinery (Mann–Whitney U, rank-biserial effect sizes, Bonferroni correction, WEAT) is standard, but the experimental logic and regulatory framing are accessible to non-specialists.

Scope: A six-model, four-experiment audit showing that the linguistic style of a job posting — not only the demographic label attached to a candidate — causally triggers gender and racial bias in LLMs acting as both recruiters and job seekers, translated into a concrete pre-deployment audit protocol.

What This Paper Is About

Most prior audits of bias in AI hiring hold the job posting constant and vary the candidate's name, race, or gender. This paper inverts that design: it holds candidate qualifications constant and varies the language of the job advertisement itself — agentic versus communal wording for gender, and inclusive versus culturally coded exclusionary wording for race. The goal is to establish whether LLM hiring bias is triggered by the linguistic surface of a posting, and if so, to convert that finding into an auditable, regulation-aligned protocol before any ad goes live.

Key Contributions

1. Posting language as the primary experimental variable. The first multi-model audit to systematically manipulate job-ad vocabulary across both a gender dimension (agentic vs. communal) and a racial dimension (inclusive vs. coded-exclusion) within one unified framework.

2. Dual-perspective design. Each experiment is run twice — once with the LLM cast as a recruiter scoring a candidate against a posting (outbound job-matching), and once with the LLM cast as a job seeker rating its own interest. This reveals that the same linguistic trigger affects evaluators and applicants asymmetrically.

3. Triangulation via ablation and embedding analysis. A stimulus-component ablation separates job title from description body, a label-ablation experiment isolates the explicit demographic persona label as the causal driver of race-group differences, and Word Embedding Association Tests corroborate the behavioral results at the representational level.

4. A deployable audit protocol. The authors operationalize their findings into a pre-deployment procedure — posting-vocabulary scoring, persona-conditioned LLM probing, and adverse-impact flagging against the EEOC four-fifths threshold — mapped onto EU AI Act Annex III high-risk documentation obligations.

Main Findings

• Agentic language penalizes female candidates under the recruiter framing. Pooled across six models, female candidates scored 6.72 on agentic postings versus 7.53 on communal postings (rank-biserial rrb = 0.309, pBonf = 7.2 × 10⁻⁵). Male candidates showed only a small, non-significant shift. A model-fixed-effects replication strengthened the female effect (rrb = 0.448, pBonf = 3.9 × 10⁻⁹), ruling out cross-model score calibration as an explanation.

• The description body, not the job title, carries the effect. When only the description was shown, the female penalty persisted (rrb = 0.272, pBonf = 5.4 × 10⁻⁴). When only the title was shown, neither gender effect reached significance (pBonf > 0.92).

• The gender effect disappears under the job-seeker framing. Female personas rated agentic (8.35) and communal (8.47) postings almost identically (pBonf = 0.623). Stereotype associations appear more strongly activated when evaluating an external candidate than when adopting a self-referential persona.

• Coded-exclusion language severely suppresses non-White recruiter scores. Black candidates saw the largest drop (8.45 → 6.25, rrb = 0.742), followed by Hispanic (Δ = −1.97, rrb = 0.758) and Asian American (Δ = −1.57, rrb = 0.646) candidates. White candidates also declined but far less (Δ = −1.00, rrb = 0.324). Under model-fixed effects, non-White effect sizes exceeded 0.77 in every case.

• Inclusive language produces "prosocial overcorrection." Under inclusive postings, White candidates received the lowest mean recruiter score of any group (7.16), trailing Black (8.45), Hispanic (8.38), and Asian American (8.26) candidates by 1.10–1.29 points — a gap larger in magnitude than the entire White coded-exclusion decline. The authors argue this is still a departure from merit-neutral evaluation and is implicated by adverse-impact frameworks regardless of which group it favors.

• Coded-exclusion language selectively chills minority job-seeker interest. Non-White personas dropped 1.08–1.60 points versus 0.68 for White personas, operationalizing a chilling-effect mechanism at scale and mirroring the recruiter-side differential.

• The explicit demographic label is the primary causal driver. With race set to "not specified," inclusive versus coded-exclusion postings produced a uniform gap of −0.73 with no differential pattern. Comparing labeled groups against this baseline isolates label-attributable penalties of −0.35 (Asian American), −0.52 (Hispanic American), and −0.87 (Black or African American), with White candidates falling within the baseline.

• Embedding-level validation confirms the representational structure. WEAT analyses across two embedding models and both title and description variants yielded large effects (d = 1.01–1.45, all pperm ≤ 1.5 × 10⁻³), with agentic stimuli consistently closer to male attribute sets than communal stimuli in every panel.

Methodology in Plain English

The researchers selected six open-weight models small enough to run on consumer hardware — Llama 3.2 (3B), Mistral 7B v0.3, Gemma 3 (4B), Qwen 3 (8B), Phi 3 Mini (3.8B), and DeepSeek-R1 (7B distill). All were queried at temperature 0 with responses constrained to a fixed JSON schema, so outputs were deterministic and directly comparable.

For the gender experiments, they built 40 synthetic job postings, half using dominance- and achievement-oriented vocabulary (e.g., "competitive," "decisive") and half using relationship- and cooperation-oriented vocabulary (e.g., "collaborative," "supportive"), drawing on Gaucher et al.'s established word lists. For the race experiments, they built 20 matched posting pairs sharing identical job titles but differing descriptions: one version used equity-oriented phrasing, the other used cues of "cultural fit," polish, and homogeneity adapted from Rivera's work on elite hiring.

In the recruiter framing, the model was told it was a professional recruiter evaluating how well a posting suited a candidate and returned a recommendation score from 1 to 10. In the job-seeker framing, the model was given a demographic identity and rated its own interest in the posting. Candidate qualifications were identical in every condition; only gender or race/ethnicity varied. Each cell contained 20 observations, yielding 120 per pooled group across six models.

Comparisons used two-sided Mann–Whitney U tests with Bonferroni correction within each experiment's comparison family, with rank-biserial correlation as the effect size. A model-fixed-effects analysis mean-centered each score within its model before pooling to address cross-model calibration differences. Two follow-up ablations — one withholding the job title, one withholding the demographic label — isolated which stimulus component drove the observed effects, and WEAT provided embedding-level corroboration.

Why This Matters

Impact on research. The paper relocates the locus of audit from candidate attributes to the linguistic environment of the posting itself, opening a research direction that previous audits left unexamined. The "prosocial overcorrection" finding is particularly notable: it shows that debiasing language does not produce neutral evaluation but shifts the disparity rather than eliminating it, complicating the assumption that inclusive phrasing is a safe default. The chilling-effect operationalization gives researchers a scalable, pre-publication method for measuring applicant-side deterrents that would otherwise require costly field experiments.

Real-world applications:

• Job boards and ATS platforms can score posting vocabulary before publication and flag high-risk phrasing without running a live applicant pool.
• HR technology vendors can use persona-conditioned probing as a model-selection criterion when choosing which open-weight LLM to embed in a matching or recommendation stack.
• Compliance and legal teams gain a documented, replicable audit trail that maps onto EU AI Act Annex III risk-management evidence and EEOC four-fifths rule selection-rate computation.
• Staffing agencies operating outbound "recommended jobs" feeds can detect disparities at exactly the pipeline stage where those systems generate decisions.

Industry relevance. The models tested are open-weight and runnable on consumer hardware, meaning the deployment barrier is low and the risk is widely distributed across small vendors, not just large platforms. The dual US/EU framing also reflects a practical reality: multinational employers must satisfy both regimes, and the paper argues the auditing methodology must be portable even though the protected categories are not interchangeable.

Future Directions

• Intersectional analysis. The current design tests gender and race separately. Prior work (An et al. 2025) found that intersectional biases resist standard safety training, and the "prosocial overcorrection" pattern here suggests gender–race interactions may produce non-additive effects worth isolating.

• Human-in-the-loop validation. Wilson et al. found participants followed a biased AI's recommendations up to 90% of the time. Whether recruiters would act on the posting-language effects measured here — and whether the chilling effect translates to real applicant dropout — requires behavioral follow-up rather than simulation.

• Mitigation and intervention testing. The paper documents bias but does not test remedies. Whether prompt-level interventions, retrieval augmentation, or fine-tuning can suppress the description-body effect without simply relocating the disparity is an open and consequential question.

• Multilingual and cross-jurisdictional extension. The stimuli are English-language and the regulatory framing is US/EU. Whether agentic–communal and coded-exclusion registers carry equivalent force in other languages and hiring cultures is untested, and directly relevant to the portability claim the authors make.

• Closing versus outbound matching. The study models the recommendation stage. Whether the same posting-language triggers operate in inbound screening, shortlisting, and rejection decisions — where liability exposure differs — remains unexplored.

Target Audience

AI fairness researchers and NLP practitioners auditing deployed language models; HR technology vendors and compliance officers responsible for EU AI Act or EEOC obligations; employment lawyers and regulators seeking a replicable methodology for adverse-impact evidence; and recruitment platform engineers selecting or evaluating open-weight models for job-matching systems. Readers without a statistics background can follow the experimental logic and protocol sections, while the effect-size tables reward quantitative readers.

Authors’ abstract

Open-weight large language models are rapidly entering hiring pipelines, yet their discriminatory failure modes -- and the regulatory exposure these create under the EU AI Act high-risk classification (Annex III) and U.S. EEOC adverse-impact analysis -- remain poorly understood. We present the first systematic, multi-model audit of open-weight LLMs that treats job-posting language as the primary experimental variable, evaluating six models (Llama 3.2, Mistral, Gemma 3, Qwen 3, Phi 3, DeepSeek-R1) across four controlled experiments that jointly probe recruiter-simulation and job-seeker-simulation tasks. We find that (1) agentic posting language depresses recruiter recommendation scores for female candidates (r_rb = 0.309, p_Bonf = 7x10^-5; model-fixed-effects r_rb = 0.448), while communal language partially reverses the penalty; and (2) coded-exclusion language suppresses non-White recruiter scores at large effect sizes (r_rb = 0.646-0.758) and, on the job-seeker side, selectively deters non-White personas from expressing interest -- operationalizing a chilling-effect mechanism at scale. A label-ablation experiment isolates the explicit demographic persona label as the primary causal driver, and Word Embedding Association Tests corroborate these findings at the representational level (d = 1.01-1.45 under Caliskan et al.'s multi-word gender attribute lists). We translate these results into a concrete pre-deployment audit protocol -- posting-vocabulary scoring, persona-conditioned LLM probing, and adverse-impact flagging against the four-fifths threshold -- that operationalizes the documentation and risk-management obligations Annex III imposes on high-risk AI in recruitment.