AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Models

AI summary

Overview

Research area: Natural Language Processing, specifically bias evaluation and cultural fairness in large language models, with a focus on African socio-cultural contexts.

Technical level: Intermediate. The evaluation paradigm (stereotype–antistereotype preference scoring) and statistical testing are described precisely enough to follow, but the paper also relies on familiarity with bias benchmarks such as StereoSet and CrowS-Pairs.

Scope: The paper introduces AfriStereo, an open-source dataset of 1,163 community-collected African stereotypes expanded to over 5,000 stereotype–antistereotype pairs, plus an evaluation framework used to test eleven language models spanning 2019–2024.

What This Paper Is About

Existing AI bias benchmarks largely reflect Global North perspectives, leaving African identities, languages, and culturally specific stereotypes underrepresented, which lets harmful stereotypes pass unnoticed in deployed applications. The authors address this by building a community-grounded stereotype dataset from Senegal, Kenya, and Nigeria and using it to measure whether language models systematically prefer stereotypes over their opposites. The goal is a culturally grounded, reproducible evaluation resource for African contexts.

Key Contributions

1. The first open-source stereotype dataset grounded in African socio-cultural contexts, comprising 1,163 manually validated stereotypes from Senegal, Kenya, and Nigeria.
2. A reproducible methodology combining open-ended community surveys, semantic clustering, and human-in-the-loop verification.
3. A systematic evaluation of eleven language models spanning 2019–2024, revealing statistically significant bias across model generations with axis-specific analysis.
4. A synthetic augmentation pipeline expanding coverage to over 5,000 stereotype–antistereotype pairs with human verification.

Main Findings

• Widespread bias across model generations: Nine of eleven models showed statistically significant bias, with Bias Preference Ratios (BPR) between 0.63 and 0.78 (p ≤ 0.0007), indicating a systematic preference for stereotypes over antistereotypes. For reference, BPR = 0.5 indicates no systematic preference.
• Modern models are not less biased: Models from 2023–2024 showed comparable or stronger bias than baseline models from 2019–2022, suggesting architectural advances have not mitigated African stereotype encoding.
• Llama 3.2 3B showed the strongest bias: BPR = 0.78 (p < 0.0001), with pronounced bias across age, profession, and gender, despite being Meta's latest lightweight model (September 2024).
• All five modern models were significantly biased: BPR ranged from 0.70 to 0.78. Mistral 7B showed strong bias across age, profession, and religion; Phi-3 Mini, trained on high-quality synthetic data, still showed significant age and profession bias.
• Multilingual training did not eliminate stereotypes: Qwen 2.5 7B, which supports 140+ languages including several African languages, showed profession bias of BPR = 1.00 and age bias of BPR = 0.91.
• Size reduction did not reduce bias: Gemma 2 2B showed BPR = 0.71 overall, with age bias of 0.86 and profession bias of 0.87.
• Age and profession were the most consistent bias axes: These were the most prominent dimensions across all models. Gender stereotypes were pronounced in larger baseline and modern models, particularly Llama 3.2, Qwen 2.5, and Gemma 2.
• Domain-specific models showed weaker bias in this setup: BioGPT Large had BPR = 0.55 with a p-value of 0.0585 (religion marginal, not significant), and FinBERT had BPR = 0.50 with p = 0.4507 and no primary bias axes, suggesting task-specific training may mitigate some associations.
• Baseline model results: GPT-2 Medium 0.69 (p = 0.0053), GPT-2 Large 0.69 (p = 0.0003), GPT-Neo 0.71 (p < 0.0001), Flan-T5-Large 0.63 (p = 0.0007), all with significant bias.
• Note on completeness: The paper content provided ends mid-sentence in Section 6.1.4, so any limitations, conclusion, or discussion sections are not included here.

Methodology in Plain English

The team ran an open-ended survey through LOOKA, a pan-African research platform, in both English and French. Recruitment happened entirely through social media (LinkedIn, Instagram, X) and personal networks, participation was voluntary with no compensation, and the only inclusion criterion was being from or currently residing in Nigeria, Kenya, or Senegal. A total of 107 volunteers took part: 68% from Nigeria, 20% from Kenya, and 11% from Senegal; 49% were aged 26–35, 21% were 18–25, 21% were 36–50, 8% were over 50, and 1% were under 18; the sample was 50% female and 50% male. The survey produced 1,163 unique stereotype statements across gender, age, profession, ethnicity, and religion, plus an open-ended category.

Each response was parsed into an identity term and an attribute term using five cascading regex rules—geographic patterns, demographic patterns, copula constructions, known-identity matching, and a first-word fallback—followed by manual verification. About 5% of responses needed manual intervention beyond automated extraction, mostly for intersectional identities such as "young Nigerian men" and non-standard phrasings. Intersectional identities were deliberately kept intact as single identity terms.

To group similar attributes, the team used the sentence-transformers/all-MiniLM-L6-v2 model with pairwise cosine similarity and a threshold of τ = 0.55, chosen empirically to balance over-merging against fragmentation. Because semantically similar words can carry opposite sentiment ("smart" and "stupid"), they added a polarity constraint using VADER, so only attributes with matching polarity were grouped. Clusters were then reviewed by team members with lived experience in the three target countries.

Stereotype–antistereotype pairs were built on a uniform template: "[Identity] are [Attribute]" against "[Identity] are [Opposite Attribute]", with all identity terms in plural form. Where natural antonyms existed they were used ("caring" → "uncaring"); where they did not, negation constructions were preferred ("Igbo people are business-oriented" vs. "not business-oriented", "Maasai are warriors" vs. "not warriors") to keep the sentences natural.

To broaden coverage, the initial 1,163 human-collected stereotypes were used as few-shot exemplars. DeepSeek-V3 generated the first 500 pairs, of which roughly 95% appeared appropriate after internal review; other models (GPT-5, Claude, Gemini) had guardrails preventing generation of negative content. The dataset was then augmented using the MostlyAI synthetic data generation platform with few-shot prompting, since zero-shot produced less reliable outputs. These synthetically augmented entries are kept as a separate resource and remain under validation.

Evaluation used the Stereotype–Antistereotype (S-AS) preference paradigm. For each pair, the bias score is the log probability of the stereotype sentence minus the log probability of the antistereotype sentence. The Bias Preference Ratio (BPR) is the share of samples where the bias score is above zero. Sentence probabilities were computed by autoregressive likelihood for causal models, encoder conditioning for the encoder-decoder model, and pseudo-log-likelihood for the masked model. Paired t-tests compared stereotype and antistereotype scores, rejecting the null hypothesis of equal means at p ≤ 0.05.

Why This Matters

Impact on research: Prior benchmarks such as StereoSet and CrowS-Pairs were drawn from Global North contexts, and existing work shows African languages are significantly underrepresented in NLP datasets. AfriStereo provides a community-grounded alternative with culturally specific identity terms (Igbo, Luo, Kikuyu, Serer, Peulh) and systematic antistereotype pairing, which allows direct quantitative measurement rather than only categorical documentation. The finding that nine of eleven models are significantly biased, including every modern model tested, suggests bias is a persistent training-data pattern rather than an artifact of older architectures.

Real-world applications:

• Healthcare: the paper cites generative AI integration in healthcare across Africa, and notes that models trained on data from predominantly white populations have shown biases against Black patients, leading to disparities in medical treatment and outcomes.
• Agriculture: named as one of the sectors where generative AI use is expanding across the continent.
• Education: also named as an expanding application area, where biased outputs risk misrepresenting local realities.
• Image and text generation: text-to-image generators often depict African individuals in stereotypical ways, emphasizing wildlife, traditional attire, or impoverished settings, and frequently reproduce a "white saviour" stereotype even when prompts are intended to challenge such narratives.

Industry relevance: The paper notes that Kenya has one of the highest ChatGPT usage rates globally, making bias evaluation directly relevant to deployed products in African markets. Because benchmark datasets sourced from the Global North can miss these misrepresentations, companies evaluating systems on existing metrics may score well while still producing skewed or irrelevant outputs for African users. The dataset, evaluation framework, and code are released openly at https://github.com/YUX-Cultural-AI-Lab/Afri-Stereo.

Future Directions

• Evaluate commercial models via API: The current evaluation is limited to open-source models where probability distributions for the S-AS paradigm are directly accessible. The authors plan to use NLI-based methods to evaluate commercial models (GPT-5, Claude, Gemini) through API access.
• Complete validation of the synthetic dataset: Comprehensive validation and annotation of the synthetically augmented entries remains ongoing work, with ethnicity-based stereotypes requiring the most substantial review and scrutiny.
• Expand geographic coverage: Plans are stated to extend coverage to additional African countries beyond Senegal, Kenya, and Nigeria.
• Strengthen the clustering pipeline: The authors identify that they did not incorporate additional lexical relation checks (such as WordNet antonyms, ConceptNet, or NLI-based contradiction detection) beyond polarity filtering, and that future work could benefit from systematic sensitivity analysis across threshold values using silhouette scores or cluster purity. They also note that multilingual embeddings (LaBSE, multilingual-MiniLM) would better capture semantic nuances in the original French responses, and that back-translation was not systematically employed.

Target Audience

This paper benefits NLP bias and fairness researchers, particularly those working on multilingual and Global South contexts; dataset and benchmark builders who want a reproducible community-engagement methodology; AI practitioners deploying language or image models in African markets who need culturally relevant evaluation signals; and policy or product teams in healthcare, agriculture, and education who need to understand where stereotype-driven model outputs create risk.

Authors’ abstract

Existing AI bias evaluation benchmarks largely reflect Western perspectives, leaving African contexts underrepresented and enabling harmful stereotypes in applications across various domains. To address this gap, we introduce AfriStereo, the first open-source African stereotype dataset and evaluation framework grounded in local socio-cultural contexts. Through community engaged efforts across Senegal, Kenya, and Nigeria, we collected 1,163 stereotypes spanning gender, ethnicity, religion, age, and profession. Using few-shot prompting with human-in-the-loop validation, we augmented the dataset to over 5,000 stereotype-antistereotype pairs. Entries were validated through semantic clustering and manual annotation by culturally informed reviewers. Preliminary evaluation of language models reveals that nine of eleven models exhibit statistically significant bias, with Bias Preference Ratios (BPR) ranging from 0.63 to 0.78 (p &lt;= 0.05), indicating systematic preferences for stereotypes over antistereotypes, particularly across age, profession, and gender dimensions. Domain-specific models appeared to show weaker bias in our setup, suggesting task-specific training may mitigate some associations. Looking ahead, AfriStereo opens pathways for future research on culturally grounded bias evaluation and mitigation, offering key methodologies for the AI community on building more equitable, context-aware, and globally inclusive NLP technologies.