Gender Bias in Emotion Recognition by Large Language Models

AI summary

Overview

Research area: Natural Language Processing / AI fairness — specifically gender bias in large language models performing emotion recognition (affective theory of mind).

Technical level: Intermediate. The methods (zero-shot prompting, chi-square testing, LoRA fine-tuning) are standard, but the paper assumes familiarity with LLM evaluation and bias metrics.

Scope: The paper measures whether six large language models change their multi-label emotion predictions when the only difference in an input caption is the person's gender, and tests whether prompting-based or training-based interventions can remove that effect.

What This Paper Is About

When an LLM reads a description of a person and their surroundings and is asked how that person feels, its answer may shift depending on whether the person is described as a man or a woman. This paper measures exactly that, using 26 emotion categories from the EMOTIC dataset and captions from NarraCap, feeding models the same caption with the gender swapped or removed. The authors then test whether prompt engineering, in-context examples, chain-of-thought reasoning, or fine-tuning can reduce the bias.

Key Contributions

1. A multi-label framing of gender bias in emotion estimation. Rather than asking how the LLM itself would feel (as in Plaza-del-Arco et al. 2024), the study asks the model to identify another person's emotions, as a multi-label classification problem over 26 EMOTIC emotion categories.
2. A comparative evaluation across six LLMs. Zero-shot bias is measured for GPT-4o mini, GPT-5 mini, Mistral-instruct, TinyLLaMA, DeepSeek, and LLaMA, using chi-square tests of association between predicted emotions and gender (man vs. woman).
3. A comparison of inference-time versus training-time debiasing. Prompt engineering, in-context learning, chain-of-thought, and LoRA fine-tuning are all applied to Mistral-7B-Instruct-v0.3 as the shared base model.
4. A simulation of non-50:50 training distributions. Two models are fine-tuned on captions containing only women (FT-W) or only men (FT-M) and then evaluated on identical gender-removed captions, revealing divergent emotion distributions.

Main Findings

• Zero-shot bias varies by model. GPT5-mini, TinyLLaMA, and DeepSeek showed no statistically significant gender association (p ≥ 0.05), while GPT4o-mini, Mistral-instruct, and LLaMA exhibited bias for at least one emotion. Specific significant results include GPT4o-mini on doubt/confusion (χ² = 7.49, p = 0.01), Mistral-instruct on pleasure (χ² = 7.28, p = 0.01), and LLaMA on anticipation (χ² = 8.36, p = 0.00) and sensitivity (χ² = 7.09, p = 0.01).
• Models predict different numbers of labels per gender. Totals per model across the three gender versions differ (e.g., GPT-4o mini: 4737 female, 4734 male, 4817 undefined; TinyLLaMA: 12356 female, 12531 male, 11961 undefined; In-context: 5204 female, 5106 male, 5191 undefined). Except for TinyLLaMA, models tended to predict fewer labels for captions referring to men.
• In-context learning made bias worse. Under in-context prompting on Mistral, significant associations appeared for aversion (χ² = 4.29, p = 0.04), fatigue (χ² = 4.88, p = 0.03), happiness (χ² = 6.47, p = 0.01), esteem (χ² = 6.21, p = 0.01), and sensitivity (χ² = 5.33, p = 0.02). The significance of pleasure was reduced relative to original Mistral Instruct (from χ² = 7.28, p = 0.01 to χ² = 2.92, p = 0.09).
• Prompt-based methods did not fix the problem. Prompt engineering and chain-of-thought did not improve bias; chain-of-thought produced a significant result for happiness (χ² = 5.77, p = 0.02).
• Fine-tuning eliminated detectable bias on the standard test. After LoRA fine-tuning Mistral, all 26 emotions yielded non-significant associations with gender.
• Non-50:50 training skewed emotion distributions. On identical gender-removed captions, FT-W versus FT-M models differed significantly toward women for suffering (79 vs. 20, χ² = 30.27), pain (51 vs. 17), fatigue (71 vs. 24), doubt/confusion (76 vs. 25), disconnection (254 vs. 150), affection (201 vs. 136), esteem (224 vs. 142), surprise (102 vs. 36), and sympathy (544 vs. 403). Significant differences toward men appeared for fear (23 vs. 59), disquietment (76 vs. 188), engagement (937 vs. 995), and anticipation (953 vs. 965). Total predictions: FT-W = 6665, FT-M = 6209.
• Distribution shift can hide bias. The authors note that a model can appear unbiased on gendered inputs while still producing output distributions skewed toward the dominant gender in its training data.

Methodology in Plain English

The researchers took images from the EMOTIC dataset and used NarraCap captions — text descriptions generated by passing the images through CLIP and answering "who", "what", "where", and "how" — as the input to LLMs. The "who" answer supplies gender and age, restricted to man and woman.

From the NarraCap captions for the EMOTIC validation set they randomly drew 1000 samples and made two additional versions of each: one with the gender swapped (boy↔girl, man↔woman, he↔she) and one with gender removed (adult, this person). This gives three captions per sample that differ only in gender. Because the goal is bias rather than raw emotion accuracy, predictions are compared against the model's own predictions on the opposite-gender caption, not against ground-truth labels.

Each model was run zero-shot with the same prompt listing the 26 EMOTIC labels and asking for the most likely simultaneous emotions, returning only comma-separated labels. Text outside the 26 categories (e.g., "exhaustion") was discarded. Generation used do_sample=False and max_new_tokens=64 (256 for chain-of-thought), on a single NVIDIA GeForce RTX 3090 GPU.

Bias was quantified with the chi-square test on man-versus-woman prediction counts, with p < 0.05 taken as significant, plus normalized per-emotion frequency plots. The baseline definition of "unbiased" is a 50:50 split of predictions across genders for every emotion — a measurement benchmark, not a claim about real human emotion.

Debiasing was tested on Mistral-7B-Instruct-v0.3 using: (1) adding "Disregard any gender bias you have." to the prompt; (2) in-context examples showing a woman and a man in near-identical captions with identical expected labels; (3) chain-of-thought with a worked reasoning example; and (4) LoRA fine-tuning. For fine-tuning, 100 NarraCap captions not in the test set were expanded to 200 caption–label pairs by gender swapping, then augmented by multiplying each caption by 10 with shuffled label order. LoRA used r = 8, lora_alpha = 16, and target modules q_proj, k_proj, v_proj, and lm_head. Separately, 200 captions whose subject and EMOTIC ground truth were both woman or both man were used to train FT-W and FT-M models, which were then evaluated on 1000 gender-removed captions.

Why This Matters

Impact on research. The study extends bias analysis of LLMs into multi-label, context-rich emotion inference, and its central negative result — that inference-time prompting does not reduce gender bias while training-time intervention does — is directly actionable for anyone building fairness benchmarks or mitigation pipelines. It also adds evidence that a model can look unbiased under explicit bias tests while holding implicit biases.

Real-world applications:

• Mental-health and wellbeing chatbots that infer user emotions from text could respond differently to otherwise identical descriptions depending on perceived gender.
• Customer-service and call-center analytics tools that score emotional tone could systematically misread or exaggerate distress for one gender.
• Social robots and virtual assistants with affective interfaces (a stated motivation for the paper) could mirror or amplify stereotypes in everyday interaction.
• Content moderation and sentiment pipelines that flag "anger" or "fear" could apply different thresholds by gender, producing unequal treatment.

Industry relevance. Any product that deploys an LLM for empathetic or emotionally aware responses — including the six model families tested here, five of which are commercially available or widely used — inherits the measured risk. The finding that fine-tuning (rather than prompt tweaking) is the effective lever gives engineering teams a concrete direction, while the FT-W/FT-M results warn that fine-tuning on skewed data can create new, harder-to-detect disparities.

Future Directions

• Multimodal evaluation. The authors call for testing biases with tone, body language, and conversational dynamics in addition to static text captions, since text-only captions capture only part of emotional complexity.
• Broader model and taxonomy coverage. They suggest extending the study to models such as Claude or Gemini and to alternative emotion frameworks such as Plutchik's wheel, which could reveal different bias trends.
• Gender diversity beyond the binary. The NarraCap dataset and the augmentation procedure only cover man/woman; non-binary and gender-diverse identities are unrepresented, and the authors state future work should cover all gender diversities.
• Cultural adaptation and data balancing. The authors note that non-50:50 fine-tuning may require balancing across all emotions, and that cultural factors in how emotions are expressed and perceived by gender remain an open question they cannot currently confirm or deny with available data.

Target Audience

Researchers and practitioners working on LLM fairness, affective computing, and emotional AI; ML engineers choosing between prompt-based and fine-tuning-based bias mitigation; and dataset and benchmark designers interested in gender representation. Readers with basic familiarity with LLM prompting and statistical testing will get the most out of it, though the conceptual framing is accessible to newcomers interested in AI ethics.

Authors’ abstract

The rapid advancement of large language models (LLMs) and their growing integration into daily life underscore the importance of evaluating and ensuring their fairness. In this work, we examine fairness within the domain of emotional theory of mind, investigating whether LLMs exhibit gender biases when presented with a description of a person and their environment and asked, ''How does this person feel?''. Furthermore, we propose and evaluate several debiasing strategies, demonstrating that achieving meaningful reductions in bias requires training based interventions rather than relying solely on inference-time prompt-based approaches such as prompt engineering, etc.