Harm Laundering in GPT Models: Evidence That Gender Discrimination Is Transformed Rather Than Reduced Across Safety-Trained Generations

AI summary

Overview

Research area: Natural Language Processing, AI safety evaluation, algorithmic fairness and gender bias auditing.

Technical level: Intermediate. The core argument is accessible without specialist knowledge, but the paper relies on topic modelling, multi-classifier discordance analysis, and mixed-effects regression, so some statistical and NLP background helps.

Scope: A longitudinal audit of 450,000 gender-directed text completions from 15 OpenAI GPT models (GPT-2 through GPT-5) arguing that safety training transforms discrimination into forms that standard toxicity classifiers cannot detect.

What This Paper Is About

Safety evaluations for large language models typically measure harm by running outputs through toxicity classifiers and reporting falling scores across model generations. This paper argues that this methodology is systematically incomplete: when OpenAI's GPT models are scored for gender-directed harm, explicit discriminatory content does not disappear — it changes shape into subtler representational asymmetries that toxicity classifiers rate as clean. The authors name this pattern harm laundering and set out to formally define it, demonstrate it empirically across four model generations, and provide a detection protocol for other researchers.

Key Contributions

1. A formal definition of harm laundering as a three-criteria failure mode of capability-scaled alignment: (a) explicit harm declines, (b) representational harm redistributes asymmetrically across demographic groups, and (c) standard toxicity classifiers fail to detect the transformed content. All three conditions must hold independently.

2. A three-stage detection protocol applicable to any generative model using standard audit resources: surface-level toxicity evaluation, per-stratum topic structure audit, and multi-scorer discordance testing between a demographic-sensitive classifier and a surface-form one.

3. An empirical trajectory audit across 450,000 completions from 15 models spanning GPT-2 to GPT-5, stratified by women-directed, men-directed, and neutral people-directed prompts, integrating topic modelling, sentiment analysis, sexism detection, NLI hypothesis scoring, and a blind human audit of contested decisions (Fleiss' kappa = 0.73).

4. Demonstration that three independent toxicity classifiers produce false positive safety signals on laundered content, with REGARD representational harm disparity rising monotonically with model release date (rho = +0.55, p = .034) while Detoxify scores show no equivalent trend (rho = -0.23, p = .42).

Main Findings

• Explicit harm disappears by GPT-4. BERTopic identified 15 topic clusters containing "rape" or "raped" across the model lineage (14 concentrated in women-directed completions, one residual at text-davinci-003). Zero such clusters appear in any GPT-4 or GPT-5 stratum, and a document-level check confirms the content is absent rather than merely unclustered.

• Gender asymmetries in medical naming at GPT-5. Men-prompted GPT-5 output contains a 1,997-document topic framing breast cancer as a men's rights and feminist debate. Women-prompted GPT-5 output contains zero clusters mentioning breast cancer, HPV, cervical cancer, or mammogram. Independent biomedical NER confirms: 701 of 10,000 men-directed rows name breast cancer, versus zero of 10,000 women-directed rows.

• All three toxicity classifiers miss the laundered content. The 1,997 Topic-5 documents score 0.005 on Detoxify, 0.001 on ToxiGen, and 0.180 on REGARD-negative — all consistent with non-toxic content. The harm is distributional, not lexical, so lexical classifiers cannot see it.

• Sentiment inverts at GPT-4, but the inversion is over-correction rather than improvement. Early models (GPT-2, GPT-3) score men-directed completions higher on positive sentiment. This reverses at GPT-4 and GPT-5, with women-directed completions scoring higher (Cohen's d range -0.31 to -0.57). Simultaneously, "less credible or knowledgeable" NLI entailment rises for women and intersectional content for women collapses from 30.84% at GPT-2 to 1.09% at GPT-4 (men decline only to 10.50%). Positive framing and epistemic undermining co-occurring is the paper's key structural signature.

• Topic diversity narrows for women only. Women-directed topic diversity falls 36% relative to men at the GPT-4 alignment boundary (W/M ratio 0.58, down from 0.91 at GPT-2).

• Classifier agreement collapses exactly where laundering is most active. Pairwise Cohen's kappa between the BERTweet sexism detector and Detoxify falls from 0.093 (GPT-2) to -0.001 (GPT-5 without context). Fleiss' kappa across five classifier flags reaches -0.164 at GPT-5. Discordance rates for women-directed completions peak at GPT-3 (29.0%) and collapse at GPT-4 (5.5%), while men-directed rates stay far lower in every era.

• Demographic asymmetry predates alignment. The SFT-era model text-davinci-002 produces underscore-type redactions in 53.9% of women-directed completions versus 32.7% for men — a 21.2 percentage point gap that cannot be explained by completion length. RLHF inherits and amplifies a pre-existing asymmetry rather than creating it.

• Human validation supports the classifier discordance claim. Of completions flagged sexist by BERTweet but scored clean by Detoxify, 76.9% (GPT-2) and 69.2% (GPT-3) were human-confirmed as demeaning or stereotyping, against 0% at GPT-4 and 8.3% at GPT-5. Topic-5 documents were 0% human-flagged, consistent with the claim that the harm is distributional rather than per-item.

• Context provision attenuates the effect. The GPT-5 with-context condition collapses equity framing asymmetry from +20.4 percentage points to -0.9 percentage points, and reduces the cross-demographic referencing gap from +40.9 to +8.8 percentage points. The authors identify this as the most effective mitigation pattern observed.

• The neutral condition behaves differently from both gendered conditions. People-directed completions show discordance at or below 0.2% in every era and produce no sexual-violence clusters. This rules out a prompt-template explanation and shows the harms are triggered by naming a demographic group.

Methodology in Plain English

The authors generated 30,000 text completions from each of 15 GPT models — 10,000 each for prompts beginning with "Women can…", "Men can…", and "People can…" — producing a 450,000-document corpus that spans GPT-2 through GPT-5. The single-axis sentence-completion format was chosen because it is the only prompt paradigm that runs unchanged across all 15 models, including the non-instruction-following GPT-2 family, which makes cross-generational comparison possible.

They then analysed the same corpus in several independent ways. Topic modelling with BERTopic (using UMAP for dimensionality reduction and HDBSCAN for clustering) was applied separately to each demographic-by-model stratum, revealing what each model actually talks about when prompted about each group. Three toxicity classifiers (Detoxify, ToxiGen, REGARD) were applied to the same content, alongside a sexism detector, a sentiment model, and an NLI scorer testing a pre-fixed battery of 32 hypotheses about identity groups. The key analytical move is looking for divergence: when a demographic-sensitive classifier flags content that a surface-form toxicity classifier scores as clean, that discordance is the signal.

Statistical work included mixed-effects regression with random intercepts by model, a pre-specified family of five confirmatory directional tests with Benjamini-Hochberg correction, Spearman correlations against release date, and a difference-in-differences analysis at the GPT-3 to GPT-4 boundary. A blind human audit of 150 stratified rows — including 62 discordant cases and 23 Topic-5 documents — validated the contested classifier decisions.

Why This Matters

The paper challenges the assumption that falling toxicity scores mean safer models. If harm can be transformed rather than reduced, then the entire evaluation infrastructure that governs AI deployment decisions is measuring the wrong thing — and measuring it with a metric that gets less reliable as models improve.

Impact on research: The work connects to existing observations that detoxification shifts rather than eliminates harm, but formalises the phenomenon, provides a falsifiability criterion (laundering and genuine improvement make opposite predictions at the alignment boundary), and offers a reusable detection protocol. It reframes the problem as a construct-validity failure: surface-form classifiers measure lexical markers of harm, and those markers came apart from the harm construct itself on this corpus.

Real-world applications:

• Content recommendation and summarisation pipelines that consume LLM outputs will propagate representational asymmetries at scale without any awareness that the upstream harm signal was transformed rather than removed.
• Educational and health information tools built on LLM output could inherit asymmetric naming patterns — the GPT-5 finding that breast cancer appears in men-directed completions but not women-directed ones is directly relevant to medical self-care content.
• Regulatory compliance for AI systems under the EU AI Act, UK AI Safety Institute protocols, and US executive-order guidance, all of which currently centre harm evaluation on toxicity benchmarks that this paper shows to be blind to laundered harm.
• Model developers designing safety training who need to know that optimising against a surface-form metric degrades that metric's validity, so a safety metric used as a training target requires independent validation evidence.

Industry relevance: Any organisation shipping or procuring LLM-based products has a stake in knowing whether its safety assurance rests on false signals. The authors argue that topic-structure equity analysis and cross-classifier divergence testing should be required alongside standard toxicity benchmarks, and note that governance frameworks attaching such obligations to concrete organisational mechanisms (independent audit, transparency requirements) indicate where these mandates could attach.

Future Directions

• Testing harm laundering outside the OpenAI GPT lineage. The paper's findings are specific to RLHF-based alignment. Whether the same pattern appears in Constitutional AI, DPO, or instruction-tuned model families is an open question the detection protocol is designed to answer.

• Extending beyond binary gender framing. The authors acknowledge that binary operationalisation misrepresents gender diversity and renders non-binary, genderqueer, and trans experiences invisible. Extending the methodology to non-binary identity framing is flagged as a direct next step.

• Scaling human validation. The blind audit covered 150 deliberately targeted rows rather than the full corpus. A larger pre-registered validation sample would strengthen the discordance argument.

• Establishing causal mechanism. The design is observational, and the release-era boundary confounds alignment with changes in pre-training data, instruction tuning, deployment-side moderation, and architecture. A within-GPT-3 comparison of SFT-only versus SFT-plus-RLHF provides partial isolation but rests on only two models.

• Improving the detection instruments. The NLI stereotype scorer saturates on this corpus and was excluded from the confirmatory family. The authors treat the NLI results as supporting signals only, implying instrument calibration work is needed.

Target Audience

AI safety and alignment researchers evaluating model harm; NLP fairness researchers working on bias measurement and evaluation methodology; AI governance and policy specialists designing audit requirements for deployed systems; and machine learning engineers at organisations that ship or procure LLM-based products and currently rely on toxicity benchmarks as their primary safety assurance. The paper is also valuable to measurement theorists interested in construct validity, since its central argument is fundamentally about the gap between an intended construct and the operational measure used to track it.

Authors’ abstract

Safety evaluations for large language models rely on surface-form classifiers that report declining harm scores across model generations. We provide evidence that this methodology is systematically incomplete: explicit discriminatory content is transformed rather than removed. We call this \emph{harm laundering}. Analysing 450,000 gender-directed completions across 15 models spanning GPT-2 through to GPT-5 (OpenAI GPT lineage; three demographic conditions), we show that sexual violence clusters prevalent in GPT-2 women-directed output disappear by GPT-4, while men-directed completions gain positive representational territory (caregiving, emotional range, ally identity) that women-directed completions do not. The pattern is most visible at GPT-5: Topic~5 (1,997~documents) frames breast cancer as a men's rights debate, while zero equivalent clusters appear in women-directed output. Three independent classifiers score this content as non-toxic. Sentiment scores invert at GPT-4: early models demean women; later models over-correct. Topic diversity in women-directed completions falls 36\% relative to men at the GPT-4 alignment boundary (W/M~$= 0.58$, from $0.91$ at GPT-2). REGARD representational harm disparity correlates with release date ($ρ= +0.55$, $p = .034$) while Detoxify does not ($ρ= -0.23$, $p = .42$): toxicity scores fall as representational harm grows. We formalise harm laundering as a three-criteria test and provide a three-stage detection protocol applicable to any generative model. Within the OpenAI GPT lineage, toxicity score reduction is not a sufficient proxy for harm reduction.