Two Centuries of Sexism in British Parliament: A Computational Analysis of Women's Representation in the Hansard Corpus

AI summary

Overview

Research area: Computational social science / NLP applied to historical parliamentary text, combining large language model (LLM) classification with the social-psychological Ambivalent Sexism Inventory.

Technical level: Intermediate. The methods (keyword retrieval, LLM-as-judge classification, human validation, agreement metrics) are conceptually accessible, but the evaluation machinery — Cohen's kappa, per-class precision/recall/F1, cross-model comparisons — assumes some familiarity with NLP evaluation.

Scope: The paper classifies stance and sexism type in 6,531 suffrage-related speeches drawn from 200 years of UK Hansard debate (1803–2005), and releases a metadata-enriched version of the Hansard corpus containing 6.7 million speeches across 1.2 million debates.

What This Paper Is About

Historians have documented the British suffrage movement thoroughly, but the actual arguments made inside Parliament have rarely been analysed at scale. The authors ask whether supporters and opponents of women's political representation used the same rhetorical strategies to reach opposite conclusions, or fundamentally different kinds of gendered reasoning. To answer this, they use LLMs to classify the stance of 6,531 parliamentary speeches toward women's suffrage and to tag any sexism in those speeches as hostile or benevolent following the Ambivalent Sexism Inventory.

Key Contributions

1. A large gender-matched parliamentary dataset. The authors release an organized, metadata-enriched version of the publicly available Hansard Corpus with 6.7 million speeches across 1.2 million debates, achieving 89.3% gender-matching for House of Commons speeches (compared to 1.2% for the House of Lords).

2. A dual-axis annotation taxonomy and validation. They introduce a taxonomy covering stance (For / Against / Both / Irrelevant) and sexism type (hostile and benevolent, each with three subcategories), validated against human annotations on a 300-speech validation set.

3. Naturalistic evidence for the Ambivalent Sexism Inventory. They apply Glick and Fiske's (1996) framework — developed mainly in laboratory and survey settings — to real institutional discourse across two centuries.

4. A finding about reasoning, not just conclusions. They show that opposing political stances deploy different types of gendered reasoning: anti-suffrage rhetoric combines hostile and benevolent framing, while pro-suffrage sexism is overwhelmingly benevolent.

Main Findings

• Sexism rates diverge sharply by stance: 54% of speeches opposing women's representation contain sexist content, compared to 21% of speeches supporting the cause. Of 2,942 relevant speeches, 886 (30%) contain at least one form of sexism: 392 flagged hostile, 706 benevolent, and 212 flagged for both.

• The two sides use different types of sexism: Among the 570 against-speeches, 37% are hostile-only, 19% benevolent-only, and 44% contain both. Among the 2,167 for-speeches, the sexist subset is almost always benevolent (81% benevolent-only; only 11% hostile-only). The 205 "Both"-stance speeches sit in between: 57% are sexist, mostly benevolent (52% benevolent-only).

• Pro-suffrage sexism takes the form of "praise that confines": 462 for-speeches exhibit sexism, almost all benevolent and paternalistic. Examples include Sir George Bowyer (1867) arguing it would be "manifestly indecorous" for women to attend the hustings and proposing voting papers instead, and Rev. Ian Paisley (1992) praising women's "tenderness that some males do not have."

• A gender gap in support that closes only after enfranchisement: Female MPs support women's representation at 93%, compared to 70% for male MPs (chi-squared = 86.75, p < 0.001). A logistic regression controlling for decade gives an odds ratio of 2.01 (p = 0.002). Female MPs remain steady at 92–97% support across eras, while male support rises from 54% in 1870–1899 to 91% post-1950.

• Support grows and opposition collapses over time: Of the 2,942 relevant speeches, 74% are for, 19% against, and 7% both. By era, for-percentages rise from 54% (pre-1870) and 53% (1870–1899) to 92% (post-1950), with opposition falling to 6% post-1950.

• Hostile sexism declines while benevolent sexism persists: Hostile sexism falls from 60% of sexist speeches in 1870–1899 to 27–30% after 1929, whereas benevolent sexism stays at 74–83% throughout the corpus.

• The distinction is not just sentiment: Among anti-suffrage speeches, 88% of hostile and 86% of benevolent speeches are negative in sentiment (overall 87% hostile, 77% benevolent), so a sentiment classifier cannot separate the two forms.

• A paradox: Despite 74% of relevant speeches favouring women's suffrage versus 19% opposed, suffrage legislation took decades to pass — suggesting opponents, though a parliamentary minority, exerted disproportionate rhetorical influence.

• The LLM judge performs comparably to human annotators: Claude Sonnet 4.6 reaches 83.3% agreement and kappa = 0.711 against consensus human labels, versus human–human agreement of 79.3% and kappa = 0.644. Baselines trail: majority class 53.7% (kappa 0.000), DeBERTa-v3 zero-shot 50.3% (kappa 0.269), TF-IDF with logistic regression 68.7% (kappa 0.419).

• Sexism detection is hard and likely undercounted: When Claude flags a speech as sexist it is usually correct (precision 0.77–0.86), but it misses roughly half the cases annotators would flag (recall 0.43–0.46), so the reported sexism rates are likely underestimates.

• Stance classification is weakest on the rare "Both" class: Per-class F1 is 0.80 for For, 0.75 for Against, 0.90 for Irrelevant, but only 0.22 for Both (13 cases in the 300-speech validation set); macro average 0.67, weighted 0.82.

Methodology in Plain English

The researchers started with the Hansard Parliamentary Corpus and extracted 6,783,015 individual speeches by 52,661 speakers spanning 203 years (1803–2005) of House of Lords and House of Commons proceedings, drawn from 1,197,828 debates. They matched speakers to individual MPs using a cascaded procedure combining ministerial titles, constituency records, temporal matching, and fuzzy string matching with Levenshtein distance thresholds, deliberately prioritizing precision over recall to avoid misgendering. This yielded 89.3% gender-matching coverage in the Commons.

To find suffrage-related speeches, they used a two-tier case-insensitive keyword search: Tier 1 matched explicit suffrage terms (n = 2,725), and Tier 2 matched "women" or "female" within 25 words of a voting-related term (n = 3,806), producing 6,531 speeches from 1809 to 2004.

They then built an annotation taxonomy with two axes. The stance axis has four mutually exclusive categories: For, Against, Both, and Irrelevant (the last serving to filter false positives from keyword retrieval). The sexism axis follows Glick and Fiske's Ambivalent Sexism Inventory, distinguishing hostile sexism (dominative paternalism, competitive gender differentiation, heterosexual hostility) from benevolent sexism (protective paternalism, complementary gender differentiation, heterosexual intimacy); hostile and benevolent are independent flags.

Two computational social science researchers (one male, one female) independently annotated a 300-speech validation set, each investing roughly 40 hours, working in randomized order via a Streamlit application to avoid era-clustering bias, and resolving 106 disagreements (62 on stance, 44 on sexism subcategories) through discussion. Median speech length was 890 words, with the 95th percentile at 3,521 words.

Claude Sonnet 4.6 served as the primary LLM judge via the Anthropic API, with up to 5 preceding and 5 following speeches supplied as context to mirror the human setup. The authors checked for model-specific quirks by running three additional judges (GPT-5, Gemini 2.5 Flash, DeepSeek V3) plus non-LLM baselines. A DistilBERT model fine-tuned on SST-2 was used to test whether the hostile/benevolent distinction merely reflects sentiment. A noise-induction robustness check flipped 3% of speaker gender labels in each of 1,000 iterations.

Why This Matters

Impact on research. The paper provides naturalistic, large-scale evidence for a theory previously studied mainly in surveys and labs, and it demonstrates a reusable design pattern: pairing LLM-as-judge classification with an established social-science taxonomy to study historical text far too large for close reading. The released dataset lowers the barrier for other computational social science work on Hansard. The authors are careful to note that feminist historians and sociologists have long examined overt hostility and paternalistic reasoning, and that their contribution is quantitative scale rather than new insight.

Real-world applications:

• Analysing legislative debates on any group-rights question for the type of discriminatory reasoning, not merely its direction — the authors suggest the methods apply to debates over equal marriage, reproductive rights, and international deliberations.
• Monitoring contemporary discourse for benevolent sexism, which the paper argues is harder to notice and challenge than overt hostility and can coexist with supportive policy positions.
• Building or auditing historical corpora for computational social science, using the paper's gender-matching and metadata-enrichment approach as a template.
• Screening LLM-based content analysis pipelines: the paper's validation tables and cross-model agreement matrix offer a concrete benchmark for how reliable an LLM judge is on difficult, subjective labels.

Industry relevance. The findings matter for anyone deploying LLMs as annotators at scale: the paper shows that a strong judge can match human agreement on a well-defined task (kappa 0.711 versus 0.644) but still miss roughly half of hard cases, which directly bears on how much confidence to place in automated content moderation, bias auditing, or data-labeling pipelines.

Future Directions

• Extend beyond the House of Commons. The study covers only the Commons, where gender-matching reached 89.3% compared to 1.2% in the Lords — extending speaker resolution to the Lords is an open problem.

• Improve retrieval beyond keyword filtering. The keyword approach both misses relevant speeches (e.g., Mr. Hopkins's 1920 speech on extending the Representation of the People Bill, where "vote" was not within 25 words of "women") and admits false positives, so recall-oriented or embedding-based retrieval is a natural next step.

• Address the detection asymmetry. Hostile sexism is more obvious to a classifier than benevolent sexism, which is embedded in superficially positive language; the authors flag a possible residual asymmetry despite their sentiment-confound analysis.

• Apply the framework to other contested-group debates. The authors suggest any legislative debate corpus could be analysed for the type of discriminatory reasoning, and note that richer LLM capabilities make historical text tractable at scale across centuries.

Target Audience

Computational social scientists and NLP researchers interested in political discourse, bias, and historical corpora will get the most from this paper, as will digital humanities scholars working with Hansard or comparable parliamentary records. Researchers studying gender representation, sexism measurement, or LLM-as-judge methodology — including those who need to know how reliable automated annotation is on subjective labels — are also a natural audience. A secondary audience is practitioners building content-analysis pipelines who want a worked example of validating an LLM judge against human consensus and across multiple models.

Authors’ abstract

The language a legislature uses to debate women's rights, even in favour of them, encodes systematic patterns of sexism that persist across two centuries. In this work, we analyse 6,531 speeches over 200 years of UK parliamentary debate (Hansard, 1803-2005) by using large language models to classify a speaker's perspective towards women's suffrage and political representation, as well as analyse sexist speech in parliament from the lens of the Ambivalent Sexism Inventory. We also release this parliamentary dataset, an organized and metadata-enriched version of the publicly available Hansard Corpus optimized for computational social science research, with 6.7 million speeches across 1.2 million debates, with 89% gender-matching for speeches by MPs from the House of Commons. We find that 54% of speeches opposing women's representation contain sexist content, compared to 21% of speeches that are for the cause, and that the two sides use fundamentally different types of sexism: anti-suffrage rhetoric combines hostile and benevolent framing, while pro-suffrage sexism is overwhelmingly benevolent. Female MPs support women's political rights at 93% compared to 70% for male MPs, a gap that closes only after enfranchisement. Our findings are evidence that benevolent and hostile sexism are used in different rhetorical contexts in a manner consistent with the theory of Ambivalent Sexism.