Self-Explaining Hate Speech Detection with Moral Rationales

AI summary

Overview

Research area: Natural Language Processing — explainable hate speech detection, attention-based interpretability, and moral reasoning in language models.

Technical level: Intermediate. The paper assumes familiarity with transformer encoders, attention mechanisms, supervised rationale alignment, and Moral Foundations Theory, but the underlying idea can be grasped without deep mathematical background.

Scope: The paper proposes a self-explaining hate speech detection framework (SMRA) that supervises neural attention with expert-annotated moral rationales, releases a Brazilian Portuguese benchmark corpus (HateBRMoralXplain) with moral categories and span-level rationales, and evaluates both fine-tuned BERT models and prompted LLMs on hate speech and moral sentiment classification.

What This Paper Is About

Most hate speech detection models are opaque and lean on surface-level lexical cues, which makes them fragile, prone to spurious correlations, and poorly equipped to handle culturally or morally contingent language. The authors' goal is to build a model whose explanations are part of its training objective rather than generated after the fact, by aligning the model's token-level attention with human-annotated moral rationales grounded in Moral Foundations Theory. To make this possible for a low-resource language, they also build a new Brazilian Portuguese dataset that pairs hate labels with moral categories, token-level moral rationales, and socio-political metadata.

Key Contributions

1. Supervised Moral Rationale Attention (SMRA) — described as the first self-explaining framework that aligns model attention with expert-annotated moral rationales, integrating a moral attention alignment loss directly into the training objective rather than using post-hoc explanation methods.

2. HateBRMoralXplain — the third version of the HateBR corpus, extending the original and HateBRXplain with moral categories, moral rationales, and socio-political metadata. The authors state it is the first large-scale expert-annotated corpus of its kind to combine moral categories with human-annotated moral rationales. The dataset, models, annotator disagreements, and code are publicly released.

3. Evaluation of multiple LLMs on HateBRMoralXplain for hate speech and moral sentiment classification, using prompts with different informational components (definitions, data collection context, joint hate-and-moral framing, English translations), showing that including moral rationales yields statistically significant improvements in hate speech detection.

4. Comparison against prior supervised attention (SRA) and classic deep learning baselines (Bag-of-Words, CNN, BiRNN with MaxPool, BiRNN with attention), providing a broader picture of where moral rationale supervision helps and where it does not.

Main Findings

• Binary hate speech classification: SMRA improves over the base models, with Accuracy +0.0004, Macro F1 +0.0082, IoU F1 +0.0743, and Token F1 +0.0503. The largest plausibility gains come from BERTimbau, which moves from IoU F1 0.7612 to 0.8355 and Token F1 0.8455 to 0.8958.

• More concise explanations: Comprehensiveness decreases by −0.0203 and Sufficiency improves by −0.0231, which the authors interpret as rationales becoming more compact while remaining faithful.

• Multi-label moral sentiment classification: SMRA improves Macro F1 by +0.015, AUROC by +0.002, and Token F1 by +0.241. For BERTimbau, Token F1 rises from 0.7250 to 0.9660 and Macro F1 from 0.7570 to 0.7720.

• Backbone choice matters more than supervision: Across both tasks, the choice between BERTimbau and mBERT has a larger impact on overall performance than the use of supervised attention, pointing to the importance of language-specific pretraining. BERTimbau-base reaches 0.9029 Accuracy on hate classification while mBERT-base reaches 0.5343.

• SMRA versus SRA (hate rationales instead of moral rationales): SMRA achieves higher plausibility than SRA, with IoU F1 +0.1195 and Token F1 +0.1508. SRA attains higher Comprehensiveness but shows negative Sufficiency (−0.0360), which the authors flag as inconsistent causal alignment; SMRA maintains positive Sufficiency (0.0426 for BERTimbau-smra).

• Fairness remains stable: The authors report that fairness does not degrade, suggesting explanation-quality improvements do not introduce significant bias trade-offs. They also note that improvements in rationale alignment do not consistently translate into gains in fairness.

• Classic deep learning comparison: For binary hate speech, CNN performs best across all reported metrics (Accuracy 0.8214, Macro F1 0.8180, AUROC 0.9089), while BiRNN with attention (SMRA) performs worst (Accuracy 0.8014, Macro F1 0.8014, AUROC 0.8903). For multi-label moral classification, results are mixed: SMRA has the highest Accuracy (0.4386), Bag-of-Words the best Macro F1 (0.1982), and CNN the highest AUROC (0.7283).

• LLMs benefit from moral framing: Incorporating moral rationales improves hate speech classification by approximately 2–3%, but LLMs perform poorly on moral sentiment classification, with a maximum F1 of 0.38. Jointly predicting hate and moral labels benefits both tasks, with a stronger improvement for moral classification.

• Definitions help, context hurts: Adding the hate speech definition improves performance for both LLMs on hate detection and for Llama70B on moral classification, whereas providing data collection context tends to degrade performance.

• English translation lowers performance: Translating Portuguese prompts into English reduces performance for both models on both tasks, particularly hate speech classification. The best reported hate speech F1 values are in Portuguese (GPT-4o-mini 0.897 with hate moral w/ definition; Llama70B 0.901 with hate w/ definition).

• Ablation on moral guidance: Removing explicit Moral Foundations Theory guidance and reasoning increases hate speech performance by 2% for GPT-4o-mini but decreases it by 6% for Llama70B, while moral value classification drops substantially (34% for GPT-4o-mini and 36% for Llama70B).

• LLM rationales are sparse: BERTScore between LLM and human moral rationales is 0.76 for GPT-4o-mini and 0.71 for Llama70B, but Jaccard similarity is 0.11 for both. Humans identify around 5 words on average versus 1.93 for LLMs, indicating LLMs find correct words but miss relevant portions, lowering recall.

• Hard cases are culturally dependent: The authors extracted 235 offensive comments containing sarcasm/irony or heavy cultural dependence. Adding moral framing enabled Llama-70B to correctly label these nuanced cases, whereas prompts without moral framing misclassified them.

Methodology in Plain English

The authors start from a standard transformer text classifier. Alongside the usual cross-entropy loss for predicting a label, they add a second loss term that compares the model's attention distribution over tokens with a human-annotated "rationale mask" — a binary vector marking which tokens a human expert considered morally salient. The two losses are combined as total = cross-entropy + α × mean-squared-error, where α controls how strongly attention is pushed toward human rationales. This auxiliary loss is applied only to training examples that have moral content and human rationales, so no rationale supervision is needed for everything else.

To support this, they built HateBRMoralXplain: 7,000 Brazilian Portuguese Instagram comments from public political accounts (3,500 offensive and 3,500 non-offensive), previously annotated by three experts, now extended with moral categories based on Moral Foundations Theory. Comments can carry between one and three moral labels, ordered by salience (primary, secondary, tertiary), so the label set is {NN, HN, FN, PN, AN, LN} — Non-Morality, Harm/Care, Fairness/Cheating, Purity/Degradation, Authority/Subversion, and Loyalty/Betrayal — giving 6 classes. Rationale spans follow the same span-based protocol as MFTCXplain.

They fine-tuned mBERT and BERTimbau with HuggingFace Transformers and PyTorch using batch size 16, learning rate 2e-5, maximum sequence length 128, 20 epochs, an 80/10/10 train/validation/test split, and the AdamW optimizer. Attention supervision used the [CLS]-to-token attention weights from the last encoder layer, with an attention weighting of 0.001 and cross-entropy loss. They also prompted GPT-4o-mini (via the OpenAI API) and Llama3.1-70b (on local RTX A6000 GPUs) at temperature 0, testing prompt variants that add the hate speech definition, data collection context, moral framing, or combinations. Translations used the Google Translate API.

Evaluation covered classification (Accuracy, Macro F1, AUROC, AUPRC), plausibility (token-level IoU F1 and Token F1 against human rationales), faithfulness (Comprehensiveness and Sufficiency), and bias (GMB-Sub, GMB-BPSN, GMB-BNSP across gender, race, politics, and religion identity terms in Portuguese).

Why This Matters

The paper argues that opaque, lexically driven hate speech models inherit biases from training data, stereotypical associations, and annotator subjectivity, producing systematic disparities and unfair over-targeting of marginalized groups. Because judgments of offensiveness are culture-dependent, and moral values appear to be a more transferable latent representation than lexical markers, aligning attention with moral rationales offers a route to models that are both more interpretable and more culturally contextualized.

Potential real-world applications:

• Content moderation platforms that need to justify automated removals to users and auditors, where a model that points to the morally salient span behind a decision is more actionable than a bare label.
• Culturally grounded moderation in low-resource languages, where surface lexical markers are scarce or misleading and moral framing is a more stable signal.
• Political and public discourse analysis, using the dataset's socio-political metadata to study how hate speech and moral framing vary by political party and politician gender.
• Bias auditing and governance, using faithfulness and subgroup fairness metrics together to check whether interpretability gains come at a fairness cost.

Industry relevance: Deployed moderation systems face regulatory and reputational pressure to explain decisions, and the paper frames explanation quality and fairness as coupled concerns rather than separate ones. The finding that language-specific pretraining (BERTimbau over mBERT) matters more than the supervision mechanism is a practical signal about where engineering effort pays off. The dataset's restriction to Brazilian Portuguese and the cost of expert rationales are noted constraints on immediate scalability.

Future Directions

• Scale beyond manual annotation. The authors list reliance on high-quality human-annotated moral rationales as a key limitation, since these are costly and time-consuming, potentially limiting scalability to new languages or domains.
• Test cross-lingual and cross-platform generalization. SMRA has been evaluated primarily on Brazilian Portuguese social media data, so generalization across languages, platforms, and cultural settings remains an open question.
• Separate sub-dimensions within moral foundations. The authors explicitly do not distinguish sub-dimensions of the Fairness foundation (e.g., equality vs. proportionality), which prior work suggests capture meaningful variation across cultures and political contexts and may distinguish language used to identify versus justify harm.
• Understand why LLM rationales have low recall. Human rationales average around 5 words versus 1.93 for LLMs; the authors state that further studies are required to understand this phenomenon more effectively.

Target Audience

Researchers and practitioners working on hate speech detection, explainable NLP, and attention-based interpretability will get the most from this paper, particularly those interested in rationale supervision as an alternative to post-hoc explanation methods. It is also relevant to computational social scientists studying morality, political discourse, and annotator subjectivity, and to dataset builders working on low-resource languages such as Brazilian Portuguese. Trust-and-safety engineers evaluating whether interpretable moderation models can be deployed without fairness trade-offs form a secondary audience. Readers should note the paper carries an offensive-content warning.

Authors’ abstract

Existing hate speech detection models are often opaque and rely on surface-level lexical cues, which makes them vulnerable to spurious correlations and limits robustness, interpretability and cultural contextualization. We propose Supervised Moral Rationale Attention (SMRA), the first self-explaining hate speech detection framework to incorporate moral rationales as direct supervision for attention alignment. Based on Moral Foundations Theory, SMRA aligns token-level attention with expert-annotated moral rationales, guiding models to attend to morally salient spans. Unlike prior rationale-supervised or post-hoc approaches, SMRA integrates moral rationale supervision directly into the training objective, producing inherently interpretable and contextualized explanations. To support our framework, we also introduce HateBRMoralXplain, a Brazilian Portuguese benchmark dataset annotated with hate labels, moral categories, token-level moral rationales, and socio-political metadata. Across binary hate speech detection and multi-label moral sentiment classification, SMRA consistently improves performance while enhancing both faithful and plausible explanations. Although explanations become more concise, sufficiency decreases, indicating more compact and informative rationales. Fairness remains stable, suggesting that improvements in explanation quality do not introduce significant bias trade-offs.