RV-HATE: Reinforced Multi-Module Voting for Implicit Hate Speech Detection

AI summary

Overview

Research area: Natural Language Processing — implicit hate speech detection, contrastive learning, and reinforcement learning-based ensembling.

Technical level: Intermediate to Advanced. Readers should be comfortable with contrastive learning, clustering-based anchor selection, ensemble voting, and policy-gradient reinforcement learning (PPO).

Scope: The paper introduces RV-HATE, a framework of four specialized detection modules whose outputs are combined by a reinforcement learning-optimized soft voting mechanism that assigns dataset-specific weights, evaluated on five hate speech datasets (IHC, SBIC, DYNA, Hateval, Toxigen).

What This Paper Is About

Hate speech datasets are built from different platforms and sources, so they differ in linguistic style, degree of implicitness, and annotation criteria. Most prior detection methods apply a single fixed methodology regardless of these differences. RV-HATE instead trains several modules that each capture a different linguistic or contextual property of hate speech, then uses reinforcement learning to decide how much each module should contribute for a given dataset.

Key Contributions

1. Four specialized modules designed to capture diverse dataset-specific characteristics of implicit hate speech: clustering-based contrastive learning with cosine similarity (M0), hate-target tagging with [ORG]/[NORP]/[GPE] entities (M1), outlier removal within clusters using the InterQuartile Range method (M2), and hard negative samples drawn from a queue spanning multiple batches (M3).
2. A reinforcement learning-based soft voting mechanism, trained with Proximal Policy Optimization, that assigns dataset-specific weights to each module rather than using fixed or equal weights.
3. State-of-the-art macro-F1 performance across five benchmarks, improving on the prior leading model SharedCon by an average of 1.8 percentage points, plus interpretable insight into how individual modules contribute under different dataset characteristics.

Main Findings

• Best macro-F1 on all five datasets: Averaged over three runs with different random seeds, RV-HATE scores 79.07 on IHC, 84.62 on SBIC, 81.82 on DYNA, 83.44 on Hateval, and 93.41 on Toxigen, averaging 84.47. The closest baseline, SharedCon, averages 82.67; CE averages 82.29, SCL 82.63, and LAHN 82.57.
• Gains over specific baselines: On Hateval, contrastive baselines SharedCon and LAHN underperform the cross-entropy baseline (81.11%), while RV-HATE reaches 83.44%, exceeding CE by 2.33 percentage points. On Toxigen, RV-HATE outperforms SharedCon by 2.2 percentage points.
• Reinforced weights beat equal weights: Compared with fixed weights of [0.25, 0.25, 0.25, 0.25], the learned weights improve the average by 0.68 percentage points (84.47 versus 83.78).
• Keeping modules separate beats merging them: A single model trained on all four modules jointly ("combined modules") averages 81.64, an average decrease of 2.83 percentage points relative to RV-HATE, indicating that joint training causes a loss of specialization.
• Cosine similarity beats Euclidean distance: RV-HATE improves by an average of 0.46 percentage points over the ℓ2 variant and shows a lower standard deviation across datasets.
• Module ablation: Each module alone scores below the full framework (M0 averages 82.68, M1 82.43, M2 82.89, M3 83.00), and removing any single module lowers the average (removing M0 gives 84.17, M1 84.09, M2 84.15, M3 83.99 versus 84.47). M3's standalone average is the highest among individual modules, and removing M3 has the greatest impact across all datasets.
• Dataset-specific dependence on target tagging: Excluding M1 causes the largest drop on IHC (-0.7 percentage points), consistent with IHC containing the highest proportion of entity-tagged data (67.83%). On SBIC, which is an offensive language dataset, excluding M1 causes only a marginal drop (-0.12 percentage points).
• Outlier removal matters most where noise is highest: Excluding M2 causes drops of 0.47 percentage points on IHC and 0.43 percentage points on Hateval, the two datasets with the highest proportions of removed data (0.59 and 0.69, respectively).
• Data quality statistics: spaCy tagging covers approximately 18.75% of the data on average, and supplementary gpt-4o tagging raises this to almost 50.88%. Analysis of 500 randomly sampled instances per dataset with gpt-4.1 finds broken sentences in all datasets, averaging 30.28%, and an average of 18.24% mislabeled data.
• Error analysis: Ambiguous (type2) errors account for 45.15% of IHC errors, and mislabeled (type3) errors account for 44.76% of DYNA errors and 41.78% of Hateval errors. Toxigen, as a machine-generated dataset, has no broken-sentence (type1) errors but type2 errors make up 50.00% of its error samples.
• Computational cost: RV-HATE uses four BERT-base modules (110M each) plus 4,085 parameters for PPO (Table 5), with training time of 1 hour per module plus 5-10 minutes for PPO on the IHC dataset, and inference latency of 0.5-1.5 ms per module. The paper notes PPO's parameter count is negligible and that the four modules can be trained in parallel across GPUs.

Methodology in Plain English

The researchers start from SharedCon, an existing state-of-the-art approach that clusters sentence embeddings and picks the sample closest to each cluster center as the anchor for contrastive learning. They change the selection criterion from Euclidean distance to cosine similarity, on the intuition that vector direction better reflects semantic alignment than absolute magnitude in high-dimensional space.

Around this base module they build three variants, each adding one capability. The first tags tokens referring to specific groups or institutions in hate-labeled data, using spaCy for named entity recognition and gpt-4o as a backup when spaCy fails, focusing on three entity types. The rationale is that hate speech is distinguished from merely offensive text by the presence of a specific target. The second module removes outliers inside clusters: broken or fragmented sentences from web crawling tend to sit far from cluster centers, so the authors apply the InterQuartile Range method, delete points beyond an upper-bound threshold, and recompute the cluster center. The third module collects hard negatives — samples with high cosine similarity to an anchor but a different label, plus high-confidence false positives — into a queue that spans multiple batches, giving the model a sharper decision boundary.

Each of the four modules is fine-tuned into its own classifier, and each classifier outputs a logit vector for the non-hate and hate classes. The final prediction takes a weighted average of the four logit vectors and picks the higher-scoring class. The weights are not set by hand: a policy network conditioned on the current state samples a weight vector, the soft voting is applied, and an F1-based reward on a validation set guides policy optimization with PPO. Weights start at 0.25 each and are constrained during training to remain positive and sum to 1.

Experiments use BERT-base-uncased (110M) with Sim-CSE as the text embedding model, trained for 6 epochs on NVIDIA RTX 4090 hardware, with 10,000 reinforcement learning steps and three random seeds.

Why This Matters

Impact on research: The paper argues that hate speech detection performance typically plateaus around the 80% range, and that 1-2 percentage point improvements are therefore meaningful. It reframes the problem from "find the one best method" to "match the method to the dataset," and it produces interpretable evidence, via module weights and ablation results, that different datasets depend on different linguistic and contextual signals.

Real-world applications:

• Content moderation on social platforms where posts come from many communities and linguistic styles, requiring per-context rather than one-size-fits-all detection.
• Reducing annotator burden: the authors note that their approach learns from representative samples without requiring manually annotated implications, which they argue lowers the mental load on annotators.
• Triage of low-quality training corpora, since the framework quantifies broken sentences, entity coverage, and mislabeled data in existing datasets.
• Auditing and explaining moderation decisions, since the learned module weights reveal which signals drive predictions for a given data source.

Industry relevance: The framework is built on BERT-base encoders and remains a simple forward-pass ensemble at inference, with no iterative decoding or reasoning. Per-module latency is 0.5-1.5 ms, training is 1 hour per module (parallelizable), and the reinforcement learning stage runs once and takes about 5-10 minutes. The main cost is that inference requires four forward passes, scaling linearly with the number of modules, plus the cost of deploying four models.

Future Directions

• Improving the target-tagging module M1 on machine-generated samples, which the authors identify as inconsistent in their experiments, potentially due to style and distribution differences.
• Addressing the dominant error sources the paper identifies: ambiguous instances (type2) and mislabeled instances (type3), which account for the largest shares of errors across datasets.
• Extending the modular design to artificially generated text more broadly, which the authors name as future work for broadening applicability.
• Reducing the linear inference cost of running four forward passes, potentially by compressing or selectively activating modules.

Target Audience

Researchers and practitioners working on hate speech and toxic content detection, particularly those interested in implicit hate speech, contrastive learning, or reinforcement learning-based ensembling. It is also useful for dataset builders and moderation teams who need to understand how dataset construction choices — source platform, annotation criteria, and resulting noise — affect model behavior. Readers without background in contrastive learning or policy-gradient methods will need some preparation, since the framework combines both.

Authors’ abstract

Hate speech remains prevalent in human society and continues to evolve in its forms and expressions. Modern advancements in internet and online anonymity accelerate its rapid spread and complicate its detection. However, hate speech datasets exhibit diverse characteristics primarily because they are constructed from different sources and platforms, each reflecting different linguistic styles and social contexts. Despite this diversity, prior studies on hate speech detection often rely on fixed methodologies without adapting to data-specific features. We introduce RV-HATE, a detection framework designed to account for the dataset-specific characteristics of each hate speech dataset. RV-HATE consists of multiple specialized modules, where each module focuses on distinct linguistic or contextual features of hate speech. The framework employs reinforcement learning to optimize weights that determine the contribution of each module for a given dataset. A voting mechanism then aggregates the module outputs to produce the final decision. RV-HATE offers two primary advantages: (1)~it improves detection accuracy by tailoring the detection process to dataset-specific attributes, and (2)~it also provides interpretable insights into the distinctive features of each dataset. Consequently, our approach effectively addresses implicit hate speech and achieves superior performance compared to conventional static methods. Our code is available at https://github.com/leeyejin1231/RV-HATE.