Leveraging LLMs for Context-Aware Implicit Textual and Multimodal Hate Speech Detection

AI summary

Overview

Research area: Natural Language Processing — hate speech detection (HSD), specifically implicit hate speech in text and misogyny in multimodal memes, with a focus on using LLMs as context generators rather than as classifiers.

Technical level: Intermediate. The paper assumes familiarity with sentence embeddings (SBERT), transformer architectures, MLP classifiers, and standard F1-based evaluation, but it explains each pipeline component clearly enough for readers with a general NLP background.

Scope: The paper evaluates whether background context generated by Gemini 2.0 Flash from the full content of a post improves an SBERT + MLP hate speech classifier, testing four context-incorporation strategies on the Latent Hatred (textual) and MAMI (multimodal memes) datasets. Note: the paper contains examples of hate speech, including explicit, racist, or sexist language.

What This Paper Is About

Implicit hate speech — hostility conveyed through irony, euphemism, coded language, or external world knowledge — is much harder to detect than explicit hate speech, and most HSD models operate in a "zero-context" setting on isolated posts. Prior attempts to add background knowledge relied on entity linking to static resources such as Wikipedia, ConceptNet, and Wikidata, which often fails because of poor entity detection or irrelevant retrieved content. This paper asks whether an LLM can instead act as a dynamic knowledge base that generates post-specific background context from a post's entire content, and which method of feeding that context into a classifier works best.

Key Contributions

1. Post-level context generation instead of entity-level retrieval. Rather than linking named entities to external knowledge bases, the authors prompt Gemini 2.0 Flash with the full post (or meme) to produce objective background context covering entities, events, and implicit meanings or rhetorical devices, with the prompt framed as a neutral knowledge-generation task so the model is not told the output will be used for hate speech classification.

2. A systematic comparison of four context-incorporation strategies. The paper tests Append & Embed (text concatenation), Embed & Concat (embedding-level concatenation), Context-Embed (a hierarchical transformer-based fusion adapted from Bourgeade et al. 2024), and LLM Enhance (LLM rewriting of the post into a context-enhanced version), isolating the effect of the integration method from the generation method.

3. Evaluation across textual and multimodal settings. The approaches are tested on the Latent Hatred dataset of implicitly hateful tweets (binary and six-class implicit hate categories) and on the MAMI dataset of misogynous memes (binary and multi-label), with the multimodal pipeline representing each meme entirely as text via LLM-extracted meme text plus an LLM-generated one-sentence image description.

4. A comparison against four baselines and an error analysis. Baselines are Zero-Context, a replication of Lin's (2022) REL-based entity linking, a ConceptNet Numberbatch replication of ElSherief et al. (2021), and a zero-shot LLM classifier, plus a qualitative analysis of cases where added context flipped correct predictions to incorrect ones.

Main Findings

• Embedding concatenation wins consistently. The Embed & Concat strategy achieved the best results in both settings. On Latent Hatred it reached macro F1 of 0.75 in binary classification and 0.53 in the multi-class task; on MAMI it reached macro F1 of 0.85 in binary and 0.63 in multi-label.

• Gains over the zero-context baseline. From the zero-context baseline to the best system, incorporating LLM-generated context improved performance by up to 3 F1 points in the textual setting and up to 6 F1 points in the multimodal setting.

• Entity-linking baselines underperform. The REL replication underperformed the zero-context model, with a 4-point F1 drop on Latent Hatred multi-class, and also underperformed on MAMI, which the authors attribute to limited named entity coverage (only 9,595 of 21,480 Latent Hatred posts, 44.67%, were linked to any entity; on MAMI only 3,394 of 10,995 memes, 31%, contained at least one entity, averaging 0.6 entities per meme). The ConceptNet baseline stayed within 1 F1 point of the baseline in both MAMI tasks.

• Deeper fusion and rewriting hurt. Context-Embed and LLM Enhance performed worse than the simpler concatenation approaches, and both performed worse than the baselines on Latent Hatred, indicating that keeping post and context separate is more effective than fusing them at the text or embedding level.

• Direct LLM classification is competitive but uneven. On Latent Hatred, the LLM classifier was comparable to other baselines in binary classification (macro F1 0.70) but far behind on multi-class (0.26). On MAMI, it surpassed all other setups on binary classification (macro F1 0.86) while remaining competitive on multi-label (0.60).

• Context can cause semantic drift in both directions. On Latent Hatred, 158 posts (3.6% of the test set) were false negatives for the context-enhanced model but true positives for the zero-context model, often when lengthy context was added to short, explicitly hateful posts. Conversely, 187 non-hateful posts (4.4%) were wrongly flagged by Embed & Concat but not by the zero-context model, sometimes because context invented hateful associations. On MAMI, 68 misogynous memes (3.1%) and 63 non-misogynous memes (2.9%) flipped in the same two directions.

• Context does not majorly redistribute multi-label errors on MAMI. The authors report slight improvements across the board for all labels, bringing minority classes closer to level rather than shifting the error distribution.

Methodology in Plain English

The pipeline has two stages: generating context, then incorporating it.

Context generation. Each post is passed to Gemini 2.0 Flash with a prompt asking for brief, factual background context — identifying implicit meaning or sarcasm, explaining obscure named entities, and giving relevant world knowledge. The prompt deliberately avoids mentioning hate speech so that implicit labels are not leaked into the generated text. For memes, the same model extracts the text embedded in the image, produces a one-sentence description of the image (ignoring text), and generates background context. The extracted text plus the image description together serve as the textualised representation of the meme.

Context incorporation. Four strategies are compared: appending context to the post after a [SEP] token before embedding; embedding post and context separately and concatenating the vectors; the Context-Embed approach, in which context is encoded with SBERT, projected to the token-embedding dimension, and prepended to the post's token embeddings before passing through the encoder layers and pooling; and LLM Enhance, in which the LLM rewrites the post and its context into a single context-enhanced post.

Classification and evaluation. Posts are encoded with SBERT all-mpnet-base-v2, which produces normalised 768-length embeddings from sentences up to 384 tokens. These feed an MLP with three hidden layers of dimension 512, ReLU activations, trained for 500 epochs with the Adam optimiser at a learning rate of 0.001. Each experiment is run ten times and mean results are reported, using macro F1 as the primary metric, plus positive-class F1 in the binary settings and per-class F1 elsewhere.

Data. Latent Hatred contains 21,480 tweets from known hate group accounts, roughly 62% non-hateful, 33% implicit hate, and 5% explicit, with a six-way implicit typology (incitement to violence, inferiority language, irony, stereotyping, threatening, white grievance). MAMI contains 10,995 distinct memes with a 50/50 binary misogynous split and multi-label subtypes (shaming, stereotyping, objectification, violence); five duplicate memes were found during preprocessing and removed. Both datasets were split 80/20 into train and test while maintaining class balance.

Why This Matters

Impact on research. The paper reframes contextualised hate speech detection from an entity-retrieval problem into a post-level semantic generation problem, and shows that how context is integrated matters as much as whether it is generated. It also provides evidence that LLMs may be more useful as context providers for lightweight, fine-tuned classifiers than as drop-in replacements for them, which contrasts with studies reporting that generalist LLMs outperform encoder models.

Real-world applications.

• Content moderation systems that need to flag subtly coded or euphemistic hate speech rather than only explicit slurs.
• Meme moderation pipelines that must reason jointly over image and text, where the offence often only emerges from the combination.
• Human moderation support tools that surface explanatory background about a post, helping reviewers interpret obscure references and cultural context.
• Dataset construction and annotation workflows, where generated background context could help annotators interpret ambiguous posts.

Industry relevance. The modular design — a generative model for knowledge plus a small classifier for decisions — is attractive to platforms that need low-latency, low-cost inference and interpretable components. The paper's error analysis also warns that probabilistic, unverifiable LLM outputs can inject misinformation, stereotypes, or false associations directly into a moderation pipeline, implying that production deployments need validation, human review for high-impact decisions, and clear disclaimers.

Future Directions

• Test other LLMs, prompts, and temperatures. The study used only Gemini 2.0 Flash with a fixed set of simple natural-language prompts; alternative models or prompt formulations could produce very different context and downstream performance.

• Evaluate the generated context directly. The paper measures context only through its effect on downstream classification, and suggests future work should assess context quality in its own right.

• Extend beyond English and beyond short time spans. Both datasets are English-language and were collected over relatively brief periods, leaving multilingual, non-English, and temporal generalisability open, especially given how quickly online references change.

• Develop finer-grained evaluation and challenge sets. Reliance on macro F1 can hide class-specific trends and failure modes; the authors call for more fine-grained analysis and dedicated challenge datasets, along with better multimodal named entity recognition, given the low entity recall in memes.

Target Audience

Researchers and graduate students working on hate speech detection, implicit meaning, and multimodal content moderation; NLP practitioners building moderation pipelines who want a lightweight classifier augmented by LLM-generated context; and anyone interested in how generative models can be used as knowledge sources rather than as end-task classifiers. The methodology sections are readable for readers with a general machine learning background, while the error analysis and discussion are useful for those focused on fairness, safety, and ethical deployment of automated moderation.

Authors’ abstract

This paper investigates the use of an LLM to generate auxiliary background context for social media posts, and explores four methods to incorporate this context into the input of an SBERT-based Hate Speech Detection (HSD) classifier. These are: text concatenation, embedding concatenation, a hierarchical transformer-based fusion, and LLM-driven text enhancement. We evaluate the impact of our context generation and incorporation strategies in a textual setting on the Latent Hatred dataset of implicitly hateful tweets and a multimodal setting on the MAMI dataset of misogynous internet memes. Results are evaluated against a zero-context baseline, two previous approaches based on entity linking, and a zero-shot LLM classifier. Findings indicate that incorporating generated context improves HSD performance by up to 3 and 6 F1 points on textual and multimodal settings respectively, from a zero-context baseline to the highest-performing system, based on embedding concatenation.