How Semantically Stable Are LLM Refusals? Measuring Confusion in Local Safety Boundaries

AI summary

Overview

• Research area: Natural Language Processing, specifically LLM safety alignment, refusal behavior, and over-refusal evaluation.
• Technical level: Intermediate. The paper assumes familiarity with sentence embeddings, token embeddings, next-token probabilities, and perplexity, but each metric is defined explicitly with equations and each pipeline stage is described step by step.
• Scope: The paper introduces "semantic confusion" as a failure mode in which a model accepts one phrasing of an intent but rejects a meaning-preserving paraphrase, and presents a dataset (ParaGuard) plus three token-level metrics (Confusion Index, Confusion Rate, Confusion Depth) for measuring it across model families.

What This Paper Is About

Existing safety evaluations of large language models mostly report global numbers such as false rejection rate (FRR) or compliance rate, scoring each prompt in isolation. This means they can say how often a model refuses a benign prompt, but not whether those refusals are locally consistent — that is, whether the model accepts one paraphrase of an intent while rejecting a near-equivalent one. The paper builds a controlled paraphrase corpus and a model-agnostic measurement pipeline to quantify this local inconsistency, which the authors call semantic confusion, and shows across nine model variants that global FRR hides structure in the refusal boundary.

Key Contributions

1. Formalizes "semantic confusion" as a measurable failure mode of safety-aligned LLMs, defined as contradictory decisions (ACCEPT vs. REJECT) across inputs expressing the same intent, and provides the first framework to quantify it.
2. Introduces ParaGuard, an approximately 10,000-prompt corpus of controlled paraphrase clusters derived from OR-Bench, USEBench, and PHTest, where each of 2,000 seeds yields up to five intent-preserving variants that pass semantic similarity, lexical divergence, and ensemble safety gates.
3. Proposes three model-agnostic, token-level metrics — Confusion Index (CI), Confusion Rate (CR), and Confusion Depth (CD) — that compare each rejected prompt against its top-k nearest accepted neighbors using token-embedding drift, next-token probability shift, and perplexity contrast.
4. Demonstrates empirically, across Llama, Mistral, Qwen, and GPT families, that global false rejection rate and semantic confusion capture different aspects of refusal behavior, separating how often a model refuses from how locally consistent those refusals are.

Main Findings

• FRR and confusion diverge sharply. Mistral 7B has an FRR of 4.65% but a Confusion Rate of 40.88%, while Qwen 2.5 7B has an FRR of 1.90% and a CR of 2.45%. Global FRR alone would make Mistral look nearly error-free.
• Llama 2 models are the most confused and the most uniformly confused. Llama 2 13B reaches CI 0.749 and CR 50.13%; Llama 2 7B reaches CI 0.748 and CR 47.92%. Their low CD (0.028 and 0.029) indicates confusion is spread fairly uniformly rather than concentrated in a few clusters.
• Mistral shows localized inconsistency pockets. Despite low FRR (4.65% and 4.81%), Mistral 7B and Mistral Nemo have the highest CD values reported (0.038 and 0.040), meaning confusion is concentrated in specific semantic regions where small changes trigger inconsistent refusals.
• Qwen is the most stable, and stricter refusal does not imply more confusion. Qwen 2.5 7B has CI 0.683 and CR 2.45%; Qwen 3 8B has CI 0.670 and CR 0.40%. Qwen 3 has higher FRR than Qwen 2.5 (4.66% vs. 1.90%) yet much lower confusion.
• GPT-family models span the middle. gpt-3.5-turbo has the highest CI in the table (0.754) with CR 41.30%, behaving closer to Llama; gpt-4o-mini is closer to Qwen (CI 0.695, CR 6.10%); o3 refuses most often (FRR 8.15%) but has a lower CR (29.70%) than Llama at a comparable CI range.
• Confusion is a token-level phenomenon, not a repackaging of prompt similarity. Comparing token-level confusion against mean sentence-embedding similarity to accepted neighbors shows only a mild positive trend with substantial vertical spread, and token-level confusion stays wide across similarity bins, including [0.9, 1.0].
• High token-level confusion concentrates in dense embedding regions. The t-SNE projection shows a large central region with CI_tok approximately 0.75 to 0.95, dominated by common function words and generic phrasing, with a smooth transition to sparser peripheral regions.
• Confusion concentrates where risk is lowest and similarity is highest. In the cohort analysis, FRR varies little across safe cohorts but is highest for high-risk prompts (9.8% for HiSim–LowLex–HighRisk). Among rejected prompts, that high-risk cohort has the lowest confusion rate (CR_rej approximately 28%), while safe paraphrase cohorts reach around 60% (HiSim–HiLex–LowRisk 59.4%, LowSim–HiLex 64.3%).

Methodology in Plain English

The authors start from three public safety/over-refusal benchmarks (OR-Bench, USEBench, PHTest), normalize and deduplicate them, and sample 2,000 seed prompts with a fixed random seed. For each seed they generate five variants using lexical rewording, register shifts, keyword softening or hardening, and intent-preserving rewrites. Each candidate must pass three gates: cosine similarity of MiniLM-L6-v2 sentence embeddings at least 0.60 (intent retention), character-level Jaccard overlap at most 0.90 (a genuine rewrite rather than a cosmetic edit), and an ensemble safety score between 0.30 and 0.70 combining toxicity detection (unitary/toxic-bert), hate-speech detection (Hatexplain), and zero-shot safety classification (BART-Large-MNLI). This yields roughly 10,000 annotated prompts.

To measure confusion, each prompt is queried against a target model at temperature 0, and responses are labeled ACCEPT or REJECT based on explicit refusal cues such as "sorry", "cannot", "can't", or "not allowed". All accepted prompts are embedded and indexed with FAISS's IndexFlatL2. For every rejected prompt, the pipeline retrieves its top-k (k = 5) nearest accepted neighbors. Each accepted-rejected pair is then compared on three token-level signals: token drift (cosine distance between aligned token embeddings), token probability shift (absolute difference in next-token probabilities), and perplexity contrast (normalized absolute perplexity difference). These are combined into a weighted confusion score with weights (w_d, w_p, w_pi) = (0.4, 0.1, 0.5), chosen by grid search along with a severity threshold of tau = 0.75. Averaging the pair scores over the neighborhood gives the per-prompt Confusion Index; averaging CI over all rejected prompts gives the dataset-level CI, the share of rejections with CI at least tau gives CR, and the standard deviation of CI across rejected prompts gives CD.

Why This Matters

The paper argues that refusal evaluation should not stop at how often a model refuses, because two models with similar false rejection rates can have very different local stability. It gives safety researchers a way to diagnose boundary irregularities that global statistics average away, and it offers developers a practical signal for reducing false refusals while preserving safety. The authors also note that false refusals erode user trust, distort expected product behavior, and push developers to loosen safety constraints, which can reduce overall safety.

Real-world applications:

• Chat assistants and customer support bots: a user rephrasing a benign request and suddenly hitting a refusal is a concrete reliability failure that CI/CR can flag.
• Deployment-time moderation layers: the same pipeline can be applied to guard models to check whether a given guard is locally stable around benign paraphrases rather than only globally accurate.
• Safety tuning and model selection: teams choosing between model versions can use CD to identify whether inconsistency is spread broadly or confined to a few semantic regions, which matters for targeted fixes.
• Red-teaming and audit reports: auditors can report local stability alongside FRR, exposing cases where a model looks safe globally but behaves erratically on near-identical inputs.

Industry relevance follows from the paper's framing: over-refusal is a known cost of alignment, and the metrics require no retraining and depend only on model outputs and token-level traces, so they can be applied to any LLM or guard as a diagnostic.

Future Directions

• Apply CI/CR/CD to jailbreak defenses. The paper proposes this as a new evaluation axis — testing whether a defense that looks strong globally inadvertently destabilizes the local decision boundary around benign paraphrases — but does not report such experiments.
• Use confusion metrics as a training or tuning signal to reduce false refusals, since the authors present the metrics as a practical signal for exactly that purpose.
• Investigate the localized confusion pockets found in the Mistral models (highest CD, 0.038 to 0.040) to determine what distinguishes those semantic regions from the uniformly confused Llama boundary.
• Extend coverage beyond the nine model variants reported, including dedicated guard models (Llama Guard, ShieldGemma, WildGuard) named in the paper's motivation, and report per-source-benchmark breakdowns, which the current results do not provide.

Target Audience

This paper is most useful to LLM safety and alignment researchers, evaluation and benchmark designers, and engineers who operate guardrails or moderation systems in production. It also suits auditors and product teams who need to justify model choice beyond a single refusal-rate number. Readers with basic familiarity with embeddings, perplexity, and next-token probabilities will follow the methodology; the paper labels two authors as equal contributors and is affiliated with Bangladesh University of Engineering and Technology (BUET). The acknowledgment states that AI assistance was used to polish a few sentences and for minor code debugging, with all study design, analyses, figures, and claims authored and verified by the authors.

Authors’ abstract

As safety alignment becomes standard in large language models, refusal behavior has become an important part of model reliability. However, models may still reject benign prompts, especially when the wording resembles risky content. Existing evaluations usually report global scores, such as false rejection rate or compliance rate. These scores are useful, but they treat each prompt independently. As a result, they miss local inconsistency, where a model accepts one phrasing of an intent but rejects a close paraphrase. This makes it difficult to understand whether refusals are only frequent or also semantically unstable. We address this gap by introducing Semantic Confusion, a failure mode that captures contradictory refusal decisions across meaning-preserving paraphrases. We build ParaGuard, a 10k-prompt corpus of controlled paraphrase clusters that keep intent fixed while varying surface form. We also propose three model-agnostic token-level metrics: Confusion Index, Confusion Rate, and Confusion Depth. These metrics compare each rejected prompt with its nearest accepted neighbors using token embeddings, next-token probabilities, and perplexity signals. Experiments across diverse model families show that global false rejection rate can hide important structure in the refusal boundary. Our metrics reveal cases where confusion is spread broadly, cases where it appears only in specific semantic regions, and cases where stricter refusal does not lead to more local inconsistency. These findings show that refusal evaluation should measure not only how often a model refuses, but also how consistently it refuses across nearby paraphrases. This gives developers a practical signal for reducing false refusals while preserving safety.