Investigating Social Bias Changes in Quantized Language Models

AI summary

Overview

• Research area: Natural Language Processing — social bias evaluation of compressed (post-training quantized) large language models.
• Technical level: Advanced. Assumes familiarity with quantization schemes (RTN, GPTQ, AWQ, SmoothQuant), logit-based response selection, permutation testing, and bias benchmark conventions.
• Scope: A large-scale paired, response-level study of how post-training quantization changes social bias across 50 quantized models and 13 bias datasets.

What This Paper Is About

Post-training quantization (PTQ) is the standard way to shrink LLMs so they fit on cheaper hardware, but prior work disagrees about whether it makes them more or less socially biased, largely because each study used different models, datasets, and scoring rules. This paper builds a unified benchmark, PostTrainingBiasBench (85K questions, described in Figure 1), that pairs every pre-quantization response with its post-quantization counterpart so changes can be measured directly rather than compared across disjoint setups. The goal is to determine what quantization actually does to bias at the level of individual responses and individual social groups, not just in aggregate scores.

Key Contributions

1. PostTrainingBiasBench: A unified framework for evaluating bias changes from post-training modifications, standardizing response extraction and bias metric comparison with rigorous pairwise statistical testing (permutation-style bootstrap tests). It is used to conduct the largest systematic study of bias changes in 50 quantized models across 9 closed-ended and 4 open-ended datasets.
2. Empirical discovery of quantization-induced bias flipping driven by uncertainty: Up to 21% of responses flip between biased and unbiased states after quantization while aggregate metrics remain unchanged. This flipping correlates with model uncertainty and quantization strength, but not with model size. A preference-tuning experiment provides evidence for a causal link between increasing pre-quantization uncertainty and response flipping.
3. Evidence of asymmetric social group impacts: Aggregate metrics can appear neutral while specific groups change by as much as −14% to +18.6% within the same model.
4. Deployment-relevant guidance: Quantization method and bit-width matter more than parameter count; behavior-flipping rates averaged 2% for RTN W8A16 versus 9% (GPTQ W4A16), 11% (AWQ W4A16), 12% (RTN W4A16), and 13% (RTN-SmoothQuant W4A16).

Main Findings

• Uncertainty predicts flipping: Responses with high uncertainty (entropy > 0.66) flip 10–20% of the time across datasets, while low-uncertainty responses (entropy < 0.33) rarely change (<2% for most datasets). BBQ shows the most dramatic pattern, with 21% of high-uncertainty responses changing post-quantization, whereas SocialStigmaQA (entropy ≡ 0, near-certain "cannot answer" responses) shows <1% flipping. The abstract states high-uncertainty responses are 3–11× more likely to change than confident ones.
• Uncertainty distribution stays stable: Although individual responses flip, the overall distribution of model uncertainty across questions is largely unchanged post-quantization; the authors conclude quantization redistributes uncertainty rather than systematically increasing or decreasing it.
• Stronger quantization amplifies uncertainty change: RTN W8A16 shows minimal deviation from baseline, while RTN W4A16 exhibits 2–3× larger variance in uncertainty changes, with entropy shifts of up to 0.25 points in Credit, StereoSet, and BBQ. Per the abstract, 4-bit models show 4–6× more behavioral changes than 8-bit.
• Aggregate metrics hide most of the action: Permutation tests flag 17.8% of quantization-induced changes as significant, dropping to 11.4% after multiple-testing correction. Up to 41% of cases show behavioral changes, led by BiasLens-Choice, while Adult, Credit, StereoSet, and BBQ show negligible effects. Changes are bidirectional and effect sizes are zero-centered.
• Flipping without aggregate change: 13–14% of responses flipped on IAT and BBQ, and 21% flipped in FMT10K, even where aggregate bias metrics did not change significantly. The abstract reports up to 21% flipping overall despite no change in aggregate scores.
• Open-ended generation is volatile: The widest Cohen's d distributions occur in open-ended datasets: CEB-Continuation (−2.5 to 2), CEB-Conversation (−2.28 to +3.7), BiasLens-GenWhy (−3.7 to 2.5), and FMT10K (−3.9 to 3.14).
• No scaling advantage: Across Qwen 2.5 variants (0.5B through 14B), behavior flipping shows no monotonic relationship with size. Qwen 2 7B shows among the lowest rates (2%), while similarly sized LLaMA 3.1 8B and Ministral 8B show much higher rates (7% and 9%). The discussion notes 14B models show similar or worse stability than 0.5B models.
• Rankings shuffle after quantization: On FMT10K, original models rank LLaMA variants as least biased with Qwen 2.5 14B (ranks 1–4) and smaller Qwen models (0.5B to 7B) showing higher bias (ranks 5–8). After RTN W4A16 quantization, Qwen 2.5 3B jumps from rank 5 to 1 and LLaMA 3.2 1B drops from rank 2 to 4.
• Question-level vulnerability varies widely: Within each dataset, some questions flip as much as 50% of the time post-quantization while others show little to no flipping; the distribution is heavily right-skewed, with most questions flipping less than 25% of the time.
• Asymmetric group impacts: Aggregating across all models shows minor changes (−1.1% for "short" individuals, +1.6% for "male" individuals), but finer granularity reveals large swings. Qwen 2.5 14B variants yield −10.3% for "short" but +7% for "male"; individual model–quantization pairs show −14.1% for "short" (GPTQ W4A16 Qwen 2.5 14B) and +18.6% for "male" (RTN W4A16 Qwen 2.5 0.5B). The abstract states bias can worsen by up to 18.6% for some groups while improving by 14.1% for others.
• Dataset context modulates group effects: For the "male" demographic, the percentage of behavior-flipped responses differs by dataset at 10.5% (BBQ), 2.1% (BiasLens-GenWhy), and 18% (FMT10K).

Methodology in Plain English

The authors gathered 13 bias datasets covering three capabilities: identifying biased or toxic text (CEB-Recognition, Jigsaw), producing equal outcomes when the context is informative (Adult, Credit), and preferring refusal or uncertainty under biased prompts (BiasLens-Choices, SocialStigmaQA, BBQ ambiguous-context subset, an IAT adaptation with 13,858 unique questions, StereoSet intersentence). Four datasets evaluate unconstrained generation (BiasLens-GenWhy, CEB-Continuation, CEB-Conversation, and the Interference Misinformation subset of FMT10K).

They evaluated 10 instruction fine-tuned models — LLaMA 3.1 8B, LLaMA 3.2 1B/3B, Ministral 8B, Qwen 2 7B, and Qwen 2.5 0.5B/1.5B/3B/7B/14B — each compressed with 5 PTQ strategies (RTN at W4A16 and W8A16, GPTQ W4A16, AWQ W4A16, SmoothQuant W4A16), producing the 50 quantized models. Figure 1's caption describes this as 10 models in 5 quantized formats (60 models) over 85K questions; the abstract and body refer to 50 quantized models.

For closed-ended datasets, instead of relying on next-token selection or parsing generated text (both known to be fragile), they compute each option's geometric mean token probability from unscaled logits at temperature 1 and pick the option with the lowest perplexity — the same approach used in lm-eval. For open-ended datasets, they use greedy decoding with top_k=1 (temperature 0), capped at 512 tokens, except FMT10K which runs 5 turns at 150 tokens per turn. Model-specific chat templates are used everywhere except CEB-Continuation and CEB-Conversation. Biased generated responses are identified with LLaMA Guard 3 8B under the MLCommons hazards taxonomy.

Because responses exist both before and after quantization, the authors can pair them. They distinguish response flipping (selection changed), bias flipping (a move between biased and unbiased answers), and behavior flipping (a statistically significant change in an aggregate metric). Uncertainty is measured as normalized Shannon entropy over the geometric mean probabilities. Significance uses permutation-style bootstrap tests with 1000 null simulations, two-tailed p-values, Cohen's d effect sizes, and Benjamini–Hochberg FDR control at α = 0.05. Social groups for BiasLens and FMT10K were extracted with gpt-4o-2024-08-06. Total scale: 5.1M responses.

Why This Matters

The paper's impact on research is to reframe how quantization should be evaluated. It shows that aggregate bias scores are a lossy summary: they can be stable while individual responses flip bidirectionally, and they can look neutral while subgroups move sharply in opposite directions. It also offers a mechanistic handle — uncertainty — as a pre-screening signal, and it undercuts the common heuristic that larger models are safer to compress.

Real-world applications:

• Edge and on-device deployment: Choosing 8-bit over 4-bit quantization for bias-sensitive use, since 8-bit (RTN W8A16) averaged 2% behavior changes versus 13% for RTN-SmoothQuant W4A16.
• Clinical and legal decision support: Where an LLM's apparent neutrality in aggregate may conceal deterioration for a specific demographic group by as much as 18.6%.
• Model selection and procurement: Quantized rankings do not preserve full-precision rankings, so leaderboards built on uncompressed models cannot be used to pick a deployment model.
• Safety auditing pipelines: Subgroup-level, task-specific evaluation is needed rather than benchmark-level aggregate reporting.

Industry relevance centers on a concrete recommendation: practitioners should prefer 8-bit over 4-bit quantized LLMs, run task-specific rather than benchmark-only evaluations, and prioritize subgroup-level over dataset-level metrics.

Future Directions

• Establishing a full causal link: The preference-tuning experiment in Section A.11 is described as illustrative and may alter latent model properties beyond uncertainty; the authors state that a full causal link remains future work.
• Beyond English and hypothetical scenarios: Extending to conversational deployment contexts, multilingual settings, and intersectional analyses, which the current benchmark set does not cover.
• Better bias detection tooling: LLaMA Guard 3 8B showed poor precision on BiasLens-GenWhy and CEB-Continuation (PPV 40–55%, NPV 88%), so reported flipping rates on those datasets should be treated as upper bounds; improved automatic detectors would tighten these estimates.
• Non-deterministic generation: Current analyses rely on deterministic decoding, leaving bias assessment under sampling-based generation unexplored.

Target Audience

Researchers and engineers working on model compression, fairness, and LLM evaluation; practitioners who ship quantized models in high-stakes domains such as healthcare, law, and any setting with resource-constrained inference; and benchmark designers who need to know why aggregate bias metrics can be misleading after post-training modification.

Authors’ abstract

Post-training quantization reduces the memory needed to run large language models but alters their social biases in ways that aggregate metrics fail to capture. We present the first large-scale study of 50 quantized models evaluated on PostTrainingBiasBench, a unified benchmark of 13 closed- and open-ended bias datasets. We identify a phenomenon we term quantization-induced bias flipping, in which quantization causes models to change responses from biased to unbiased and vice versa, up to 21% of the time, despite no change in aggregate bias scores. These flips are strongly associated with model uncertainty, where the responses with high uncertainty are 3-11x more likely to change than the confident ones. Quantization strength amplifies this effect, with 4-bit quantized models exhibiting 4-6x more behavioral changes than 8-bit quantized models. Critically, these changes create asymmetric impacts across demographic groups, where bias can worsen by up to 18.6% for some groups while improving by 14.1% for others, yielding misleadingly neutral aggregate outcomes. Larger models show no consistent robustness advantage, and group-specific shifts vary unpredictably across model families. Our findings demonstrate that compression fundamentally alters bias patterns, requiring crucial post-quantization evaluation and interventions to ensure reliability in practice.