Calibrating WEAT Against Anisotropy: ZCA Whitening as a Geometric Pre-Processing Step for Embedding Association Tests

AI summary

Overview

Research area: Natural language processing, specifically intrinsic bias measurement in word and sentence embeddings, with cross-disciplinary ties to computational social science and AI fairness.

Technical level: Intermediate. The central idea is geometric (whitening a covariance matrix), and the paper assumes familiarity with cosine similarity, embedding spaces, and the WEAT test procedure. The linear algebra is standard and fully derived.

One-sentence scope: The paper proposes ZCA whitening as a pre-processing step that removes anisotropy from embedding spaces so that the cosine-similarity-based WEAT bias measurement operates under the isotropy assumption it was designed for, and evaluates this across 7 models and 10 WEAT test suites (70 model-task combinations).

What This Paper Is About

The Word Embedding Association Test (WEAT) measures bias by comparing cosine similarities between target word sets (for example, European-American vs. African-American names) and attribute word sets (for example, pleasant vs. unpleasant words). This only works as intended if the embedding space is approximately isotropic, meaning variance is spread roughly evenly in all directions. Prior work has shown that many widely used language models violate this assumption, with contextualized models such as GPT-2 producing near-unity pairwise cosine similarities. The paper's goal is to restore the geometry WEAT assumes by applying ZCA whitening, which transforms the embedding covariance toward the identity matrix while perturbing the original vectors as little as possible, and then to quantify how much bias measurements actually change as a result.

Key Contributions

1. A calibration method for embedding association tests. The authors propose ZCA whitening as a pre-processing step for WEAT and show it substantially reduces anisotropy across all seven evaluated models, regardless of each model's initial degree of distortion.

2. Evidence that calibrated spaces better capture semantic associations. Using WordSim-353, SimLex-999, and STS-B, the paper shows that cosine similarity in the whitened space more faithfully reflects human similarity judgments, especially for highly anisotropic models.

3. A systematic demonstration that anisotropy distorts bias measurement in both directions. Across 70 model-task combinations, the authors show calibration reduces measured associations in some configurations and increases them in others, meaning uncalibrated WEAT can both overestimate and underestimate encoded associations.

4. A reported distribution of significance changes by bias category and architecture family, connecting where distortion appears (race vs. valence tests, contextualized vs. static vs. contrastive models) to the geometry of the embedding space.

Main Findings

• Anisotropy is severe in contextualized models, milder in static ones. Mean pairwise cosine similarity across 2,000 sampled vectors was 0.997 (SD 0.003) for GPT-2, 0.954 (SD 0.024) for RoBERTa, 0.821 (SD 0.068) for BERT, 0.475 (SD 0.090) for unsup-RoBERTa, 0.386 (SD 0.094) for unsup-BERT, 0.192 (SD 0.211) for GloVe, and 0.059 (SD 0.074) for word2vec.

• ZCA whitening reduces anisotropy across the board. After whitening, all seven models converged to near-zero mean pairwise cosine similarity (0.001 to 0.040). Explained variance ratios, which were extremely concentrated in the first principal components for contextualized models, flattened substantially for every model.

• Semantic similarity benchmarks improve most for highly anisotropic models. GPT-2's Spearman correlation on WordSim-353 rose from 0.263 to 0.620 (Δ = +0.358, p < 0.001) and on SimLex-999 from 0.097 to 0.411 (Δ = +0.314, p < 0.001). RoBERTa gained on SimLex-999 (0.266 to 0.457, Δ = +0.191) and WordSim-353 (0.446 to 0.559, Δ = +0.113). BERT gained on WordSim-353 (0.560 to 0.658) and SimLex-999 (0.412 to 0.465).

• Some sentence-level scores declined slightly. BERT's STS-B fell by 0.055 (0.633 to 0.578), unsup-BERT's by 0.055 (0.840 to 0.786), and unsup-RoBERTa's by 0.034 (0.845 to 0.811), all significant. word2vec, the least anisotropic model, showed mixed and small changes (|Δ| ≤ 0.027).

• Roughly 30% of WEAT results change significance status. Of 70 model-task combinations, 48 were Stable (68.6%), 12 Disappearing (17.1%), and 10 Emerging (14.3%). Counting also the seven Stable cases with |Δd| > 0.5, 29 of 70 combinations (41.4%) show either a significance change or a substantial effect-size shift.

• Effect-size shifts can be large even when significance does not change. Examples include GPT-2 on W3 (d = 1.24 to 0.57) and unsup-RoBERTa on W6 (d = 1.61 to 0.99).

• Distortion direction depends on bias category. Disappearing cases concentrate in the race tests (W3-W5): 8 of 12, or 66.7%. Emerging cases are most frequent in the valence tests (4 of 10), with the remaining 6 split evenly between gender (3) and health/age (3).

• The most striking single reversal is a masked valence association. GPT-2 on W2 went from d = -0.87 to d = 1.05, and on W9 (health) from d = -0.01 to d = 1.55. GPT-2 on W7 and W8 also flipped sign, from d = -0.58 to 0.69 and from d = -0.54 to 0.81 respectively, indicating that severe anisotropy can invert the apparent sign of an association.

• Static models can inflate measurements despite low anisotropy. The static models contribute 4 of 12 Disappearing cases (33.3%); word2vec on W2 compressed from d = 1.63 to 1.03, and GloVe on W3 dropped from d = 0.95 to 0.24.

• Contrastive models reduce but do not eliminate the problem. unsup-BERT showed Emerging bias on W7 (d = 0.47 to 1.18) and W8 (d = 0.59 to 0.98), with effect sizes comparable to or larger than those in standard contextualized models, which the authors interpret as evidence that contrastive training reduces geometric distortion but does not remove bias encoded in pre-training data.

• Anisotropy severity does not map monotonically onto measurement instability. GPT-2 (mean cosine 0.997) changed significance in 5 of 10 tests, BERT (0.821) in 5, unsup-BERT (0.386) in 4, and word2vec (0.059) in 3, but RoBERTa (0.954) changed in only 1 test and GloVe (0.192) in only 1.

Methodology in Plain English

The authors treat anisotropy as a measurement problem and address it with a standard linear-algebra operation. They first estimate a whitening matrix from a large reference sample: 100,000 embedding vectors drawn from WikiText-103 (vocabulary items for static models, sentences for contextualized and contrastive models). From that sample they compute the mean vector and covariance matrix, then eigendecompose the covariance and rescale each eigenvector direction by the inverse square root of its eigenvalue. Directions with large variance are compressed and directions with small variance are expanded, so the transformed data has identity covariance. This particular form, ZCA, is chosen because among the family of whitening transformations it uniquely minimizes the expected squared distance between the original and transformed vectors, which matters when the goal is to preserve semantic structure.

The pipeline has three stages: estimate the whitening matrix once per model, apply it to the specific word vectors that appear in the WEAT suites (centering each by subtracting the reference mean first), and then compute WEAT effect sizes and permutation p-values in the calibrated space using exactly the same metric as before. The authors note the pipeline is modular and can be applied to any embedding model and any cosine-similarity-based bias metric without modifying the metric. Numerically, they regularize eigenvalues with epsilon = 10^-3 before inversion to avoid blowups from near-zero eigenvalues.

For contextualized and contrastive models, word-level vectors are built following the Aggregated procedure of Bommasani et al. (2020): sample up to 20 sentences containing each target or attribute word from WikiText-103, mean-pool the subword tokens of the word within each sentence, then average across the 20 sentences. Preprocessing retains sentences of 7 to 75 tokens and excludes section headers, yielding up to 200,000 indexed sentences with a fixed random seed. The authors note that Bommasani et al. reported bias evaluation results were fairly stable across 20, 50, and 100 contexts per word, so they adopted 20 to match that protocol and limit computation.

Evaluation runs in three experiments. First, anisotropy is measured as the mean and standard deviation of pairwise cosine similarities among 2,000 randomly sampled vectors, before and after whitening, supplemented by explained-variance plots and similarity heatmaps. Second, semantic quality is tested with Spearman rank correlations on WordSim-353, SimLex-999, and STS-B, with significance from paired bootstrap resampling over 10,000 iterations. Third, WEAT effect sizes and permutation p-values (100,000 permutations) are computed on both raw and whitened embeddings, and each of the 70 combinations is classified as Stable, Disappearing (significant before, not after), or Emerging (not significant before, significant after) at p < 0.05.

Why This Matters

Impact on research. The WEAT and its many descendants (SC-WEAT, SEAT, CEAT, iEAT, Grounded-WEAT, CLIP-based tests, SpEAT, VEAT, SD-WEAT) all rely on cosine similarity and are therefore subject to the same geometric artifacts this paper targets. If over 30% of WEAT results change significance status under calibration, then a substantial fraction of published intrinsic bias measurements may be unreliable, and the paper argues these should be interpreted with caution or re-evaluated with calibrated methods. The authors also note that ML-EAT (Wolfe et al. 2024) correctly identified anisotropy as a threat to validity but aimed at interpretability rather than correction, leaving the calibration problem open.

Real-world applications:

• Bias audits of deployed language models. Teams auditing models for racial, gender, age, or health-related bias would add an isotropy check before applying cosine-based measurements and calibrate when anisotropy is detected.
• Computational social science using historical corpora. Studies tracing stereotypes over a century of text, 200 years of social-group stereotypes, cross-linguistic bias patterns, and temporal shifts in Wikipedia depend on the validity of embedding-based associations.
• High-stakes applied domains. The paper cites prior EAT applications to biomedical research, clinical notes, legal opinions, and court proceedings, where measurement error has direct consequences.
• Benchmark and evaluation design. Benchmark builders for fairness and safety evaluation can adopt the whitening step as a modular pre-processing stage without redesigning the metrics themselves.

Industry relevance. Any organization that produces fairness reports, model cards, or bias benchmarks using embedding association tests inherits the anisotropy assumption. Because the proposed fix is a per-model pre-computation that reuses the same whitening statistics across all test suites, it is inexpensive to integrate into existing evaluation pipelines. The paper's code is available at github.com/seigit/zca-weat.

Future Directions

• Identify what determines the direction of distortion. The authors state explicitly that the mechanism behind the category-level asymmetry (race tests skewing toward Disappearing, valence tests toward Emerging) is not fully clear from their results. They hypothesize that the geometric relationship between target sets and the dominant variance directions of the embedding space differs across bias categories, and that pinpointing the exact factors is important future work.

• Explain why anisotropy severity does not predict instability. RoBERTa (mean cosine 0.954) showed only 1 significance change while GPT-2 (0.997) and BERT (0.821) showed 5 each. The paper suggests this depends on the specific geometry of the embedding space relative to the WEAT stimulus words, but does not resolve it.

• Re-evaluate and re-interpret existing published bias findings. The paper calls for annotated caution toward previously reported measurements in anisotropic spaces. Whether prior computational social science results survive recalibration is an open empirical question the paper raises but does not answer.

• Extend calibration to the broader EAT family. Because all cosine-based variants share the same geometric vulnerability, applying the calibration step to sentence-level, multimodal, speech, and video-based association tests is a natural extension that the paper sets up but leaves untested.

Note on limits of the available content: the supplied text is truncated in Section 6 (Discussion), so the paper's stated limitations, the remainder of the discussion, and the appendices containing full numeric results (Appendix C) and the whitening derivation (Appendix A) are not reported here.

Target Audience

This paper is most useful to AI fairness and NLP evaluation researchers who build or consume embedding-based bias benchmarks; computational social scientists who apply EATs to corpora and need to defend the validity of their measurements; and machine learning engineers responsible for bias auditing pipelines in industry. It also speaks to methodologists interested in the geometric assumptions underlying similarity-based metrics. Readers need comfort with vectors, covariance, and p-values, but not deep expertise in linear algebra, since the whitening derivation is given in full.

Authors’ abstract

We propose Zero-phase Component Analysis (ZCA) whitening as a geometric pre-processing step for the Word Embedding Association Test (WEAT). WEAT is a bias measurement method widely used in both computational social science and AI fairness research. It relies on cosine similarity as a measure of semantic association, which assumes that the embedding space is approximately isotropic. However, prior work has reported that many widely used language models do not satisfy this assumption, raising concerns about the reliability of bias measurements. ZCA whitening transforms the covariance of the embedding space into the identity matrix while minimizing perturbation to the original vectors. This transformation restores the isotropy condition on which WEAT relies. We evaluate our approach on ten standard WEAT test suites and seven models spanning three architectural families, yielding 70 model-task combinations. The results show that ZCA whitening substantially reduces the anisotropy of the embedding spaces across all models. Particularly for highly anisotropic models, we further observe improvements on standard semantic similarity benchmarks, indicating that the calibrated space better captures semantic associations. After calibration, over 30% of WEAT results change significance status, and effect sizes shift in both directions depending on bias category. These shifts suggest that uncalibrated measurements may both overestimate and underestimate the associations encoded in the embedding space. These findings indicate that previously reported bias measurements in anisotropic embedding spaces should be interpreted with caution and may benefit from re-evaluation with calibrated methods. Our approach contributes to restoring the measurement foundation of WEAT across both computational social science and AI fairness research.