Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation

AI summary

Overview

Research area: Natural language processing — uncertainty quantification and factuality/hallucination detection in large language model (LLM) long-form text generation, with a focus on text embeddings, cosine kernels, and von Neumann entropy.

Technical level: Intermediate. The paper assumes familiarity with LLM sampling, text embeddings, and entropy, but the core idea is described with accessible geometric intuition and the method itself involves no training or tuning.

Scope: The paper proposes "semantic isotropy" — the angular dispersion of normalized embeddings of independently sampled responses — as a cheap, training-free proxy for nonfactuality in long-form LLM outputs, and evaluates it against existing uncertainty baselines across three datasets scored with a new protocol called Segment-Score.

Paper details: arXiv:2510.21891v2 [cs.CL]; authors Dhrupad Bhardwaj (New York University), Julia Kempe (New York University), and Tim G. J. Rudner (University of Toronto and Vijil). The paper lists equal contribution. Code is available at github.com/dhrupadb/semantic_isotropy.

What This Paper Is About

Deploying LLMs in high-stakes domains requires knowing whether a long, open-ended answer is trustworthy, but existing approaches rely on claim-by-claim fact-checking, which is expensive and brittle, needs structured references or labels, and scales poorly with response length. The authors ask whether a much cheaper signal — how widely the embeddings of several sampled responses to the same prompt spread out on the unit sphere — can predict factual inconsistency. Their goal is a lightweight first-pass trustworthiness signal that requires no labels, no fine-tuning, and no hyperparameter search.

Key Contributions

1. Introduces semantic isotropy and semantic isotropy scoring. Semantic isotropy is defined as the degree of uniformity across normalized text embeddings on the unit sphere; the paper formalizes a set of responses as semantically isotropic when the cosine kernel of their embeddings equals the identity matrix, and measures the level of isotropy as the von Neumann entropy of the cosine kernel normalized by log N. The resulting score lies between 0 and 1, where lower values mean tighter alignment (more trustworthy) and higher values mean greater dispersion (less trustworthy).

2. Develops the Segment-Score protocol and dataset. Segment-Score segments a response into atomic segments with an oracle LLM and verifies each segment against a reference document in context, returning the fraction of segments verified as true. Using it, the authors create a dataset of 1,182 unique entities with approximately 65,450 scored responses from three models, each response containing between 25 and 60 distinct claims.

3. Demonstrates that semantic isotropy scores are a strong proxy for factual inconsistency across a range of generative models, embedding models, and experimental settings, outperforming existing aggregate trust signals.

4. Releases code to reproduce the results and implement Segment-Score, with the Segment-Score-annotated dataset to be made publicly available.

Main Findings

• Isotropy beats existing baselines: Under the Segment-Score algorithm, semantic isotropy scores are superior to existing methods in predicting nonfactuality on FactScore-Bio (FS-BIO), TriviaQA Entities, and CMU Book Summaries (BS), outperforming all existing metrics "in most cases by a wide margin."

• Results hold under a second scoring scheme: Repeating the evaluation on FS-BIO using the FactScore (FS) algorithm produced results mirroring the Segment-Score results, and in fact "even improving the predictive performance of semantic isotropy scoring." The authors read this as evidence that results are not an artifact of Segment-Score.

• Example numbers from the FactScore-Bio / FactScore setting (R², with 1-SD error bars from 1,500 bootstrapped samples): LUQ reaches 0.657 ± 0.07 (Llama 3.1 8B), 0.367 ± 0.06 (Phi 3.5 Mini), and 0.192 ± 0.06 (GPT 4.1 Mini). Semantic isotropy with Cohere v3.0 reaches 0.75 ± 0.06, 0.665 ± 0.05, and 0.503 ± 0.08; with Cohere v4.0, 0.747 ± 0.06, 0.671 ± 0.05, and 0.5 ± 0.08; with Nomic v1, 0.737 ± 0.05, 0.6 ± 0.05, and 0.547 ± 0.08; and with Gemini, 0.7 ± 0.06, 0.661 ± 0.05, and 0.48 ± 0.08.

• The choice of embedding model matters but not dramatically: Semantic isotropy consistently outperforms LUQ across all three datasets/scoring methods and embedding models; even the least effective embedding model yields improvements over LUQ. Nomic v1 performed exceptionally well on all datasets, occasionally surpassing larger and closed-source models such as Gemini and OpenAI Embeddings. Comparing small and large variants of closed-source models from OpenAI and Cohere, the performance difference was negligible and likely attributable to sampling noise.

• Discriminative power at a concrete threshold: When factuality is discretized at a 50% Segment-Score threshold (labels 1 and 0, with 1 − ℐ as the confidence score), the AUC reaches approximately 0.95 on Book Summaries (Phi 3.5 Mini) and 0.90 on FS-BIO (Llama 3.1).

• Large computational savings: Computing isotropy scores for one batch of N = 20 responses of roughly 500 words takes 1.8 ± 0.05 seconds on a V100 (amortized, Nomic v1), compared with 302 ± 48 seconds for LUQ-Atomic using DeBERTa-v3 Large (MNLI) — approximately 170× faster. LUQ and Graph Uncertainty scale as O(MN²) in the number of facts/segments M and samples N; semantic entropy and related methods are O(N²) in entailment computations and cannot be vectorized naively; semantic isotropy is also O(N²) in kernel construction but the cosine kernel is fully parallelizable and the von Neumann entropy of an N × N matrix with N ≤ 20 is negligible.

• Robustness to the isotropy measure: Ablating alternative isotropy measures (the Frobenius norm ‖K_E^cos‖, log(det(K_E^cos)), and trace(K_E^cos)^{-1}), von Neumann entropy performed best overall on average, though most measures performed nearly equivalently.

• Robustness to response length: Performance is generally consistent across response lengths, with some models performing better (Gemini and Nomic v1). OpenAI's Text Embedding 3 is a notable exception, with performance consistently increasing with response length, independent of model size.

• Segment-Score shows high agreement with oracle labels: On a random sample of 30 entities per dataset, with up to 5 sampled True/False labels per segment from GPT 4.1, Claude 4 Sonnet, and DeepSeek, label agreement was 89.24% / 81.33% / 75.87% for BS with Llama 3.1; 91.28% / 86.13% / 80.38% for BS with Phi 3.5 Mini; 83.92% / 86.13% / 86.39% for FS-BIO with Llama 3.1; and 76.98% / 86.67% / 88.30% for FS-BIO with Phi 3.5 Mini.

• Two named methods were excluded: Kernel Language Entropy (KLE) was found to be highly numerically unstable given the degree of semantic entailment overlap among sampled long-form responses, and Graph Longform Uncertainty's distinct claim union algorithm was computationally difficult to implement at the response lengths studied — a typical set of responses had over 600 distinct claims, implying 6,000 LLM calls for one topic assuming N = 10.

• Known failure modes: The signal weakens for generators whose outputs remain stylistically uniform even when hallucinating, as observed with GPT 4.1 Mini on TriviaQA, and is uninformative for intrinsic hallucinations that remain self-consistent across samples.

Methodology in Plain English

The intuition is geometric. If a prompt has one factually grounded answer, independently sampled responses from a model should look similar to each other and cluster tightly when converted into embedding vectors. If the model is making things up, different samples drift in different directions, and the vectors spread out over the sphere.

Concretely, the pipeline works in four steps. First, for a given prompt, sample N independent responses from the generative LLM. Second, pass each response through an off-the-shelf embedding model to get a vector in R^D, forming an N × D matrix E. Third, normalize the vectors and build an N × N cosine kernel matrix K_E^cos, whose entries are pairwise cosine similarities. Fourth, compute the von Neumann entropy of that kernel (normalized to trace 1) and divide by log N. That ratio is the semantic isotropy score, bounded in [0, 1]: it hits log N (score 1) for perfectly isotropic, uniformly dispersed embeddings and 0 when all embeddings are aligned and parallel. A low score means tight agreement and higher trustworthiness; a high score means dispersion and lower trustworthiness.

To test whether this score tracks actual factuality, the authors needed long-form ground truth. They built Segment-Score: an oracle LLM splits a response into atomic segments, then each segment is verified against a reference document in the context of the preceding segments, and the score is the fraction of segments labeled true. They note this is designed to be more token-efficient than prior approaches, to scale to longer responses, and to offer clearer criteria for labeling statements true or false.

Evaluation uses the R² (explained variance) of a simple linear model with factuality scores as the dependent variable and isotropy scores as the independent variable, with 1-SD error bars from 1,500 bootstrapped samples. Baselines include Perplexity, LN-Entropy, U.Eign, U.Deg, U.NLI, Semantic Entropy, LUQ-Atomic, and EigenScore, each implemented with its original backbone (DeBERTa-v3 Large (MNLI) for the NLI components of LUQ, SelfCheckNLI, and semantic entropy; internal-state activations for EigenScore).

Data construction: three sources, all using Wikipedia as the ground truth for reference documents — FactScore-Bio (182 unique entities), TriviaQA (1,000 unique entities), and CMU Book Summaries (509 unique entities). For TriviaQA, entities corresponding to days, dates, or numerical values were excluded and only those matching the underlying Wikipedia page title were kept; of 5,245 qualifying entities across the training and validation set, 1,000 were randomly selected. For Book Summaries, the dataset was constrained to entities whose reference text is at least 5,000 characters with a known publish date on or after Jan 1, 2000; of 16,559 entries, 509 met the criterion.

Generation used Meta Llama 3.1 8B Instruct, Microsoft Phi 3.5 Mini Instruct, and OpenAI GPT 4.1 Mini for TriviaQA and FS-BIO; for BS, only Llama 3.1 and Phi 3.5 Mini were used. TriviaQA used up to k = 20 samples per prompt and FS-BIO and BS used k = 10, targeting approximately 500 words per response. A longer TriviaQA dataset of approximately 1,000 words was also generated with Llama 3.1 8B Instruct, with intermediate datasets of approximately 125, 250, 375, 500, and 750 words derived by truncating to the nearest sentence. Inference ran at temperature τ = 0.7 with FP16 quantization using the vLLM framework. GPT 4.1 Mini served as the oracle LLM for both Segment-Score and FactScore.

For open-weight embedding models, the authors took activations of the last token in the final hidden state, except for Nomic v1, where they averaged activations over the token dimension.

Why This Matters

Impact on research. The paper offers an aggregate, distribution-level factuality signal that avoids claim-by-claim verification entirely, and it shows the signal survives across three datasets, two scoring protocols, many embedding models, and multiple response lengths. It also contributes a new scoring protocol (Segment-Score) and a dataset of 1,182 entities and roughly 65,450 scored responses for future benchmarking, plus code for reproduction.

Real-world applications:

• First-pass triage in LLM deployments: high-isotropy prompts can be routed to expensive claim-level verification while low-isotropy ones pass with minimal additional checking.
• High-stakes domains named in the impact statement: healthcare, law, and education, where untrustworthy long-form output carries real risk.
• Cheap monitoring of open-ended generation at scale: the method needs no labels, no fine-tuning, and no hyperparameter selection, and works with open- or closed-weight embedding models.
• Complement to existing fact-checking pipelines: the authors position it as a complementary early-warning signal rather than a replacement for comprehensive fact-checking.

Industry relevance. The cost profile is the headline: 1.8 ± 0.05 seconds per batch of N = 20 responses of approximately 500 words versus 302 ± 48 seconds for LUQ-Atomic, roughly 170× faster, using a fully parallelizable cosine kernel. For teams already sampling multiple responses from a model — a common practice — the marginal cost of a trustworthiness score is small. The authors also note the repeated-sampling overhead is decreasing with modern high-throughput inference engines and reasoning-oriented LLMs.

Future Directions

• Combining isotropy with claim-level verification: the authors state the method is prompt-level and does not localize errors to particular claims, so pairing the two is the natural next step — routing high-isotropy prompts to segment-level checks while low-isotropy ones pass.
• Addressing style-uniform hallucination: the signal weakens when a generator's outputs stay stylistically uniform even while hallucinating (observed with GPT 4.1 Mini on TriviaQA) and is uninformative for intrinsic hallucinations that remain self-consistent across samples; the paper does not report a solution.
• Revisiting the excluded baselines: KLE was set aside for numerical instability and Graph Longform Uncertainty for computational difficulty at long response lengths, leaving open whether stabilized or more efficient versions of these methods would change the comparison.
• Reducing sampling cost and calibration: repeated sampling remains an added inference cost relative to single-pass probe-based uncertainty estimates, and the paper states threshold calibration is inherently deployment-specific, following standard practice for sampling-based uncertainty scores.

Target Audience

This paper is most useful to researchers and engineers working on LLM reliability: uncertainty quantification and hallucination-detection researchers, practitioners building evaluation or guardrail pipelines for long-form generation, and teams in high-stakes domains (healthcare, law, education) who need a cheap first-pass trust signal. It is also relevant to readers interested in applying embedding geometry — cosine kernels and von Neumann entropy — to practical model-evaluation problems, and to those who need a segment-level factuality scoring protocol for building their own annotated long-form datasets.

Authors’ abstract

To deploy large language models (LLMs) in high-stakes application domains that require substantively accurate responses to open-ended prompts, we need reliable, computationally inexpensive methods that assess the trustworthiness of long-form responses generated by LLMs. However, existing approaches often rely on claim-by-claim fact-checking, which is computationally expensive and brittle in long-form responses to open-ended prompts. In this work, we introduce semantic isotropy -- the degree of uniformity across normalized text embeddings on the unit sphere -- and use it to assess the trustworthiness of long-form responses generated by LLMs. To do so, we generate several long-form responses, embed them, and estimate the level of semantic isotropy of these responses as the angular dispersion of the embeddings on the unit sphere. We find that higher semantic isotropy -- that is, greater embedding dispersion -- reliably signals lower factual consistency across samples. Our approach requires no labeled data, no fine-tuning, and no hyperparameter selection, and can be used with open- or closed-weight embedding models. Across multiple domains, our method consistently outperforms existing aggregate trust signals in predicting nonfactuality using only a handful of samples, offering a practical, low-cost first-pass signal that complements claim-level verification in real-world LLM workflows.