Mapping Similarity Spaces across Embedding Models with Synthetic Query Probing

AI summary

Overview

Research area: Natural Language Processing — retrieval-augmented generation, text embedding models, and similarity score calibration.

Technical level: Intermediate. The paper assumes familiarity with vector embeddings, cosine similarity, and retrieval thresholds, but its arguments are framed around practical system behavior rather than deep mathematical machinery.

Scope: The paper introduces Synthetic Query Probing, an annotation-free method for measuring how cosine similarity scores differ across embedding models and corpora, and shows that learned mappings — especially isotonic regression — can partially align those score spaces so that retrieval thresholds transfer better.

What This Paper Is About

Retrieval-augmented generation pipelines retrieve document chunks by comparing embedding vectors and keeping those whose similarity score exceeds a threshold. The problem is that a similarity score of, say, 0.4 means something different depending on which embedding model produced it and which corpus it was computed over, so thresholds cannot be carried over when a system migrates from one model to another. This paper asks whether those score spaces can be systematically related by learning conversion functions between score distributions — rather than between embeddings — and it builds a method called Synthetic Query Probing to gather the paired scores needed to fit and test those conversions.

Key Contributions

1. Problem formulation. The authors formalize the lack of similarity score comparability across embedding models as an underexplored challenge, arguing it limits both fundamental understanding of embedding spaces and practical tasks such as threshold calibration and model migration.

2. Synthetic Query Probing (SQP). A scalable, reference-free method that generates controlled query–document pairs from document chunks using an LLM, at three levels of semantic relatedness (PARAPHRASE, RELEVANT, IRRELEVANT), so that cross-model similarity behavior can be analyzed without human annotation.

3. Cross-model similarity analysis. An empirical characterization of how similarity distributions differ across four embedding configurations, showing systematic distortions in scale and dispersion even though ranking order is preserved.

4. Score calibration and transfer. A demonstration that these distortions can be modeled with learned transfer functions — linear regression, isotonic regression, and quantile (CDF) mapping — with isotonic regression performing best, plus a precision-first threshold analysis quantifying how thresholds must shift across models and corpora.

Main Findings

• Ranking order is preserved, absolute scores are not. In every configuration, PARAPHRASE scores highest, followed by RELEVANT, then IRRELEVANT, and this holds across mean, median, and all quartiles — but the scale and spread of the scores differ substantially between models.

• Titan dimensionalities form a consistent gradient. As Titan dimensionality decreases, all class means shift upward. SciFact Titan 256 shows a PARAPHRASE mean of 0.54 versus 0.52 at both 512 and 1024, and an IRRELEVANT mean of 0.06 versus 0.02 at 1024.

• Ada-002 occupies a fundamentally different score space. Its scores are compressed into a narrow high range, with standard deviations two to six times smaller than the Titan variants, and the gap between IRRELEVANT and RELEVANT means two to four times smaller. On SciFact, Ada's full score band is reported as 0.626–0.930; on the enterprise corpus it is 0.603–0.963.

• Class overlap is worst for Ada on enterprise data. The maximum IRRELEVANT score for Ada exceeds the PARAPHRASE mean on the enterprise corpus, while on SciFact Ada's IRRELEVANT scores remain separated from the positive classes.

• Thresholds are roughly six times higher on the enterprise corpus. To reach precision of at least 0.95 on SciFact, Titan requires thresholds of 0.063 (1024-d), 0.090 (512-d), and 0.144 (256-d). On the enterprise corpus the same operating point demands 0.393, 0.400, and 0.439.

• Ada requires much higher absolute thresholds. At the 0.95 precision operating point, Ada needs 0.718 on SciFact (versus Titan's 0.063–0.144) and 0.821 on the enterprise corpus (versus Titan's 0.393–0.439).

• Recall behavior is corpus-dependent. On SciFact, Ada achieves perfect recall of 1.000 against Titan's 0.981–0.996. On the enterprise corpus the situation reverses and Titan variants retain higher recall at equivalent precision.

• False positive counts stay comparable within a corpus. At precision 0.95, all configurations produce 96–104 false positives on SciFact and 72–79 on the enterprise corpus, despite very different threshold scales.

• Cross-dimension mapping is near-lossless. Titan-to-Titan mappings reach R² of at least 0.967 on SciFact (MAE ≤ 0.035) and at least 0.980 on the enterprise corpus (MAE ≤ 0.023). The 512↔1024 pair is tightest, at R² = 0.990 on SciFact and 0.996 on enterprise.

• Cross-model mapping is noisier and non-linear. Titan↔Ada pairs show an S-shaped relationship — at Ada scores around 0.65, Titan scores are widely dispersed (roughly 0.0–0.4), while at Ada scores around 0.90 the Titan scores converge to roughly 0.5–0.8. On the enterprise corpus these mappings reach R² of only 0.820–0.918 with isotonic regression; on SciFact they reach 0.848–0.945 for Titan→Ada and 0.85–0.89 for the harder Ada→Titan direction.

• Isotonic regression wins consistently. Across all evaluated pairs and both corpora, isotonic regression delivers the highest R² and lowest MAE, whereas linear OLS systematically misses the S-shaped non-linearity in Titan–Ada conversions.

• Directional error asymmetry. MAE for Ada→Titan conversions is roughly three to four times higher than for Titan→Ada — for example 0.073 versus 0.019 on the enterprise corpus and 0.060 versus 0.014 on SciFact for the 1024↔Ada pair. The authors attribute this to Titan operating over a wider score range, which amplifies absolute error even when fit quality in R² is identical.

• Two regimes of conversion difficulty. Changing dimensionality within a model family has relatively little effect on optimal thresholds, while changing the corpus induces large shifts — implying thresholds are primarily corpus-dependent and per-corpus calibration remains necessary even with a fixed model.

Methodology in Plain English

The authors partition a corpus into chunks, sample a subset of those chunks, and use a large language model to write questions about each sampled chunk. Each chunk gets questions of three kinds: ones fully answerable from the chunk (PARAPHRASE), ones topically related but needing outside context (RELEVANT), and ones from unrelated domains (IRRELEVANT). These generated pairs carry reliable relevance labels without any human annotation.

Each question–chunk pair is then embedded and scored with cosine similarity under several embedding configurations, producing a table where every pair has a score from every model. The authors then do three things with that table. First, they describe the distributions of scores per class per model. Second, they sweep thresholds across 500 evenly spaced candidates and measure precision, recall, and F1, treating PARAPHRASE plus RELEVANT as the positive class and IRRELEVANT as negative, to find the thresholds needed for precision of at least 0.93, 0.95, and 0.97. Third, because every pair carries scores from all four models, they treat each pair of models as a paired regression problem and fit twelve directional conversion functions using linear ordinary least squares, isotonic regression, and quantile (CDF) mapping, comparing them with MAE and R².

The evaluation uses the SciFact corpus (5,183 life-science and biomedical abstracts) and a proprietary enterprise corpus (114,648 chunks from 33,492 Pegasystems documents). From each corpus, 100 chunks are sampled and Claude Sonnet 4.6 generates 10 questions per chunk for each of the three relevance classes, yielding 3,000 question–chunk pairs per corpus. Embeddings come from Amazon Titan Text Embeddings V2 at 256, 512, and 1024 dimensions and OpenAI text-embedding-ada-002 at 1,536 dimensions.

Why This Matters

Impact on research. The paper reframes embedding evaluation away from benchmark leaderboards and toward the score distributions that actually govern threshold decisions in deployed systems. It connects to prior findings that benchmark rank is a weak guide for embedding selection in RAG (Caspari et al.) and to work on calibration to human judgments (Tacheny), and it argues that cross-model score distributions had not previously been compared systematically across multiple corpora.

Real-world applications

• Model migration in production RAG. When a team swaps an embedding model or changes embedding dimensionality, learned conversion functions let existing thresholds be mapped to the new model instead of being re-tuned from scratch.
• Threshold initialization for retrieval systems. The precision-first threshold tables show what operating points are needed for a target precision level, and warn that reusing a 1024-d threshold at lower dimensionalities causes a noticeable precision drop.
• Corpus drift monitoring. Separation statistics derived from SQP data can act as an early indicator that a corpus has shifted enough to warrant recalibration, before conversion quality actually degrades.
• Multi-model and multi-tenant retrieval. Systems that serve several embedding configurations or customer-specific corpora can use these mappings to make scores and thresholds comparable across them.

Industry relevance. The two authors are affiliated with Pegasystems and Leiden University, and the method is deliberately annotation-free, scalable, and configurable, with one evaluation corpus drawn from a real enterprise knowledge base of product guides, API references, knowledge-base articles, marketing materials, and community content. That framing targets practitioners who need an operational answer to model migration rather than a new benchmark score.

Future Directions

• Extending SQP to newer embedding models and a wider, more diverse set of corpora, since the current study covers only two corpora and four configurations.
• Modeling conversion quality directly from corpus statistics and downstream task performance, so that expected conversion reliability can be predicted before fitting any transfer function.
• Using SQP for continuous monitoring of corpus drift to trigger automatic re-calibration of thresholds.
• Comparing different methods and models for the question generation step within SQP itself, alongside addressing the acknowledged limitations: potential bias from synthetic queries, the assumption of normalized embeddings, static thresholds, and single-stage retrieval without re-ranking, and the fact that conversion functions were fitted on the full dataset without a train–test protocol.

Target Audience

Engineers and applied researchers who build or maintain retrieval-augmented generation systems and need to migrate between embedding models, change embedding dimensionality, or reuse similarity thresholds across corpora. It also suits NLP researchers interested in the geometry of embedding spaces, cross-model representational comparison, and calibration methods such as isotonic regression and quantile mapping. Readers looking for a new embedding benchmark or state-of-the-art retrieval results will not find them here — the contribution is an evaluation and calibration framework, not a model.

Authors’ abstract

Retrieval-Augmented Generation systems rely on similarity scores to retrieve relevant content, yet scores are not directly comparable across embedding models due to differing geometric properties, complicating model migration and limiting threshold reuse. We study how similarity scores can be related by learning mappings between score distributions rather than embeddings. We introduce Synthetic Query Probing, generating queries from documents to create controlled query-chunk pairs, enabling large-scale, reference-free analysis of cross-model similarity behavior. We evaluate the approach on multiple embedding configurations and learn score conversion functions using linear, isotonic, and quantile mappings. Experiments on SciFact and a proprietary corpus show that while models largely agree on rankings, their absolute scores exhibit systematic distortions. Learned mappings partially align these spaces and improve threshold portability, with isotonic regression performing best. Our results highlight the need for cross-model calibration and position Synthetic Query Probing as a scalable framework for analyzing embedding comparability.