# Mapping Similarity Spaces across Embedding Models with Synthetic Query Probing

arXiv: https://arxiv.org/abs/2608.05857 · HTML: https://mlsanity.com/paper/2608.05857/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 12 of its 16 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 12 of 12)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Agentic Large Language Models, a survey](https://mlsanity.com/paper/2503.23037/) (2025)
- [Semantics at an Angle: When Cosine Similarity Works Until It Doesn't](https://mlsanity.com/paper/2504.16318/) (2025)
- [Beyond Benchmarks: Evaluating Embedding Model Similarity for Retrieval Augmented Generation Systems](https://mlsanity.com/paper/2407.08275/) (2024)
- [Can we Evaluate RAGs with Synthetic Data?](https://mlsanity.com/paper/2508.11758/) (2025)
- [Calibrated Similarity for Reliable Geometric Analysis of Embedding Spaces](https://mlsanity.com/paper/2601.16907/) (2026)
- [The Harder Text Embedding Benchmark (HTEB): Beyond One-dimensional Static Robustness](https://mlsanity.com/paper/2605.28190/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation systems rely on similarity scores to retrieve relevant content, yet scores are not directly comparable across embedding models due to differing geometric properties, complicating model migration and limiting threshold reuse. We study how similarity scores can be related by learning mappings between score distributions rather than embeddings. We introduce Synthetic Query Probing, generating queries from documents to create controlled query-chunk pairs, enabling large-scale, reference-free analysis of cross-model similarity behavior. We evaluate the approach on multiple embedding configurations and learn score conversion functions using linear, isotonic, and quantile mappings. Experiments on SciFact and a proprietary corpus show that while models largely agree on rankings, their absolute scores exhibit systematic distortions. Learned mappings partially align these spaces and improve threshold portability, with isotonic regression performing best. Our results highlight the need for cross-model calibration and position Synthetic Query Probing as a scalable framework for analyzing embedding comparability.