mlsanityData to 7 Oct 2026

From Topology to Retrieval: Decoding Embedding Spaces with Unified Signatures

arXiv 2511.22150 ↗First posted 27 Nov 2025cs.LG cs.CL cs.IRcited by 4

Not on the map. Why?

Abstract (arXiv)

Studying how embeddings are organized in space not only enhances model interpretability but also uncovers factors that drive downstream task performance. In this paper, we present a comprehensive analysis of topological and geometric measures across a wide set of text embedding models and datasets. We find a high degree of redundancy among these measures and observe that individual metrics often fail to sufficiently differentiate embedding spaces. Building on these insights, we introduce Unified Topological Signatures (UTS), a holistic framework for characterizing embedding spaces. We show that UTS can predict model-specific properties and reveal similarities driven by model architecture. Further, we demonstrate the utility of our method by linking topological structure to ranking effectiveness and accurately predicting document retrievability. We find that a holistic, multi-attribute perspective is essential to understanding and leveraging the geometry of text embeddings.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 54 of its 92 references that are in the map. First posted on arXiv 27 Nov 2025.

Builds on

54 of 92 references.

References · 54

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  5. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  6. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  7. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  8. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  9. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  10. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  11. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  12. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
38 more
  1. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  2. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  3. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
  4. 2024The Platonic Representation Hypothesiscited by 362
  5. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
  6. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 299
  7. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 246
  8. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  9. 2019Visualizing and Measuring the Geometry of BERTcited by 215
  10. 2020CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claimscited by 195
  11. 2023Getting aligned on representational alignmentcited by 142
  12. 2022Dense Text Retrieval based on Pretrained Language Models: A Surveycited by 136
  13. 2019Ripser: efficient computation of Vietoris-Rips persistence barcodescited by 115
  14. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 98
  15. 2025Gemini Embedding: Generalizable Embeddings from Geminicited by 93
  16. 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 82
  17. 2021Scikit-dimension: a Python package for intrinsic dimension estimationcited by 68
  18. 2021Representation Topology Divergence: A Method for Comparing Neural Network Representationscited by 66
  19. 2024Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Modelscited by 60
  20. 2024Emergence of a High-Dimensional Abstraction Phase in Language Transformerscited by 56
  21. 2021IsoScore: Measuring the Uniformity of Embedding Space Utilizationcited by 52
  22. 2020The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizescited by 48
  23. 2024Improving Retrieval for RAG based Question Answering Models on Financial Documentscited by 48
  24. 2019Evaluating Word Embedding Models: Methods and Experimental Resultscited by 37
  25. 2023The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Modelscited by 37
  26. 2025Shared Global and Local Geometry of Language Model Embeddingscited by 34
  27. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32
  28. 2024Text Clustering with Large Language Model Embeddingscited by 27
  29. 2020TREC-COVID: Constructing a Pandemic Information Retrieval Test Collectioncited by 23
  30. 2025The Intrinsic Dimension of Prompts in Internal Representations of Large Language Modelscited by 17
  31. 2023Metric Space Magnitude for Evaluating the Diversity of Latent Representationscited by 16
  32. 2024Objective drives the consistency of representational similarity across datasetscited by 16
  33. 2024Geometric Signatures of Compositionality Across a Language Model's Lifetimecited by 14
  34. 2024Beyond Benchmarks: Evaluating Embedding Model Similarity for Retrieval Augmented Generation Systemscited by 12
  35. 2024Persistent Topological Features in Large Language Modelscited by 11
  36. 2024When is an Embedding Model More Promising than Another?cited by 7
  37. 2025Measuring Intrinsic Dimension of Token Embeddingscited by 5
  38. 2024The Shape of Word Embeddings: Quantifying Non-Isometry With Topological Data Analysiscited by 4

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embeddingcited by 0
  2. 2026Isotropy Cliffs: The Geometric Signature of Decision-Making in Large Language Modelscited by 0
  3. 2026Topological Attribution Distance (TAD): Revealing Segment-Level RAG Influence on LLM Output Geometry for Incident Log Analysiscited by 0
  4. 2026Repurposing Unified Topological Signatures for Graph Representation Learningcited by 0