mlsanityData to 14 Oct 2026

Harnessing the Universal Geometry of Embeddings

arXiv 2505.12540 ↗First posted 18 May 2025cs.LGcited by 55

Not on the map. Why?

Abstract (arXiv)

We introduce the first method for translating text embeddings from one vector space to another without any paired data, encoders, or predefined sets of matches. Our unsupervised approach translates any embedding to and from a universal latent representation (i.e., a universal semantic structure conjectured by the Platonic Representation Hypothesis). Our translations achieve high cosine similarity across model pairs with different architectures, parameter counts, and training datasets. The ability to translate unknown embeddings into a different space while preserving their geometry has serious implications for the security of vector databases. An adversary with access only to embedding vectors can extract sensitive information about the underlying documents, sufficient for classification and attribute inference.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Builds on 55 of its 71 references that are in the map. First posted on arXiv 18 May 2025.

Builds on

55 of 71 references.

References · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2022Generative Adversarial Networkscited by 5,627
  5. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  6. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  7. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  8. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  9. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  10. 2017Unsupervised Image-to-Image Translation Networkscited by 603
  11. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
  12. 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
38 more
  1. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  2. 2017Word Translation Without Parallel Datacited by 487
  3. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  4. 2024The Platonic Representation Hypothesiscited by 382
  5. 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
  6. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  7. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 273
  8. 2015Convergent Learning: Do different neural networks learn the same representations?cited by 220
  9. 2021Revisiting Model Stitching to Compare Neural Representationscited by 209
  10. 2020Information Leakage in Embedding Modelscited by 203
  11. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 192
  12. 2022Relative representations enable zero-shot latent space communicationcited by 191
  13. 2024Stealing Part of a Production Language Modelcited by 159
  14. 2017Unsupervised Neural Machine Translationcited by 149
  15. 2018A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddingscited by 146
  16. 2016Counter-fitting Word Vectors to Linguistic Constraintscited by 140
  17. 2023Grounding Language Models to Images for Multimodal Inputs and Outputscited by 131
  18. 2018Gromov-Wasserstein Alignment of Word Embedding Spacescited by 122
  19. 2024Jasper and Stella: distillation of SOTA embedding modelscited by 112
  20. 2020Graph Optimal Transport for Cross-Domain Alignmentcited by 103
  21. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 93
  22. 2023Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentencecited by 81
  23. 2018Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterioncited by 70
  24. 2023Language Model Inversioncited by 68
  25. 2020Multi-domain Clinical Natural Language Processing with MedCAT: the Medical Concept Annotation Toolkitcited by 66
  26. 2023Text-To-Concept (and Back) via Cross-Model Alignmentcited by 64
  27. 2018Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representationcited by 62
  28. 2023Latent Space Translation via Semantic Alignmentcited by 52
  29. 2022ASIF: Coupled Data Turns Unimodal Models to Multimodal Without Trainingcited by 47
  30. 2024Do Vision and Language Encoders Represent the World Similarly?cited by 44
  31. 2024Extracting Prompts by Inverting LLM Outputscited by 41
  32. 2023Do Vision and Language Models Share Concepts? A Vector Space Alignment Studycited by 27
  33. 2023Discovering Universal Geometry in Embeddings with ICAcited by 19
  34. 2018Unsupervised Multilingual Word Embeddingscited by 16
  35. 2018Unsupervised Neural Machine Translation with Weight Sharingcited by 15
  36. 2019embComp: Visual Interactive Comparison of Vector Embeddingscited by 15
  37. 2025It's a (Blind) Match! Towards Vision-Language Correspondence without Parallel Datacited by 14
  38. 2025Layers at Similar Depths Generate Similar Activations Across LLM Architecturescited by 12

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 25; 2026: 30.

55 papers.

Sort:

By area:

  1. 2025Correlated Errors in Large Language Modelscited by 45
  2. 2025Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-trainingcited by 23
  3. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 21
  4. 2025The Geometry of Reasoning: Flowing Logics in Representation Spacecited by 21
  5. 2026Latent Space Communication via K-V Cache Alignmentcited by 17
  6. 2026Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scalecited by 16
  7. 2025Universally Converging Representations of Matter Across Scientific Foundation Modelscited by 14
  8. 2025Jailbreak Transferability Emerges from Shared Representationscited by 6
  9. 2025DeRAG: Black-box Adversarial Attacks on Multiple Retrieval-Augmented Generation Applications via Prompt Injectioncited by 5
  10. 2025Dynamic Reflections: Probing Video Representations with Text Alignmentcited by 5
  11. 2026Multi-Way Representation Alignmentcited by 5
  12. 2025mini-vec2vec: Scaling Universal Geometry Alignment with Linear Transformationscited by 4
38 more
  1. 2025Learning Network Sheaves for AI-native Semantic Communicationcited by 4
  2. 2026The Capability Convergence Hypothesis: Capability from Access Structure, Not Scalecited by 4
  3. 2025Atlas-Alignment: Making Interpretability Transferable Across Language Modelscited by 3
  4. 2026SOTAlign: Semi-Supervised Alignment of Unimodal Vision and Language Models via Optimal Transportcited by 3
  5. 2026The Universal Normal Embeddingcited by 3
  6. 2025Shared representations in brains and models reveal a two-route cortical organization during scene perceptioncited by 2
  7. 2025ENIGMA: The Geometry of Reasoning and Alignment in Large-Language Modelscited by 2
  8. 2025Position: Require Frontier AI Labs To Release Small "Analog" Modelscited by 2
  9. 2025Unique Lives, Shared World: Learning from Single-Life Videoscited by 2
  10. 2026Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networkscited by 2
  11. 2026Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generationcited by 2
  12. 2026Language Models Represent and Transform Concepts with Shared Geometrycited by 2
  13. 2026The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decodingcited by 2
  14. 2025Latent Reasoning via Sentence Embedding Predictioncited by 1
  15. 2025MOD-X: A Modular Open Decentralized eXchange Framework proposal for Heterogeneous Interoperable Artificial Intelligence Agentscited by 1
  16. 2025Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeedcited by 1
  17. 2025Mapping Semantic & Syntactic Relationships with Geometric Rotationcited by 1
  18. 2025Exploring possible vector systems for faster training of neural networks with preconfigured latent spacescited by 1
  19. 2026Remapping and navigation of an embedding space via error minimization: a fundamental organizational principle of cognition in natural and artificial systemscited by 1
  20. 2026ATLAS: Constitution-Conditioned Latent Geometry and Redistribution Across Language Models and Neural Perturbation Datacited by 1
  21. 2026SEMASIA: A Large-Scale Dataset of Semantically Structured Latent Representationscited by 1
  22. 2026Platonic Representations in the Human Brain: Unsupervised Recovery of Universal Geometrycited by 1
  23. 2026Scene-Centric Unsupervised Video Panoptic Segmentationcited by 1
  24. 2026Hybrid privacy-aware semantic search: SVD-truncated document geometry and CKKS-encrypted query reranking under a restricted threat modelcited by 1
  25. 2025The Statistical Validation of Innovation Lenscited by 0
  26. 2025Graph Alignment via Dual-Pass Spectral Encoding and Latent Space Communicationcited by 0
  27. 2025Knowledge distillation through geometry-aware representational alignmentcited by 0
  28. 2025Methodological Framework for Quantifying Semantic Test Coverage in RAG Systemscited by 0
  29. 2025Representing Beauty: Towards a Participatory but Objective Latent Aestheticscited by 0
  30. 2026Universal Latent Homeomorphic Manifolds: A Framework for Cross-Domain Representation Unificationcited by 0
  31. 2026Federated learning for unpaired multimodal data through a homogeneous transformer modelcited by 0
  32. 2026On the Relationship Between Representation Geometry and Generalization in Deep Neural Networkscited by 0
  33. 2026A visual observation on the geometry of UMAP projections of the difference vectors of antonym and synonym word pair embeddingscited by 0
  34. 2026Multilingual Embedding Probes Fail to Generalize Across Learner Corporacited by 0
  35. 2026LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Seriescited by 0
  36. 2026Order Is Not Control: Driven-Dissipative Response Laws Across Artificial and Biological Systemscited by 0
  37. 2026SHARD: cell-keyed residual splitting for alignment-resistant private dense retrievalcited by 0
  38. 2026DS@GT ARC at ImageCLEFmedical 2026: Architectural Diversity for Concept Detection and Foundation-Model Scaling for Caption Prediction in Medical Image Analysiscited by 0

50 most cited of 55 All 55 (JSON)