mlsanityData to 14 Oct 2026

How Far Do Simple Transformations Translate Across Text Embedding Models?

arXiv 2608.05980 ↗First posted 6 Aug 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We investigate whether simple transformations can translate representations across heterogeneous text embedding models. Understanding how independently trained models organize semantic information is an enabler for AI-to-AI latent communication without decoding into human-readable text. Focusing on lightweight translators such as linear mappings, we test the literature hypothesis of latent universality in a realistic text setting beyond simplified benchmarks. Across nine embedding models differing in architecture, pooling strategy, and training objective, we evaluate compatibility using CKA, downstream transfer, fidelity, and retrieval. Simple translators recover meaningful shared structure and support transfer for some compatible pairs, but fail sharply for others. Compatibility depends jointly on architecture, training objective, pooling, and data distribution. Overall, the results show that heterogeneous embedding spaces are not universally related by simple mappings as often suggested in some literature.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 25 references that are in the map. First posted on arXiv 6 Aug 2026.

Builds on

23 of 25 references.

References · 23

  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  2. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  3. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  4. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  5. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  6. 2017Word Translation Without Parallel Datacited by 487
  7. 2024The Platonic Representation Hypothesiscited by 382
  8. 2014Understanding image representations by measuring their equivariance and equivalencecited by 243
  9. 2021Revisiting Model Stitching to Compare Neural Representationscited by 209
  10. 2016What makes ImageNet good for transfer learning?cited by 192
  11. 2022Relative representations enable zero-shot latent space communicationcited by 191
  12. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 125
11 more
  1. 2017Offline bilingual word vectors, orthogonal transformations and the inverted softmaxcited by 96
  2. 2025A Survey on Latent Reasoningcited by 81
  3. 2021Similarity and Matching of Neural Network Representationscited by 78
  4. 2025Latent Collaboration in Multi-Agent Systemscited by 73
  5. 2019Analogies Explained: Towards Understanding Word Embeddingscited by 64
  6. 2023Latent Space Translation via Semantic Alignmentcited by 52
  7. 2025Enabling Agents to Communicate Entirely in Latent Spacecited by 34
  8. 2023From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communicationcited by 21
  9. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 21
  10. 2024Latent Space Translation via Inverse Relative Projectioncited by 10
  11. 2026Ventriloquist LLMs: Linear Alignment of Late-Stage Representationscited by 4

Built on by

Not cited yet.