# How Far Do Simple Transformations Translate Across Text Embedding Models?

arXiv: https://arxiv.org/abs/2608.05980 · HTML: https://mlsanity.com/paper/2608.05980/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 25 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Builds on (most cited first, 23 of 23)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Understanding image representations by measuring their equivariance and equivalence](https://mlsanity.com/paper/1411.5908/) (2014)
- [Revisiting Model Stitching to Compare Neural Representations](https://mlsanity.com/paper/2106.07682/) (2021)
- [What makes ImageNet good for transfer learning?](https://mlsanity.com/paper/1608.08614/) (2016)
- [Relative representations enable zero-shot latent space communication](https://mlsanity.com/paper/2209.15430/) (2022)
- [MMTEB: Massive Multilingual Text Embedding Benchmark](https://mlsanity.com/paper/2502.13595/) (2025)
- [Offline bilingual word vectors, orthogonal transformations and the inverted softmax](https://mlsanity.com/paper/1702.03859/) (2017)
- [A Survey on Latent Reasoning](https://mlsanity.com/paper/2507.06203/) (2025)
- [Similarity and Matching of Neural Network Representations](https://mlsanity.com/paper/2110.14633/) (2021)
- [Latent Collaboration in Multi-Agent Systems](https://mlsanity.com/paper/2511.20639/) (2025)
- [Analogies Explained: Towards Understanding Word Embeddings](https://mlsanity.com/paper/1901.09813/) (2019)
- [Latent Space Translation via Semantic Alignment](https://mlsanity.com/paper/2311.00664/) (2023)
- [Enabling Agents to Communicate Entirely in Latent Space](https://mlsanity.com/paper/2511.09149/) (2025)
- [From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communication](https://mlsanity.com/paper/2310.01211/) (2023)
- [Transferring Linear Features Across Language Models With Model Stitching](https://mlsanity.com/paper/2506.06609/) (2025)
- [Latent Space Translation via Inverse Relative Projection](https://mlsanity.com/paper/2406.15057/) (2024)
- [Ventriloquist LLMs: Linear Alignment of Late-Stage Representations](https://mlsanity.com/paper/2603.18908/) (2026)

## Abstract (arXiv)

We investigate whether simple transformations can translate representations across heterogeneous text embedding models. Understanding how independently trained models organize semantic information is an enabler for AI-to-AI latent communication without decoding into human-readable text. Focusing on lightweight translators such as linear mappings, we test the literature hypothesis of latent universality in a realistic text setting beyond simplified benchmarks. Across nine embedding models differing in architecture, pooling strategy, and training objective, we evaluate compatibility using CKA, downstream transfer, fidelity, and retrieval. Simple translators recover meaningful shared structure and support transfer for some compatible pairs, but fail sharply for others. Compatibility depends jointly on architecture, training objective, pooling, and data distribution. Overall, the results show that heterogeneous embedding spaces are not universally related by simple mappings as often suggested in some literature.