# Are All Good Word Vector Spaces Isomorphic?

arXiv: https://arxiv.org/abs/2004.04070 · HTML: https://mlsanity.com/paper/2004.04070/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2020.

## Built on by (most cited first, 30 of 37)

- [Relative representations enable zero-shot latent space communication](https://mlsanity.com/paper/2209.15430/) (2022)
- [Latent Space Translation via Semantic Alignment](https://mlsanity.com/paper/2311.00664/) (2023)
- [Do Vision and Language Encoders Represent the World Similarly?](https://mlsanity.com/paper/2401.05224/) (2024)
- [Understanding Cross-Lingual Alignment -- A Survey](https://mlsanity.com/paper/2404.06228/) (2024)
- [An Isotropy Analysis in the Multilingual BERT Embedding Space](https://mlsanity.com/paper/2110.04504/) (2021)
- [Obtaining Better Static Word Embeddings Using Contextual Embedding Models](https://mlsanity.com/paper/2106.04302/) (2021)
- [Do Language Models Know the Way to Rome?](https://mlsanity.com/paper/2109.07971/) (2021)
- [Latent Space Translation via Inverse Relative Projection](https://mlsanity.com/paper/2406.15057/) (2024)
- [A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space](https://mlsanity.com/paper/2109.06324/) (2021)
- [Combining Static and Contextualised Multilingual Embeddings](https://mlsanity.com/paper/2203.09326/) (2022)
- [Bootstrapping Parallel Anchors for Relative Representations](https://mlsanity.com/paper/2303.00721/) (2023)
- [IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces](https://mlsanity.com/paper/2210.05098/) (2022)
- [Neural Machine Translation for the Indigenous Languages of the Americas: An Introduction](https://mlsanity.com/paper/2306.06804/) (2023)
- [GrEmLIn: A Repository of Green Baseline Embeddings for 87 Low-Resource Languages Injected with Multilingual Graph Knowledge](https://mlsanity.com/paper/2409.18193/) (2024)
- [Cross-Lingual BERT Contextual Embedding Space Mapping with Isotropic and Isometric Conditions](https://mlsanity.com/paper/2107.09186/) (2021)
- [Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport](https://mlsanity.com/paper/2210.14378/) (2022)
- [Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization](https://mlsanity.com/paper/2310.12794/) (2023)
- [Escaping Plato's Cave: Towards the Alignment of 3D and Text Latent Spaces](https://mlsanity.com/paper/2503.05283/) (2025)
- [Majority Voting with Bidirectional Pre-translation For Bitext Retrieval](https://mlsanity.com/paper/2103.06369/) (2021)
- [Survey in Characterizing Semantic Change](https://mlsanity.com/paper/2402.19088/) (2024)
- [The Shape of Word Embeddings: Quantifying Non-Isometry With Topological Data Analysis](https://mlsanity.com/paper/2404.00500/) (2024)
- [Relative Representations: Topological and Geometric Perspectives](https://mlsanity.com/paper/2409.10967/) (2024)
- [MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer](https://mlsanity.com/paper/2401.04821/) (2024)
- [FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss for Unlabeled Clients](https://mlsanity.com/paper/2402.10191/) (2024)
- [A Comprehensive Analysis of Static Word Embeddings for Turkish](https://mlsanity.com/paper/2405.07778/) (2024)
- [Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment](https://mlsanity.com/paper/2407.14878/) (2024)
- [Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probes](https://mlsanity.com/paper/2109.04921/) (2021)
- [Isomorphic Cross-lingual Embeddings for Low-Resource Languages](https://mlsanity.com/paper/2203.14632/) (2022)
- [Linguistically-Informed Neural Architectures for Lexical, Syntactic and Semantic Tasks in Sanskrit](https://mlsanity.com/paper/2308.08807/) (2023)
- [Large language models converge toward human-like concept organization](https://mlsanity.com/paper/2308.15047/) (2023)

## Abstract (arXiv)

Existing algorithms for aligning cross-lingual word vector spaces assume that vector spaces are approximately isomorphic. As a result, they perform poorly or fail completely on non-isomorphic spaces. Such non-isomorphism has been hypothesised to result from typological differences between languages. In this work, we ask whether non-isomorphism is also crucially a sign of degenerate word vector spaces. We present a series of experiments across diverse languages which show that variance in performance across language pairs is not only due to typological differences, but can mostly be attributed to the size of the monolingual resources available, and to the properties and duration of monolingual training (e.g. "under-training").