# Semantic Drift in Multilingual Representations

arXiv: https://arxiv.org/abs/1904.10820 · HTML: https://mlsanity.com/paper/1904.10820/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2019.

## Built on by (most cited first, 13 of 13)

- [Relative Importance in Sentence Processing](https://mlsanity.com/paper/2106.03471/) (2021)
- [Language Embeddings Sometimes Contain Typological Generalizations](https://mlsanity.com/paper/2301.08115/) (2023)
- [Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders](https://mlsanity.com/paper/2205.00267/) (2022)
- [Cross-Lingual Transfer of Cognitive Processing Complexity](https://mlsanity.com/paper/2302.12695/) (2023)
- [NeoBabel: A Multilingual Open Tower for Visual Generation](https://mlsanity.com/paper/2507.06137/) (2025)
- [Locally Measuring Cross-lingual Lexical Alignment: A Domain and Word Level Perspective](https://mlsanity.com/paper/2410.07239/) (2024)
- [Exploiting Vulnerabilities in Speech Translation Systems through Targeted Adversarial Attacks](https://mlsanity.com/paper/2503.00957/) (2025)
- [Evaluating Language Translation Models by Playing Telephone](https://mlsanity.com/paper/2509.19611/) (2025)
- [Toward Robust Multilingual Adaptation of LLMs for Low-Resource Languages](https://mlsanity.com/paper/2510.14466/) (2025)
- [Is Textual Similarity Invariant under Machine Translation? Evidence Based on the Political Manifesto Corpus](https://mlsanity.com/paper/2605.00618/) (2026)
- [Examining the Limits of Word2Vec with Toki Pona](https://mlsanity.com/paper/2606.17299/) (2026)
- [Multilingual Sentence Embeddings for Linguistic-Integrated Reliability Audit](https://mlsanity.com/paper/2607.17466/) (2026)
- [Dynamics of Meaning: Towards the Evaluation of Diachronic Semantic Change in Sinhala](https://mlsanity.com/paper/2609.08609/) (2026)

## Abstract (arXiv)

Multilingual representations have mostly been evaluated based on their performance on specific tasks. In this article, we look beyond engineering goals and analyze the relations between languages in computational representations. We introduce a methodology for comparing languages based on their organization of semantic concepts. We propose to conduct an adapted version of representational similarity analysis of a selected set of concepts in computational multilingual representations. Using this analysis method, we can reconstruct a phylogenetic tree that closely resembles those assumed by linguistic experts. These results indicate that multilingual distributional representations which are only trained on monolingual text and bilingual dictionaries preserve relations between languages without the need for any etymological information. In addition, we propose a measure to identify semantic drift between language families. We perform experiments on word-based and sentence-based multilingual models and provide both quantitative results and qualitative examples. Analyses of semantic drift in multilingual representations can serve two purposes: they can indicate unwanted characteristics of the computational models and they provide a quantitative means to study linguistic phenomena across languages. The code is available at https://github.com/beinborn/SemanticDrift.