# Unsupervised Cross-lingual Transfer of Word Embedding Spaces

arXiv: https://arxiv.org/abs/1809.03633 · HTML: https://mlsanity.com/paper/1809.03633/

## Key facts

- Cited by 16 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 10 Sep 2018.

## Built on by (most cited first, 16 of 16)

- [Consistency Regularization for Cross-Lingual Fine-Tuning](https://mlsanity.com/paper/2106.08226/) (2021)
- [Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study](https://mlsanity.com/paper/2106.03958/) (2021)
- [Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings](https://mlsanity.com/paper/2110.02887/) (2021)
- [Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport](https://mlsanity.com/paper/2210.14378/) (2022)
- [MEG: Medical Knowledge-Augmented Large Language Models for Question Answering](https://mlsanity.com/paper/2411.03883/) (2024)
- [Bilingual alignment transfers to multilingual alignment for unsupervised parallel text mining](https://mlsanity.com/paper/2104.07642/) (2021)
- [Morphological Processing of Low-Resource Languages: Where We Are and What's Next](https://mlsanity.com/paper/2203.08909/) (2022)
- [CCT-Code: Cross-Consistency Training for Multilingual Clone Detection and Code Search](https://mlsanity.com/paper/2305.11626/) (2023)
- [Cross-lingual Transferring of Pre-trained Contextualized Language Models](https://mlsanity.com/paper/2107.12627/) (2021)
- [Unsupervised Translation of Emergent Communication](https://mlsanity.com/paper/2502.07552/) (2025)
- [Word Embedding Transformation for Robust Unsupervised Bilingual Lexicon Induction](https://mlsanity.com/paper/2105.12297/) (2021)
- [Learning a Reversible Embedding Mapping using Bi-Directional Manifold Alignment](https://mlsanity.com/paper/2107.00124/) (2021)
- [Words as Bridges: Exploring Computational Support for Cross-Disciplinary Translation Work](https://mlsanity.com/paper/2503.18471/) (2025)
- [Sentiment Analysis Using Aligned Word Embeddings for Uralic Languages](https://mlsanity.com/paper/2305.15380/) (2023)
- [Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph](https://mlsanity.com/paper/2109.04400/) (2021)
- [Detecting Unseen Multiword Expressions in American Sign Language](https://mlsanity.com/paper/2310.00207/) (2023)

## Abstract (arXiv)

Cross-lingual transfer of word embeddings aims to establish the semantic mappings among words in different languages by learning the transformation functions over the corresponding word embedding spaces. Successfully solving this problem would benefit many downstream tasks such as to translate text classification models from resource-rich languages (e.g. English) to low-resource languages. Supervised methods for this problem rely on the availability of cross-lingual supervision, either using parallel corpora or bilingual lexicons as the labeled data for training, which may not be available for many low resource languages. This paper proposes an unsupervised learning approach that does not require any cross-lingual labeled data. Given two monolingual word embedding spaces for any language pair, our algorithm optimizes the transformation functions in both directions simultaneously based on distributional matching as well as minimizing the back-translation losses. We use a neural network implementation to calculate the Sinkhorn distance, a well-defined distributional similarity measure, and optimize our objective through back-propagation. Our evaluation on benchmark datasets for bilingual lexicon induction and cross-lingual word similarity prediction shows stronger or competitive performance of the proposed method compared to other state-of-the-art supervised and unsupervised baseline methods over many language pairs.