# Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?

arXiv: https://arxiv.org/abs/1909.01638 · HTML: https://mlsanity.com/paper/1909.01638/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2019.

## Built on by (most cited first, 30 of 31)

- [On Cross-Lingual Retrieval with Multilingual Text Encoders](https://mlsanity.com/paper/2112.11031/) (2021)
- [Improving Word Translation via Two-Stage Contrastive Learning](https://mlsanity.com/paper/2203.08307/) (2022)
- [Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval](https://mlsanity.com/paper/2101.08370/) (2021)
- [Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining](https://mlsanity.com/paper/2105.10419/) (2021)
- [Towards a Broad Coverage Named Entity Resource: A Data-Efficient Approach for Many Diverse Languages](https://mlsanity.com/paper/2201.12219/) (2022)
- [Por Qu'e N~ao Utiliser Alla Spraak? Mixed Training with Gradient Optimization in Few-Shot Cross-Lingual Transfer](https://mlsanity.com/paper/2204.13869/) (2022)
- [Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders](https://mlsanity.com/paper/2205.00267/) (2022)
- [IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces](https://mlsanity.com/paper/2210.05098/) (2022)
- [Improving Bilingual Lexicon Induction with Cross-Encoder Reranking](https://mlsanity.com/paper/2210.16953/) (2022)
- [Neural Machine Translation for the Indigenous Languages of the Americas: An Introduction](https://mlsanity.com/paper/2306.06804/) (2023)
- [Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport](https://mlsanity.com/paper/2210.14378/) (2022)
- [Quantized Wasserstein Procrustes Alignment of Word Embedding Spaces](https://mlsanity.com/paper/2212.02468/) (2022)
- [Cross-lingual hate speech detection based on multilingual domain-specific word embeddings](https://mlsanity.com/paper/2104.14728/) (2021)
- [Self-Augmented In-Context Learning for Unsupervised Word Translation](https://mlsanity.com/paper/2402.10024/) (2024)
- [Systematic Investigation of Strategies Tailored for Low-Resource Settings for Low-Resource Dependency Parsing](https://mlsanity.com/paper/2201.11374/) (2022)
- [When your Cousin has the Right Connections: Unsupervised Bilingual Lexicon Induction for Related Data-Imbalanced Languages](https://mlsanity.com/paper/2305.14012/) (2023)
- [Shared Path: Unraveling Memorization in Multilingual LLMs through Language Similarities](https://mlsanity.com/paper/2505.15722/) (2025)
- [Word Embedding Transformation for Robust Unsupervised Bilingual Lexicon Induction](https://mlsanity.com/paper/2105.12297/) (2021)
- [An Analysis of Euclidean vs. Graph-Based Framing for Bilingual Lexicon Induction from Word Embedding Spaces](https://mlsanity.com/paper/2109.12640/) (2021)
- [On Bilingual Lexicon Induction with Large Language Models](https://mlsanity.com/paper/2310.13995/) (2023)
- [ProMap: Effective Bilingual Lexicon Induction via Language Model Prompting](https://mlsanity.com/paper/2310.18778/) (2023)
- [Isomorphic Cross-lingual Embeddings for Low-Resource Languages](https://mlsanity.com/paper/2203.14632/) (2022)
- [Don't Forget Cheap Training Signals Before Building Unsupervised Bilingual Word Embeddings](https://mlsanity.com/paper/2205.15713/) (2022)
- [Robust Unsupervised Cross-Lingual Word Embedding using Domain Flow Interpolation](https://mlsanity.com/paper/2210.03319/) (2022)
- [Domain Mismatch Doesn't Always Prevent Cross-Lingual Transfer Learning](https://mlsanity.com/paper/2211.16671/) (2022)
- [Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph](https://mlsanity.com/paper/2109.04400/) (2021)
- [Cross-language Information Retrieval](https://mlsanity.com/paper/2111.05988/) (2021)
- [Massively Multilingual Lexical Specialization of Multilingual Transformers](https://mlsanity.com/paper/2208.01018/) (2022)
- [Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages](https://mlsanity.com/paper/2408.02290/) (2024)
- [A survey of neural-network-based methods utilising comparable data for finding translation equivalents](https://mlsanity.com/paper/2410.15144/) (2024)

## Abstract (arXiv)

Recent efforts in cross-lingual word embedding (CLWE) learning have predominantly focused on fully unsupervised approaches that project monolingual embeddings into a shared cross-lingual space without any cross-lingual signal. The lack of any supervision makes such approaches conceptually attractive. Yet, their only core difference from (weakly) supervised projection-based CLWE methods is in the way they obtain a seed dictionary used to initialize an iterative self-learning procedure. The fully unsupervised methods have arguably become more robust, and their primary use case is CLWE induction for pairs of resource-poor and distant languages. In this paper, we question the ability of even the most robust unsupervised CLWE approaches to induce meaningful CLWEs in these more challenging settings. A series of bilingual lexicon induction (BLI) experiments with 15 diverse languages (210 language pairs) show that fully unsupervised CLWE methods still fail for a large number of language pairs (e.g., they yield zero BLI performance for 87/210 pairs). Even when they succeed, they never surpass the performance of weakly supervised methods (seeded with 500-1,000 translation pairs) using the same self-learning procedure in any BLI setup, and the gaps are often substantial. These findings call for revisiting the main motivations behind fully unsupervised CLWE methods.