# Massively Multilingual Word Embeddings

arXiv: https://arxiv.org/abs/1602.01925 · HTML: https://mlsanity.com/paper/1602.01925/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Feb 2016.

## Built on by (most cited first, 26 of 26)

- [Challenges and Strategies in Cross-Cultural NLP](https://mlsanity.com/paper/2203.10020/) (2022)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset](https://mlsanity.com/paper/2108.13897/) (2021)
- [BERT: A Review of Applications in Natural Language Processing and Understanding](https://mlsanity.com/paper/2103.11943/) (2021)
- [How to Adapt Your Pretrained Multilingual Model to 1600 Languages](https://mlsanity.com/paper/2106.02124/) (2021)
- [AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages](https://mlsanity.com/paper/2104.08726/) (2021)
- [Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation](https://mlsanity.com/paper/2203.09435/) (2022)
- [Match the Script, Adapt if Multilingual: Analyzing the Effect of Multilingual Pretraining on Cross-lingual Transferability](https://mlsanity.com/paper/2203.10753/) (2022)
- [Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddings](https://mlsanity.com/paper/2311.18034/) (2023)
- [Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?](https://mlsanity.com/paper/2110.13658/) (2021)
- [Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs](https://mlsanity.com/paper/2305.12818/) (2023)
- [Debiasing Multilingual Word Embeddings: A Case Study of Three Indian Languages](https://mlsanity.com/paper/2107.10181/) (2021)
- [Majority Voting with Bidirectional Pre-translation For Bitext Retrieval](https://mlsanity.com/paper/2103.06369/) (2021)
- [Cross-lingual hate speech detection based on multilingual domain-specific word embeddings](https://mlsanity.com/paper/2104.14728/) (2021)
- [Translate, then Detect: Leveraging Machine Translation for Cross-Lingual Toxicity Classification](https://mlsanity.com/paper/2509.14493/) (2025)
- [Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings](https://mlsanity.com/paper/2103.06459/) (2021)
- [GRI: Graph-based Relative Isomorphism of Word Embedding Spaces](https://mlsanity.com/paper/2310.12360/) (2023)
- [Ultra-lightweight Neural Differential DSP Vocoder For High Quality Speech Synthesis](https://mlsanity.com/paper/2401.10460/) (2024)
- [Parallel-SFT: Improving Zero-Shot Cross-Programming-Language Transfer for Code RL](https://mlsanity.com/paper/2604.20835/) (2026)
- [Effective Transfer Learning for Low-Resource Natural Language Understanding](https://mlsanity.com/paper/2208.09180/) (2022)
- [Enhancing Multilingual Language Models for Code-Switched Input Data](https://mlsanity.com/paper/2503.07990/) (2025)
- [Cross-language Information Retrieval](https://mlsanity.com/paper/2111.05988/) (2021)
- [Third-Party Aligner for Neural Word Alignments](https://mlsanity.com/paper/2211.04198/) (2022)
- [PESTS: Persian_English Cross Lingual Corpus for Semantic Textual Similarity](https://mlsanity.com/paper/2305.07893/) (2023)
- [Effective vocabulary expansion of multilingual language models for extremely low-resource languages](https://mlsanity.com/paper/2602.09388/) (2026)
- [TypeProbe: Recovering Type Representations from Hidden States of Pre-trained Code Models](https://mlsanity.com/paper/2607.08339/) (2026)

## Abstract (arXiv)

We introduce new methods for estimating and evaluating embeddings of words in more than fifty languages in a single shared embedding space. Our estimation methods, multiCluster and multiCCA, use dictionaries and monolingual data; they do not require parallel data. Our new evaluation method, multiQVEC-CCA, is shown to correlate better than previous ones with two downstream tasks (text categorization and parsing). We also describe a web portal for evaluation that will facilitate further research in this area, along with open-source releases of all our methods.