# Should All Cross-Lingual Embeddings Speak English?

arXiv: https://arxiv.org/abs/1911.03058 · HTML: https://mlsanity.com/paper/1911.03058/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2019.

## Built on by (most cited first, 14 of 14)

- [The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics](https://mlsanity.com/paper/2102.01672/) (2021)
- [XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation](https://mlsanity.com/paper/2104.07412/) (2021)
- [Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots](https://mlsanity.com/paper/2103.09593/) (2021)
- [USCORE: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translation](https://mlsanity.com/paper/2202.10062/) (2022)
- [Wine is Not v i n. -- On the Compatibility of Tokenizations Across Languages](https://mlsanity.com/paper/2109.05772/) (2021)
- [An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models](https://mlsanity.com/paper/2403.20088/) (2024)
- [Average Is Not Enough: Caveats of Multilingual Evaluation](https://mlsanity.com/paper/2301.01269/) (2023)
- [Investigating Post-pretraining Representation Alignment for Cross-Lingual Question Answering](https://mlsanity.com/paper/2109.12028/) (2021)
- [CLIRudit: Cross-Lingual Information Retrieval of Scientific Documents](https://mlsanity.com/paper/2504.16264/) (2025)
- [Multilingual Multimodality: A Taxonomical Survey of Datasets, Techniques, Challenges and Opportunities](https://mlsanity.com/paper/2210.16960/) (2022)
- [Multilingual Word Embeddings for Low-Resource Languages using Anchors and a Chain of Related Languages](https://mlsanity.com/paper/2311.12489/) (2023)
- [Backretrieval: An Image-Pivoted Evaluation Metric for Cross-Lingual Text Representations Without Parallel Corpora](https://mlsanity.com/paper/2105.04971/) (2021)
- [Decipherment-Aware Multilingual Learning in Jointly Trained Language Models](https://mlsanity.com/paper/2406.07231/) (2024)
- [A survey of neural-network-based methods utilising comparable data for finding translation equivalents](https://mlsanity.com/paper/2410.15144/) (2024)

## Abstract (arXiv)

Most of recent work in cross-lingual word embeddings is severely Anglocentric. The vast majority of lexicon induction evaluation dictionaries are between English and another language, and the English embedding space is selected by default as the hub when learning in a multilingual setting. With this work, however, we challenge these practices. First, we show that the choice of hub language can significantly impact downstream lexicon induction performance. Second, we both expand the current evaluation dictionary collection to include all language pairs using triangulation, and also create new dictionaries for under-represented languages. Evaluating established methods over all these language pairs sheds light into their suitability and presents new challenges for the field. Finally, in our analysis we identify general guidelines for strong cross-lingual embeddings baselines, based on more than just Anglocentric experiments.