# Beyond Offline Mapping: Learning Cross Lingual Word Embeddings through Context Anchoring

arXiv: https://arxiv.org/abs/2012.15715 · HTML: https://mlsanity.com/paper/2012.15715/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2020.

## Built on by (most cited first, 8 of 8)

- [Subword Mapping and Anchoring across Languages](https://mlsanity.com/paper/2109.04556/) (2021)
- [IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces](https://mlsanity.com/paper/2210.05098/) (2022)
- [Learning a Reversible Embedding Mapping using Bi-Directional Manifold Alignment](https://mlsanity.com/paper/2107.00124/) (2021)
- [Isomorphic Cross-lingual Embeddings for Low-Resource Languages](https://mlsanity.com/paper/2203.14632/) (2022)
- [Robust Unsupervised Cross-Lingual Word Embedding using Domain Flow Interpolation](https://mlsanity.com/paper/2210.03319/) (2022)
- [Multilingual Word Embeddings for Low-Resource Languages using Anchors and a Chain of Related Languages](https://mlsanity.com/paper/2311.12489/) (2023)
- [A survey of neural-network-based methods utilising comparable data for finding translation equivalents](https://mlsanity.com/paper/2410.15144/) (2024)
- [Unsupervised Bilingual Lexicon Induction for Low Resource Languages](https://mlsanity.com/paper/2412.16894/) (2024)

## Abstract (arXiv)

Recent research on cross-lingual word embeddings has been dominated by unsupervised mapping approaches that align monolingual embeddings. Such methods critically rely on those embeddings having a similar structure, but it was recently shown that the separate training in different languages causes departures from this assumption. In this paper, we propose an alternative approach that does not have this limitation, while requiring a weak seed dictionary (e.g., a list of identical words) as the only form of supervision. Rather than aligning two fixed embedding spaces, our method works by fixing the target language embeddings, and learning a new set of embeddings for the source language that are aligned with them. To that end, we use an extension of skip-gram that leverages translated context words as anchor points, and incorporates self-learning and iterative restarts to reduce the dependency on the initial dictionary. Our approach outperforms conventional mapping methods on bilingual lexicon induction, and obtains competitive results in the downstream XNLI task.