# Bilingual Distributed Word Representations from Document-Aligned Comparable Data

arXiv: https://arxiv.org/abs/1509.07308 · HTML: https://mlsanity.com/paper/1509.07308/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2015.

## Built on by (most cited first, 6 of 6)

- [MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer](https://mlsanity.com/paper/2401.04821/) (2024)
- [Isomorphic Cross-lingual Embeddings for Low-Resource Languages](https://mlsanity.com/paper/2203.14632/) (2022)
- [Learning Translations via Matrix Completion](https://mlsanity.com/paper/2406.13195/) (2024)
- [Bilingual Language Modeling, A transfer learning technique for Roman Urdu](https://mlsanity.com/paper/2102.10958/) (2021)
- [Bilingual Topic Models for Comparable Corpora](https://mlsanity.com/paper/2111.15278/) (2021)
- [Decipherment-Aware Multilingual Learning in Jointly Trained Language Models](https://mlsanity.com/paper/2406.07231/) (2024)

## Abstract (arXiv)

We propose a new model for learning bilingual word representations from non-parallel document-aligned data. Following the recent advances in word representation learning, our model learns dense real-valued word vectors, that is, bilingual word embeddings (BWEs). Unlike prior work on inducing BWEs which heavily relied on parallel sentence-aligned corpora and/or readily available translation resources such as dictionaries, the article reveals that BWEs may be learned solely on the basis of document-aligned comparable data without any additional lexical resources nor syntactic information. We present a comparison of our approach with previous state-of-the-art models for learning bilingual word representations from comparable data that rely on the framework of multilingual probabilistic topic modeling (MuPTM), as well as with distributional local context-counting models. We demonstrate the utility of the induced BWEs in two semantic tasks: (1) bilingual lexicon extraction, (2) suggesting word translations in context for polysemous words. Our simple yet effective BWE-based models significantly outperform the MuPTM-based and context-counting representation models from comparable data as well as prior BWE-based models, and acquire the best reported results on both tasks for all three tested language pairs.