# BilBOWA: Fast Bilingual Distributed Representations without Word Alignments

arXiv: https://arxiv.org/abs/1410.2455 · HTML: https://mlsanity.com/paper/1410.2455/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2014.

## Built on by (most cited first, 16 of 16)

- [Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2103.10531/) (2021)
- [Code-switched inspired losses for generic spoken dialog representations](https://mlsanity.com/paper/2108.12465/) (2021)
- [Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer?](https://mlsanity.com/paper/2212.10879/) (2022)
- [Learning to Speak from Text: Zero-Shot Multilingual Text-to-Speech with Unsupervised Text Pretraining](https://mlsanity.com/paper/2301.12596/) (2023)
- [IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces](https://mlsanity.com/paper/2210.05098/) (2022)
- [Cross-Lingual BERT Contextual Embedding Space Mapping with Isotropic and Isometric Conditions](https://mlsanity.com/paper/2107.09186/) (2021)
- [Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings](https://mlsanity.com/paper/2103.06459/) (2021)
- [Document Embedding for Scientific Articles: Efficacy of Word Embeddings vs TFIDF](https://mlsanity.com/paper/2107.05151/) (2021)
- [GRI: Graph-based Relative Isomorphism of Word Embedding Spaces](https://mlsanity.com/paper/2310.12360/) (2023)
- [Isomorphic Cross-lingual Embeddings for Low-Resource Languages](https://mlsanity.com/paper/2203.14632/) (2022)
- [The Effect of Alignment Objectives on Code-Switching Translation](https://mlsanity.com/paper/2309.05044/) (2023)
- [Learning Translations via Matrix Completion](https://mlsanity.com/paper/2406.13195/) (2024)
- [Bilingual Topic Models for Comparable Corpora](https://mlsanity.com/paper/2111.15278/) (2021)
- [Domain Curricula for Code-Switched MT at MixMT 2022](https://mlsanity.com/paper/2210.17463/) (2022)
- [PESTS: Persian_English Cross Lingual Corpus for Semantic Textual Similarity](https://mlsanity.com/paper/2305.07893/) (2023)
- [Corpus-Based Approaches to Igbo Diacritic Restoration](https://mlsanity.com/paper/2601.18380/) (2026)

## Abstract (arXiv)

We introduce BilBOWA (Bilingual Bag-of-Words without Alignments), a simple and computationally-efficient model for learning bilingual distributed representations of words which can scale to large monolingual datasets and does not require word-aligned parallel training data. Instead it trains directly on monolingual data and extracts a bilingual signal from a smaller set of raw-text sentence-aligned data. This is achieved using a novel sampled bag-of-words cross-lingual objective, which is used to regularize two noise-contrastive language models for efficient cross-lingual feature learning. We show that bilingual embeddings learned using the proposed model outperform state-of-the-art methods on a cross-lingual document classification task as well as a lexical translation task on WMT11 data.