# LearningWord Embeddings for Low-resource Languages by PU Learning

arXiv: https://arxiv.org/abs/1805.03366 · HTML: https://mlsanity.com/paper/1805.03366/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2018.

## Built on by (most cited first, 2 of 2)

- [PAUSE: Positive and Annealed Unlabeled Sentence Embedding](https://mlsanity.com/paper/2109.03155/) (2021)
- [Correspondence Analysis and PMI-Based Word Embeddings: A Comparative Study](https://mlsanity.com/paper/2405.20895/) (2024)

## Abstract (arXiv)

Word embedding is a key component in many downstream applications in processing natural languages. Existing approaches often assume the existence of a large collection of text for learning effective word embedding. However, such a corpus may not be available for some low-resource languages. In this paper, we study how to effectively learn a word embedding model on a corpus with only a few million tokens. In such a situation, the co-occurrence matrix is sparse as the co-occurrences of many word pairs are unobserved. In contrast to existing approaches often only sample a few unobserved word pairs as negative samples, we argue that the zero entries in the co-occurrence matrix also provide valuable information. We then design a Positive-Unlabeled Learning (PU-Learning) approach to factorize the co-occurrence matrix and validate the proposed approaches in four different languages.