# Distributed representation of multi-sense words: A loss-driven approach

arXiv: https://arxiv.org/abs/1904.06725 · HTML: https://mlsanity.com/paper/1904.06725/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2019.

## Built on by (most cited first, 1 of 1)

- [Measuring diachronic sense change: new models and Monte Carlo methods for Bayesian inference](https://mlsanity.com/paper/2105.00819/) (2021)

## Abstract (arXiv)

Word2Vec's Skip Gram model is the current state-of-the-art approach for estimating the distributed representation of words. However, it assumes a single vector per word, which is not well-suited for representing words that have multiple senses. This work presents LDMI, a new model for estimating distributional representations of words. LDMI relies on the idea that, if a word carries multiple senses, then having a different representation for each of its senses should lead to a lower loss associated with predicting its co-occurring words, as opposed to the case when a single vector representation is used for all the senses. After identifying the multi-sense words, LDMI clusters the occurrences of these words to assign a sense to each occurrence. Experiments on the contextual word similarity task show that LDMI leads to better performance than competing approaches.