# Do Multi-Sense Embeddings Improve Natural Language Understanding?

arXiv: https://arxiv.org/abs/1506.01070 · HTML: https://mlsanity.com/paper/1506.01070/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2015.

## Built on by (most cited first, 9 of 9)

- [LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond](https://mlsanity.com/paper/2105.12449/) (2021)
- [Specialized Document Embeddings for Aspect-based Similarity of Research Papers](https://mlsanity.com/paper/2203.14541/) (2022)
- [Multi-sense embeddings through a word sense disambiguation process](https://mlsanity.com/paper/2101.08700/) (2021)
- [PolyLM: Learning about Polysemy through Language Modeling](https://mlsanity.com/paper/2101.10448/) (2021)
- [Learning to Make Compiler Optimizations More Effective](https://mlsanity.com/paper/2102.13514/) (2021)
- [A Survey On Neural Word Embeddings](https://mlsanity.com/paper/2110.01804/) (2021)
- [EDS-MEMBED: Multi-sense embeddings based on enhanced distributional semantic structures via a graph walk over word senses](https://mlsanity.com/paper/2103.00232/) (2021)
- [Word Sense Induction with Knowledge Distillation from BERT](https://mlsanity.com/paper/2304.10642/) (2023)
- [Semantic Cells: Evolutional Process to Acquire Sense Diversity of Items](https://mlsanity.com/paper/2404.14749/) (2024)

## Abstract (arXiv)

Learning a distinct representation for each sense of an ambiguous word could lead to more powerful and fine-grained models of vector-space representations. Yet while `multi-sense' methods have been proposed and tested on artificial word-similarity tasks, we don't know if they improve real natural language understanding tasks. In this paper we introduce a multi-sense embedding model based on Chinese Restaurant Processes that achieves state of the art performance on matching human word similarity judgments, and propose a pipelined architecture for incorporating multi-sense embeddings into language understanding. We then test the performance of our model on part-of-speech tagging, named entity recognition, sentiment analysis, semantic relation identification and semantic relatedness, controlling for embedding dimensionality. We find that multi-sense embeddings do improve performance on some tasks (part-of-speech tagging, semantic relation identification, semantic relatedness) but not on others (named entity recognition, various forms of sentiment analysis). We discuss how these differences may be caused by the different role of word sense information in each of the tasks. The results highlight the importance of testing embedding models in real applications.