# Geometry of Polysemy

arXiv: https://arxiv.org/abs/1610.07569 · HTML: https://mlsanity.com/paper/1610.07569/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2016.

## Built on by (most cited first, 6 of 6)

- [Tsetlin Machine Embedding: Representing Words Using Logical Expressions](https://mlsanity.com/paper/2301.00709/) (2023)
- [Improving Entity Linking through Semantic Reinforced Entity Embeddings](https://mlsanity.com/paper/2106.08495/) (2021)
- [Low-Rank Subspaces for Unsupervised Entity Linking](https://mlsanity.com/paper/2104.08737/) (2021)
- [Sinhala Sentence Embedding: A Two-Tiered Structure for Low-Resource Languages](https://mlsanity.com/paper/2210.14472/) (2022)
- [Shades of meaning: Uncovering the geometry of ambiguous word representations through contextualised language models](https://mlsanity.com/paper/2304.13597/) (2023)
- [Socially Fair Center-based and Linear Subspace Clustering](https://mlsanity.com/paper/2208.10095/) (2022)

## Abstract (arXiv)

Vector representations of words have heralded a transformational approach to classical problems in NLP; the most popular example is word2vec. However, a single vector does not suffice to model the polysemous nature of many (frequent) words, i.e., words with multiple meanings. In this paper, we propose a three-fold approach for unsupervised polysemy modeling: (a) context representations, (b) sense induction and disambiguation and (c) lexeme (as a word and sense pair) representations. A key feature of our work is the finding that a sentence containing a target word is well represented by a low rank subspace, instead of a point in a vector space. We then show that the subspaces associated with a particular sense of the target word tend to intersect over a line (one-dimensional subspace), which we use to disambiguate senses using a clustering algorithm that harnesses the Grassmannian geometry of the representations. The disambiguation algorithm, which we call K-Grassmeans, leads to a procedure to label the different senses of the target word in the corpus -- yielding lexeme vector representations, all in an unsupervised manner starting from a large (Wikipedia) corpus in English. Apart from several prototypical target (word,sense) examples and a host of empirical studies to intuit and justify the various geometric representations, we validate our algorithms on standard sense induction and disambiguation datasets and present new state-of-the-art results.