# Topology of Word Embeddings: Singularities Reflect Polysemy

arXiv: https://arxiv.org/abs/2011.09413 · HTML: https://mlsanity.com/paper/2011.09413/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2020.

## Built on by (most cited first, 15 of 15)

- [Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Senses](https://mlsanity.com/paper/2104.14694/) (2021)
- [Acceptability Judgements via Examining the Topology of Attention Maps](https://mlsanity.com/paper/2205.09630/) (2022)
- [Token embeddings violate the manifold hypothesis](https://mlsanity.com/paper/2504.01002/) (2025)
- [Topological Singularity Detection at Multiple Scales](https://mlsanity.com/paper/2210.00069/) (2022)
- [The structure of the token space for large language models](https://mlsanity.com/paper/2410.08993/) (2024)
- [Dialogue Term Extraction using Transfer Learning and Topological Data Analysis](https://mlsanity.com/paper/2208.10448/) (2022)
- [Probing the topology of the space of tokens with structured prompts](https://mlsanity.com/paper/2503.15421/) (2025)
- [Local Topology Measures of Contextual Language Model Latent Spaces With Applications to Dialogue Term Extraction](https://mlsanity.com/paper/2408.03706/) (2024)
- [Topological Data Analysis Applications in Natural Language Processing: A Survey](https://mlsanity.com/paper/2411.10298/) (2024)
- [Exploring the Stratified Space Structure of an RL Game with the Volume Growth Transform](https://mlsanity.com/paper/2507.22010/) (2025)
- [Advancing mathematics research with generative AI](https://mlsanity.com/paper/2511.07420/) (2025)
- [LGDE: Local Graph-based Dictionary Expansion](https://mlsanity.com/paper/2405.07764/) (2024)
- [Topological quantification of ambiguity in semantic search](https://mlsanity.com/paper/2406.07990/) (2024)
- [The cell as a token: high-dimensional geometry in language models and cell embeddings](https://mlsanity.com/paper/2503.20278/) (2025)
- [Text Corpora as Concept Fields: Black-Box Hallucination and Novelty Measurement](https://mlsanity.com/paper/2605.05103/) (2026)

## Abstract (arXiv)

The manifold hypothesis suggests that word vectors live on a submanifold within their ambient vector space. We argue that we should, more accurately, expect them to live on a pinched manifold: a singular quotient of a manifold obtained by identifying some of its points. The identified, singular points correspond to polysemous words, i.e. words with multiple meanings. Our point of view suggests that monosemous and polysemous words can be distinguished based on the topology of their neighbourhoods. We present two kinds of empirical evidence to support this point of view: (1) We introduce a topological measure of polysemy based on persistent homology that correlates well with the actual number of meanings of a word. (2) We propose a simple, topologically motivated solution to the SemEval-2010 task on Word Sense Induction & Disambiguation that produces competitive results.