# From Word to Sense Embeddings: A Survey on Vector Representations of Meaning

arXiv: https://arxiv.org/abs/1805.04032 · HTML: https://mlsanity.com/paper/1805.04032/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2018.

## Built on by (most cited first, 30 of 49)

- [Counter-Interference Adapter for Multilingual Machine Translation](https://mlsanity.com/paper/2104.08154/) (2021)
- [Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Senses](https://mlsanity.com/paper/2104.14694/) (2021)
- [Hyperdimensional computing as a framework for systematic aggregation of image descriptors](https://mlsanity.com/paper/2101.07720/) (2021)
- [Using Sequences of Life-events to Predict Human Lives](https://mlsanity.com/paper/2306.03009/) (2023)
- [A comparative evaluation and analysis of three generations of Distributional Semantic Models](https://mlsanity.com/paper/2105.09825/) (2021)
- [Recent advances in text embedding: A Comprehensive Review of Top-Performing Methods on the MTEB Benchmark](https://mlsanity.com/paper/2406.01607/) (2024)
- [MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models](https://mlsanity.com/paper/2109.09237/) (2021)
- [A Survey on Aspect-Based Sentiment Classification](https://mlsanity.com/paper/2203.14266/) (2022)
- [LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond](https://mlsanity.com/paper/2105.12449/) (2021)
- [How to keep text private? A systematic review of deep learning methods for privacy-preserving natural language processing](https://mlsanity.com/paper/2205.10095/) (2022)
- [Deep Learning for Hate Speech Detection: A Comparative Study](https://mlsanity.com/paper/2202.09517/) (2022)
- [Specialized Document Embeddings for Aspect-based Similarity of Research Papers](https://mlsanity.com/paper/2203.14541/) (2022)
- [Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs](https://mlsanity.com/paper/2305.12818/) (2023)
- [Combining Discrete Choice Models and Neural Networks through Embeddings: Formulation, Interpretability and Performance](https://mlsanity.com/paper/2109.12042/) (2021)
- [Can Word Sense Distribution Detect Semantic Changes of Words?](https://mlsanity.com/paper/2310.10400/) (2023)
- [pix2rule: End-to-end Neuro-symbolic Rule Learning](https://mlsanity.com/paper/2106.07487/) (2021)
- [Large Scale Substitution-based Word Sense Induction](https://mlsanity.com/paper/2110.07681/) (2021)
- [A Bias-Variance-Covariance Decomposition of Kernel Scores for Generative Models](https://mlsanity.com/paper/2310.05833/) (2023)
- [Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy](https://mlsanity.com/paper/2110.02204/) (2021)
- [PWESuite: Phonetic Word Embeddings and Tasks They Facilitate](https://mlsanity.com/paper/2304.02541/) (2023)
- [Quantum Mathematics in Artificial Intelligence](https://mlsanity.com/paper/2101.04255/) (2021)
- [PolyLM: Learning about Polysemy through Language Modeling](https://mlsanity.com/paper/2101.10448/) (2021)
- [A Survey of Text Representation Methods and Their Genealogy](https://mlsanity.com/paper/2211.14591/) (2022)
- [Survey in Characterizing Semantic Change](https://mlsanity.com/paper/2402.19088/) (2024)
- [Knowledge graph enhanced retrieval-augmented generation for failure mode and effects analysis](https://mlsanity.com/paper/2406.18114/) (2024)
- [Deep Learning, Natural Language Processing, and Explainable Artificial Intelligence in the Biomedical Domain](https://mlsanity.com/paper/2202.12678/) (2022)
- [A Survey on Automated Sarcasm Detection on Twitter](https://mlsanity.com/paper/2202.02516/) (2022)
- [Deep representation learning: Fundamentals, Perspectives, Applications, and Open Challenges](https://mlsanity.com/paper/2211.14732/) (2022)
- [Constructing and Expanding Low-Resource and Underrepresented Parallel Datasets for Indonesian Local Languages](https://mlsanity.com/paper/2404.01009/) (2024)
- [Synthetic4Health: Generating Annotated Synthetic Clinical Letters](https://mlsanity.com/paper/2409.09501/) (2024)

## Abstract (arXiv)

Over the past years, distributed semantic representations have proved to be effective and flexible keepers of prior knowledge to be integrated into downstream applications. This survey focuses on the representation of meaning. We start from the theoretical background behind word vector space models and highlight one of their major limitations: the meaning conflation deficiency, which arises from representing a word with all its possible meanings as a single vector. Then, we explain how this deficiency can be addressed through a transition from the word level to the more fine-grained level of word senses (in its broader acceptation) as a method for modelling unambiguous lexical meaning. We present a comprehensive overview of the wide range of techniques in the two main branches of sense representation, i.e., unsupervised and knowledge-based. Finally, this survey covers the main evaluation procedures and applications for this type of representation, and provides an analysis of four of its important aspects: interpretability, sense granularity, adaptability to different domains and compositionality.