# Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation

arXiv: https://arxiv.org/abs/1906.10007 · HTML: https://mlsanity.com/paper/1906.10007/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 30 of 40)

- [minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models](https://mlsanity.com/paper/2203.13112/) (2022)
- [MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models](https://mlsanity.com/paper/2109.09237/) (2021)
- [Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddings](https://mlsanity.com/paper/2203.07523/) (2022)
- [LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond](https://mlsanity.com/paper/2105.12449/) (2021)
- [RT-LM: Uncertainty-Aware Resource Management for Real-Time Inference of Language Models](https://mlsanity.com/paper/2309.06619/) (2023)
- [Can Word Sense Distribution Detect Semantic Changes of Words?](https://mlsanity.com/paper/2310.10400/) (2023)
- [What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories](https://mlsanity.com/paper/2302.03353/) (2023)
- [Incorporating Word Sense Disambiguation in Neural Language Models](https://mlsanity.com/paper/2106.07967/) (2021)
- [LexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution](https://mlsanity.com/paper/2107.05132/) (2021)
- [Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy](https://mlsanity.com/paper/2110.02204/) (2021)
- [Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories](https://mlsanity.com/paper/2110.14091/) (2021)
- [Multilingual Word Sense Disambiguation with Unified Sense Representation](https://mlsanity.com/paper/2210.07447/) (2022)
- [Ambiguity Meets Uncertainty: Investigating Uncertainty Estimation for Word Sense Disambiguation](https://mlsanity.com/paper/2305.13119/) (2023)
- [Resolving Regular Polysemy in Named Entities](https://mlsanity.com/paper/2401.09758/) (2024)
- [Non-Parametric Few-Shot Learning for Word Sense Disambiguation](https://mlsanity.com/paper/2104.12677/) (2021)
- [Patterns of Lexical Ambiguity in Contextualised Language Models](https://mlsanity.com/paper/2109.13032/) (2021)
- [Multi-Relational Hyperbolic Word Embeddings from Natural Language Definitions](https://mlsanity.com/paper/2305.07303/) (2023)
- [Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representations](https://mlsanity.com/paper/2307.13776/) (2023)
- [Word sense extension](https://mlsanity.com/paper/2306.05609/) (2023)
- [Word Sense Disambiguation as a Game of Neurosymbolic Darts](https://mlsanity.com/paper/2307.16663/) (2023)
- [From Unstructured Text to Causal Knowledge Graphs: A Transformer-Based Approach](https://mlsanity.com/paper/2202.11768/) (2022)
- [Semantic properties of English nominal pluralization: Insights from word embeddings](https://mlsanity.com/paper/2203.15424/) (2022)
- [Learning Dynamic Contextualised Word Embeddings via Template-based Temporal Adaptation](https://mlsanity.com/paper/2208.10734/) (2022)
- [Semantic Specialization for Knowledge-based Word Sense Disambiguation](https://mlsanity.com/paper/2304.11340/) (2023)
- [Construction Grammar and Language Models](https://mlsanity.com/paper/2308.13315/) (2023)
- [SANDWiCH: Semantical Analysis of Neighbours for Disambiguating Words in Context ad Hoc](https://mlsanity.com/paper/2503.05958/) (2025)
- [EDS-MEMBED: Multi-sense embeddings based on enhanced distributional semantic structures via a graph walk over word senses](https://mlsanity.com/paper/2103.00232/) (2021)
- [MCL@IITK at SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation using Augmented Data, Signals, and Transformers](https://mlsanity.com/paper/2104.01567/) (2021)
- [Uppsala NLP at SemEval-2021 Task 2: Multilingual Language Models for Fine-tuning and Feature Extraction in Word-in-Context Disambiguation](https://mlsanity.com/paper/2104.03767/) (2021)
- [Interval Probabilistic Fuzzy WordNet](https://mlsanity.com/paper/2104.10660/) (2021)

## Abstract (arXiv)

Contextual embeddings represent a new generation of semantic representations learned from Neural Language Modelling (NLM) that addresses the issue of meaning conflation hampering traditional word embeddings. In this work, we show that contextual embeddings can be used to achieve unprecedented gains in Word Sense Disambiguation (WSD) tasks. Our approach focuses on creating sense-level embeddings with full-coverage of WordNet, and without recourse to explicit knowledge of sense distributions or task-specific modelling. As a result, a simple Nearest Neighbors (k-NN) method using our representations is able to consistently surpass the performance of previous systems using powerful neural sequencing models. We also analyse the robustness of our approach when ignoring part-of-speech and lemma features, requiring disambiguation against the full sense inventory, and revealing shortcomings to be improved. Finally, we explore applications of our sense embeddings for concept-level analyses of contextual embeddings and their respective NLMs.