# Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations

arXiv: https://arxiv.org/abs/1910.00194 · HTML: https://mlsanity.com/paper/1910.00194/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2019.

## Built on by (most cited first, 21 of 21)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Characterizing English Variation across Social Media Communities with BERT](https://mlsanity.com/paper/2102.06820/) (2021)
- [MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models](https://mlsanity.com/paper/2109.09237/) (2021)
- [What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories](https://mlsanity.com/paper/2302.03353/) (2023)
- [Incorporating Word Sense Disambiguation in Neural Language Models](https://mlsanity.com/paper/2106.07967/) (2021)
- [Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy](https://mlsanity.com/paper/2110.02204/) (2021)
- [Unsupervised Document Embedding via Contrastive Augmentation](https://mlsanity.com/paper/2103.14542/) (2021)
- [Non-Parametric Few-Shot Learning for Word Sense Disambiguation](https://mlsanity.com/paper/2104.12677/) (2021)
- [Ceasing hate withMoH: Hate Speech Detection in Hindi-English Code-Switched Language](https://mlsanity.com/paper/2110.09393/) (2021)
- [Word sense extension](https://mlsanity.com/paper/2306.05609/) (2023)
- [Word Sense Disambiguation as a Game of Neurosymbolic Darts](https://mlsanity.com/paper/2307.16663/) (2023)
- [Meta-Learning with Variational Semantic Memory for Word Sense Disambiguation](https://mlsanity.com/paper/2106.02960/) (2021)
- [SANDWiCH: Semantical Analysis of Neighbours for Disambiguating Words in Context ad Hoc](https://mlsanity.com/paper/2503.05958/) (2025)
- [EDS-MEMBED: Multi-sense embeddings based on enhanced distributional semantic structures via a graph walk over word senses](https://mlsanity.com/paper/2103.00232/) (2021)
- [Verb Sense Clustering using Contextualized Word Representations for Semantic Frame Induction](https://mlsanity.com/paper/2105.13465/) (2021)
- [Measuring Context-Word Biases in Lexical Semantic Datasets](https://mlsanity.com/paper/2112.06733/) (2021)
- [PolyBERT: Fine-Tuned Poly Encoder BERT-Based Model for Word Sense Disambiguation](https://mlsanity.com/paper/2506.00968/) (2025)
- [One Sense per Translation](https://mlsanity.com/paper/2106.06082/) (2021)
- [A Quadratic 0-1 Programming Approach for Word Sense Disambiguation](https://mlsanity.com/paper/2201.04877/) (2022)
- [HKUST at SemEval-2023 Task 1: Visual Word Sense Disambiguation with Context Augmentation and Visual Assistance](https://mlsanity.com/paper/2311.18273/) (2023)
- [COGNAC at SemEval-2026 Task 5: LLM Ensembles for Human-Level Word Sense Plausibility Rating in Challenging Narratives](https://mlsanity.com/paper/2603.15897/) (2026)

## Abstract (arXiv)

Contextualized word representations are able to give different representations for the same word in different contexts, and they have been shown to be effective in downstream natural language processing tasks, such as question answering, named entity recognition, and sentiment analysis. However, evaluation on word sense disambiguation (WSD) in prior work shows that using contextualized word representations does not outperform the state-of-the-art approach that makes use of non-contextualized word embeddings. In this paper, we explore different strategies of integrating pre-trained contextualized word representations and our best strategy achieves accuracies exceeding the best prior published accuracies by significant margins on multiple benchmark WSD datasets. We make the source code available at https://github.com/nusnlp/contextemb-wsd.