# Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences

arXiv: https://arxiv.org/abs/2009.11795 · HTML: https://mlsanity.com/paper/2009.11795/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2020.

## Built on by (most cited first, 14 of 14)

- [Modeling Event Plausibility with Consistent Conceptual Abstraction](https://mlsanity.com/paper/2104.10247/) (2021)
- [ArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD](https://mlsanity.com/paper/2205.09685/) (2022)
- [A Review of the Trends and Challenges in Adopting Natural Language Processing Methods for Education Feedback Analysis](https://mlsanity.com/paper/2301.08826/) (2023)
- [SALMA: Arabic Sense-Annotated Corpus and WSD Benchmarks](https://mlsanity.com/paper/2310.19029/) (2023)
- [What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories](https://mlsanity.com/paper/2302.03353/) (2023)
- [Context-Gloss Augmentation for Improving Arabic Target Sense Verification](https://mlsanity.com/paper/2302.03126/) (2023)
- [Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories](https://mlsanity.com/paper/2110.14091/) (2021)
- [Language Models Are Poor Learners of Directional Inference](https://mlsanity.com/paper/2210.04695/) (2022)
- [Word Sense Disambiguation as a Game of Neurosymbolic Darts](https://mlsanity.com/paper/2307.16663/) (2023)
- [Topological Information Retrieval with Dilation-Invariant Bottleneck Comparative Measures](https://mlsanity.com/paper/2104.01672/) (2021)
- [ContrastWSD: Enhancing Metaphor Detection with Word Sense Disambiguation Following the Metaphor Identification Procedure](https://mlsanity.com/paper/2309.03103/) (2023)
- [PolyBERT: Fine-Tuned Poly Encoder BERT-Based Model for Word Sense Disambiguation](https://mlsanity.com/paper/2506.00968/) (2025)
- [Lost in Context? On the Sense-wise Variance of Contextualized Word Embeddings](https://mlsanity.com/paper/2208.09669/) (2022)
- [Word Sense Disambiguation in Native Spanish: A Comprehensive Lexical Evaluation Resource](https://mlsanity.com/paper/2409.20524/) (2024)

## Abstract (arXiv)

Domain adaptation or transfer learning using pre-trained language models such as BERT has proven to be an effective approach for many natural language processing tasks. In this work, we propose to formulate word sense disambiguation as a relevance ranking task, and fine-tune BERT on sequence-pair ranking task to select the most probable sense definition given a context sentence and a list of candidate sense definitions. We also introduce a data augmentation technique for WSD using existing example sentences from WordNet. Using the proposed training objective and data augmentation technique, our models are able to achieve state-of-the-art results on the English all-words benchmark datasets.