# GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge

arXiv: https://arxiv.org/abs/1908.07245 · HTML: https://mlsanity.com/paper/1908.07245/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2019.

## Built on by (most cited first, 30 of 79)

- [CAT: A Contextualized Conceptualization and Instantiation Framework for Commonsense Reasoning](https://mlsanity.com/paper/2305.04808/) (2023)
- [NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence Prediction](https://mlsanity.com/paper/2109.03564/) (2021)
- [ArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD](https://mlsanity.com/paper/2205.09685/) (2022)
- [EmpathBERT: A BERT-based Framework for Demographic-aware Empathy Prediction](https://mlsanity.com/paper/2102.00272/) (2021)
- [LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond](https://mlsanity.com/paper/2105.12449/) (2021)
- [Acquiring and Modelling Abstract Commonsense Knowledge via Conceptualization](https://mlsanity.com/paper/2206.01532/) (2022)
- [AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph](https://mlsanity.com/paper/2311.09174/) (2023)
- [AbsInstruct: Eliciting Abstraction Ability from LLMs through Explanation Tuning with Plausibility Estimation](https://mlsanity.com/paper/2402.10646/) (2024)
- [Driving Context into Text-to-Text Privatization](https://mlsanity.com/paper/2306.01457/) (2023)
- [FEWS: Large-Scale, Low-Shot Word Sense Disambiguation with the Dictionary](https://mlsanity.com/paper/2102.07983/) (2021)
- [A Review of the Trends and Challenges in Adopting Natural Language Processing Methods for Education Feedback Analysis](https://mlsanity.com/paper/2301.08826/) (2023)
- [KI-BERT: Infusing Knowledge Context for Better Language and Domain Understanding](https://mlsanity.com/paper/2104.08145/) (2021)
- [Efficient Zero-shot Event Extraction with Context-Definition Alignment](https://mlsanity.com/paper/2211.05156/) (2022)
- [SALMA: Arabic Sense-Annotated Corpus and WSD Benchmarks](https://mlsanity.com/paper/2310.19029/) (2023)
- [What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories](https://mlsanity.com/paper/2302.03353/) (2023)
- [Incorporating Word Sense Disambiguation in Neural Language Models](https://mlsanity.com/paper/2106.07967/) (2021)
- [LexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution](https://mlsanity.com/paper/2107.05132/) (2021)
- [Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy](https://mlsanity.com/paper/2110.02204/) (2021)
- [Context-Gloss Augmentation for Improving Arabic Target Sense Verification](https://mlsanity.com/paper/2302.03126/) (2023)
- [PolyLM: Learning about Polysemy through Language Modeling](https://mlsanity.com/paper/2101.10448/) (2021)
- [Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories](https://mlsanity.com/paper/2110.14091/) (2021)
- [Multilingual Word Sense Disambiguation with Unified Sense Representation](https://mlsanity.com/paper/2210.07447/) (2022)
- [Vision Meets Definitions: Unsupervised Visual Word Sense Disambiguation Incorporating Gloss Information](https://mlsanity.com/paper/2305.01788/) (2023)
- [Ambiguity Meets Uncertainty: Investigating Uncertainty Estimation for Word Sense Disambiguation](https://mlsanity.com/paper/2305.13119/) (2023)
- [Resolving Regular Polysemy in Named Entities](https://mlsanity.com/paper/2401.09758/) (2024)
- [Multiple Sources are Better Than One: Incorporating External Knowledge in Low-Resource Glossing](https://mlsanity.com/paper/2406.11085/) (2024)
- [Non-Parametric Few-Shot Learning for Word Sense Disambiguation](https://mlsanity.com/paper/2104.12677/) (2021)
- [Semantic Role Labeling Meets Definition Modeling: Using Natural Language to Describe Predicate-Argument Structures](https://mlsanity.com/paper/2212.01094/) (2022)
- [Neural Sequence-to-Sequence Modeling with Attention by Leveraging Deep Learning Architectures for Enhanced Contextual Understanding in Abstractive Text Summarization](https://mlsanity.com/paper/2404.08685/) (2024)
- [IITK@LCP at SemEval 2021 Task 1: Classification for Lexical Complexity Regression Task](https://mlsanity.com/paper/2104.01046/) (2021)

## Abstract (arXiv)

Word Sense Disambiguation (WSD) aims to find the exact sense of an ambiguous word in a particular context. Traditional supervised methods rarely take into consideration the lexical resources like WordNet, which are widely utilized in knowledge-based methods. Recent studies have shown the effectiveness of incorporating gloss (sense definition) into neural networks for WSD. However, compared with traditional word expert supervised methods, they have not achieved much improvement. In this paper, we focus on how to better leverage gloss knowledge in a supervised neural WSD system. We construct context-gloss pairs and propose three BERT-based models for WSD. We fine-tune the pre-trained BERT model on SemCor3.0 training corpus and the experimental results on several English all-words WSD benchmark datasets show that our approach outperforms the state-of-the-art systems.