# Building on Huang et al. GlossBERT for Word Sense Disambiguation

arXiv: https://arxiv.org/abs/2112.07089 · HTML: https://mlsanity.com/paper/2112.07089/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 14 references that are in the map.
- First posted on arXiv 14 Dec 2021.

## Builds on (most cited first, 3 of 3)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge](https://mlsanity.com/paper/1908.07245/) (2019)
- [An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation](https://mlsanity.com/paper/1810.07595/) (2018)

## Built on by (most cited first, 2 of 2)

- [Context-Gloss Augmentation for Improving Arabic Target Sense Verification](https://mlsanity.com/paper/2302.03126/) (2023)
- [Atypical lexical abbreviations identification in Russian medical texts](https://mlsanity.com/paper/2206.01987/) (2022)

## Abstract (arXiv)

We propose to take on the problem ofWord Sense Disambiguation (WSD). In language, words of the same form can take different meanings depending on context. While humans easily infer the meaning or gloss of such words by their context, machines stumble on this task.As such, we intend to replicated and expand upon the results of Huang et al.GlossBERT, a model which they design to disambiguate these words (Huang et al.,2019). Specifically, we propose the following augmentations: data-set tweaking(alpha hyper-parameter), ensemble methods, and replacement of BERT with BART andALBERT. The following GitHub repository contains all code used in this report, which extends on the code made available by Huang et al.