# Non-Parametric Few-Shot Learning for Word Sense Disambiguation

arXiv: https://arxiv.org/abs/2104.12677 · HTML: https://mlsanity.com/paper/2104.12677/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [FewRel 2.0: Towards More Challenging Few-Shot Relation Classification](https://mlsanity.com/paper/1910.07124/) (2019)
- [GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge](https://mlsanity.com/paper/1908.07245/) (2019)
- [Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders](https://mlsanity.com/paper/2005.02590/) (2020)
- [Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation](https://mlsanity.com/paper/1906.10007/) (2019)
- [Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations](https://mlsanity.com/paper/1910.00194/) (2019)
- [Incorporating Glosses into Neural Word Sense Disambiguation](https://mlsanity.com/paper/1805.08028/) (2018)
- [Learning to Learn to Disambiguate: Meta-Learning for Few-Shot Word Sense Disambiguation](https://mlsanity.com/paper/2004.14355/) (2020)
- [Word Sense Disambiguation using a Bidirectional LSTM](https://mlsanity.com/paper/1606.03568/) (2016)

## Built on by (most cited first, 4 of 4)

- [Evaluating Distributional Distortion in Neural Language Modeling](https://mlsanity.com/paper/2203.12788/) (2022)
- [Few-Sample Named Entity Recognition for Security Vulnerability Reports by Fine-Tuning Pre-Trained Language Models](https://mlsanity.com/paper/2108.06590/) (2021)
- [Word sense extension](https://mlsanity.com/paper/2306.05609/) (2023)
- [Detection of Non-recorded Word Senses in English and Swedish](https://mlsanity.com/paper/2403.02285/) (2024)

## Abstract (arXiv)

Word sense disambiguation (WSD) is a long-standing problem in natural language processing. One significant challenge in supervised all-words WSD is to classify among senses for a majority of words that lie in the long-tail distribution. For instance, 84% of the annotated words have less than 10 examples in the SemCor training data. This issue is more pronounced as the imbalance occurs in both word and sense distributions. In this work, we propose MetricWSD, a non-parametric few-shot learning approach to mitigate this data imbalance issue. By learning to compute distances among the senses of a given word through episodic training, MetricWSD transfers knowledge (a learned metric space) from high-frequency words to infrequent ones. MetricWSD constructs the training episodes tailored to word frequencies and explicitly addresses the problem of the skewed distribution, as opposed to mixing all the words trained with parametric models in previous work. Without resorting to any lexical resources, MetricWSD obtains strong performance against parametric alternatives, achieving a 75.1 F1 score on the unified WSD evaluation benchmark (Raganato et al., 2017b). Our analysis further validates that infrequent words and senses enjoy significant improvement.