# SMSMix: Sense-Maintained Sentence Mixup for Word Sense Disambiguation

arXiv: https://arxiv.org/abs/2212.07072 · HTML: https://mlsanity.com/paper/2212.07072/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 14 Dec 2022.

## Builds on (most cited first, 11 of 11)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Remix: Rebalanced Mixup](https://mlsanity.com/paper/2007.03943/) (2020)
- [GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge](https://mlsanity.com/paper/1908.07245/) (2019)
- [Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders](https://mlsanity.com/paper/2005.02590/) (2020)
- [On the Generalization Effects of Linear Transformations in Data Augmentation](https://mlsanity.com/paper/2005.00695/) (2020)
- [Balanced-MixUp for Highly Imbalanced Medical Image Classification](https://mlsanity.com/paper/2109.09850/) (2021)
- [SSMix: Saliency-Based Span Mixup for Text Classification](https://mlsanity.com/paper/2106.08062/) (2021)
- [Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data](https://mlsanity.com/paper/1909.09148/) (2019)
- [Two-stage Training for Learning from Label Proportions](https://mlsanity.com/paper/2105.10635/) (2021)
- [MixBoost: Synthetic Oversampling with Boosted Mixup for Handling Extreme Imbalance](https://mlsanity.com/paper/2009.01571/) (2020)

## Built on by (most cited first, 2 of 2)

- [C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion](https://mlsanity.com/paper/2403.14119/) (2024)
- [INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-Native Speech Recognition](https://mlsanity.com/paper/2305.16371/) (2023)

## Abstract (arXiv)

Word Sense Disambiguation (WSD) is an NLP task aimed at determining the correct sense of a word in a sentence from discrete sense choices. Although current systems have attained unprecedented performances for such tasks, the nonuniform distribution of word senses during training generally results in systems performing poorly on rare senses. To this end, we consider data augmentation to increase the frequency of these least frequent senses (LFS) to reduce the distributional bias of senses during training. We propose Sense-Maintained Sentence Mixup (SMSMix), a novel word-level mixup method that maintains the sense of a target word. SMSMix smoothly blends two sentences using mask prediction while preserving the relevant span determined by saliency scores to maintain a specific word's sense. To the best of our knowledge, this is the first attempt to apply mixup in NLP while preserving the meaning of a specific word. With extensive experiments, we validate that our augmentation method can effectively give more information about rare senses during training with maintained target sense label.