# Exploiting Correlations Between Contexts and Definitions with Multiple Definition Modeling

arXiv: https://arxiv.org/abs/2305.14717 · HTML: https://mlsanity.com/paper/2305.14717/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks](https://mlsanity.com/paper/2202.12499/) (2022)
- [Definition Modeling: Learning to define word embeddings in natural language](https://mlsanity.com/paper/1612.00394/) (2016)
- [Conditional Generators of Words Definitions](https://mlsanity.com/paper/1806.10090/) (2018)
- [Learning to Explain Non-Standard English Words and Phrases](https://mlsanity.com/paper/1709.09254/) (2017)
- [Multitasking Framework for Unsupervised Simple Definition Generation](https://mlsanity.com/paper/2203.12926/) (2022)
- [Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling](https://mlsanity.com/paper/1911.05715/) (2019)
- [VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling](https://mlsanity.com/paper/2010.03124/) (2020)
- [Breaking Sticks and Ambiguities with Adaptive Skip-gram](https://mlsanity.com/paper/1502.07257/) (2015)
- [xSense: Learning Sense-Separated Sparse Representations and Textual Definitions for Explainable Word Sense Networks](https://mlsanity.com/paper/1809.03348/) (2018)
- [Knowledge Distillation Transfer Sets and their Impact on Downstream NLU Tasks](https://mlsanity.com/paper/2210.04834/) (2022)
- [Self-supervised Speaker Recognition Training Using Human-Machine Dialogues](https://mlsanity.com/paper/2202.03484/) (2022)

## Built on by (most cited first, 1 of 1)

- [LM-Lexicon: Improving Definition Modeling via Harmonizing Semantic Experts](https://mlsanity.com/paper/2602.14060/) (2026)

## Abstract (arXiv)

Definition modeling is an important task in advanced natural language applications such as understanding and conversation. Since its introduction, it focus on generating one definition for a target word or phrase in a given context, which we refer to as Single Definition Modeling (SDM). However, this approach does not adequately model the correlations and patterns among different contexts and definitions of words. In addition, the creation of a training dataset for SDM requires significant human expertise and effort. In this paper, we carefully design a new task called Multiple Definition Modeling (MDM) that pool together all contexts and definition of target words. We demonstrate the ease of creating a model as well as multiple training sets automatically. % In the experiments, we demonstrate and analyze the benefits of MDM, including improving SDM's performance by using MDM as the pretraining task and its comparable performance in the zero-shot setting.