# Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis

arXiv: https://arxiv.org/abs/2305.11993 · HTML: https://mlsanity.com/paper/2305.11993/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 19 May 2023.

## Builds on (most cited first, 17 of 17)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection](https://mlsanity.com/paper/2007.11464/) (2020)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Diachronic word embeddings and semantic shifts: a survey](https://mlsanity.com/paper/1806.03537/) (2018)
- [Analysing Lexical Semantic Change with Contextualised Word Representations](https://mlsanity.com/paper/2004.14118/) (2020)
- [Survey of Computational Approaches to Lexical Semantic Change](https://mlsanity.com/paper/1811.06278/) (2018)
- [Definition Modeling: Learning to define word embeddings in natural language](https://mlsanity.com/paper/1612.00394/) (2016)
- [Conditional Generators of Words Definitions](https://mlsanity.com/paper/1806.10090/) (2018)
- [DWUG: A large Resource of Diachronic Word Usage Graphs in Four Languages](https://mlsanity.com/paper/2104.08540/) (2021)
- [Learning to Explain Non-Standard English Words and Phrases](https://mlsanity.com/paper/1709.09254/) (2017)
- [Multitasking Framework for Unsupervised Simple Definition Generation](https://mlsanity.com/paper/2203.12926/) (2022)
- [Semeval-2022 Task 1: CODWOE -- Comparing Dictionaries and Word Embeddings](https://mlsanity.com/paper/2205.13858/) (2022)
- [Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling](https://mlsanity.com/paper/1911.05715/) (2019)
- [Understanding Jargon: Combining Extraction and Generation for Definition Modeling](https://mlsanity.com/paper/2111.07267/) (2021)
- [Lexical Semantic Change Discovery](https://mlsanity.com/paper/2106.03111/) (2021)

## Built on by (most cited first, 19 of 19)

- [SemEval-2024 Shared Task 6: SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes](https://mlsanity.com/paper/2403.07726/) (2024)
- [Using Synchronic Definitions and Semantic Relations to Classify Semantic Change Types](https://mlsanity.com/paper/2406.03452/) (2024)
- [Definition generation for lexical semantic change detection](https://mlsanity.com/paper/2406.14167/) (2024)
- [Survey in Characterizing Semantic Change](https://mlsanity.com/paper/2402.19088/) (2024)
- [Semantic Change Characterization with LLMs using Rhetorics](https://mlsanity.com/paper/2407.16624/) (2024)
- [Rapid Word Learning Through Meta In-Context Learning](https://mlsanity.com/paper/2502.14791/) (2025)
- [SCDTour: Embedding Axis Ordering and Merging for Interpretable Semantic Change Detection](https://mlsanity.com/paper/2509.11818/) (2025)
- [CoLLEGe: Concept Embedding Generation for Large Language Models](https://mlsanity.com/paper/2403.15362/) (2024)
- [Presence or Absence: Are Unknown Word Usages in Dictionaries?](https://mlsanity.com/paper/2406.00656/) (2024)
- [AXOLOTL'24 Shared Task on Multilingual Explainable Semantic Change Modeling](https://mlsanity.com/paper/2407.04079/) (2024)
- [Zero-Shot Multi-task Hallucination Detection](https://mlsanity.com/paper/2403.12244/) (2024)
- [Enriching Word Usage Graphs with Cluster Definitions](https://mlsanity.com/paper/2403.18024/) (2024)
- [CHEW: A Dataset of CHanging Events in Wikipedia](https://mlsanity.com/paper/2406.19116/) (2024)
- [Improving Interpretability of Lexical Semantic Change with Neurobiological Features](https://mlsanity.com/paper/2602.09760/) (2026)
- [GEAR: A Simple GENERATE, EMBED, AVERAGE AND RANK Approach for Unsupervised Reverse Dictionary](https://mlsanity.com/paper/2412.06654/) (2024)
- [Explaining novel senses using definition generation with open language models](https://mlsanity.com/paper/2509.26181/) (2025)
- [LM-Lexicon: Improving Definition Modeling via Harmonizing Semantic Experts](https://mlsanity.com/paper/2602.14060/) (2026)
- [SemBench: A Universal Semantic Framework for LLM Evaluation](https://mlsanity.com/paper/2603.11687/) (2026)
- [Measuring the Semantic Structure and Evolution of Conspiracy Theories](https://mlsanity.com/paper/2603.26062/) (2026)

## Abstract (arXiv)

We propose using automatically generated natural language definitions of contextualised word usages as interpretable word and word sense representations. Given a collection of usage examples for a target word, and the corresponding data-driven usage clusters (i.e., word senses), a definition is generated for each usage with a specialised Flan-T5 language model, and the most prototypical definition in a usage cluster is chosen as the sense label. We demonstrate how the resulting sense labels can make existing approaches to semantic change analysis more interpretable, and how they can allow users -- historical linguists, lexicographers, or social scientists -- to explore and intuitively explain diachronic trajectories of word meaning. Semantic change analysis is only one of many possible applications of the `definitions as representations' paradigm. Beyond being human-readable, contextualised definitions also outperform token or usage sentence embeddings in word-in-context semantic similarity judgements, making them a new promising type of lexical representation for NLP.