# Can Word Sense Distribution Detect Semantic Changes of Words?

arXiv: https://arxiv.org/abs/2310.10400 · HTML: https://mlsanity.com/paper/2310.10400/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 77 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations](https://mlsanity.com/paper/1808.09121/) (2018)
- [Mind the Gap: Assessing Temporal Generalization in Neural Language Models](https://mlsanity.com/paper/2102.01951/) (2021)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [TimeLMs: Diachronic Language Models from Twitter](https://mlsanity.com/paper/2202.03829/) (2022)
- [SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection](https://mlsanity.com/paper/2007.11464/) (2020)
- [Word Representations via Gaussian Embedding](https://mlsanity.com/paper/1412.6623/) (2014)
- [Evaluating the Underlying Gender Bias in Contextualized Word Embeddings](https://mlsanity.com/paper/1904.08783/) (2019)
- [Diachronic word embeddings and semantic shifts: a survey](https://mlsanity.com/paper/1806.03537/) (2018)
- [Analysing Lexical Semantic Change with Contextualised Word Representations](https://mlsanity.com/paper/2004.14118/) (2020)
- [Statistically Significant Detection of Linguistic Change](https://mlsanity.com/paper/1411.3315/) (2014)
- [Survey of Computational Approaches to Lexical Semantic Change](https://mlsanity.com/paper/1811.06278/) (2018)
- [Temporal Analysis of Language through Neural Language Models](https://mlsanity.com/paper/1405.3515/) (2014)
- [From Word to Sense Embeddings: A Survey on Vector Representations of Meaning](https://mlsanity.com/paper/1805.04032/) (2018)
- [Time Masking for Temporal Language Models](https://mlsanity.com/paper/2110.06366/) (2021)
- [Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation](https://mlsanity.com/paper/1906.10007/) (2019)
- [Dynamic Word Embeddings for Evolving Semantic Discovery](https://mlsanity.com/paper/1703.00607/) (2017)
- [Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift](https://mlsanity.com/paper/1912.01072/) (2019)
- [Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space](https://mlsanity.com/paper/1504.06654/) (2015)
- [Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change](https://mlsanity.com/paper/1906.01688/) (2019)
- [UiO-UvA at SemEval-2020 Task 1: Contextualised Embeddings for Lexical Semantic Change Detection](https://mlsanity.com/paper/2005.00050/) (2020)
- [Temporal Attention for Language Models](https://mlsanity.com/paper/2202.02093/) (2022)
- [Contextualized language models for semantic change detection: lessons learned](https://mlsanity.com/paper/2209.00154/) (2022)
- [AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes](https://mlsanity.com/paper/1507.01127/) (2015)
- [Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddings](https://mlsanity.com/paper/2203.07523/) (2022)
- [Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change](https://mlsanity.com/paper/2210.17127/) (2022)
- [Grammatical Profiling for Semantic Change Detection](https://mlsanity.com/paper/2109.10397/) (2021)
- [Unsupervised Semantic Variation Prediction using the Distribution of Sibling Embeddings](https://mlsanity.com/paper/2305.08654/) (2023)

## Built on by (most cited first, 10 of 10)

- [A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change](https://mlsanity.com/paper/2402.12011/) (2024)
- [Definition generation for lexical semantic change detection](https://mlsanity.com/paper/2406.14167/) (2024)
- [Survey in Characterizing Semantic Change](https://mlsanity.com/paper/2402.19088/) (2024)
- [A Semantic Distance Metric Learning approach for Lexical Semantic Change Detection](https://mlsanity.com/paper/2403.00226/) (2024)
- [SCDTour: Embedding Axis Ordering and Merging for Interpretable Semantic Change Detection](https://mlsanity.com/paper/2509.11818/) (2025)
- [Evaluating Short-Term Temporal Fluctuations of Social Biases in Social Media Data and Masked Language Models](https://mlsanity.com/paper/2406.13556/) (2024)
- [Transparent Semantic Change Detection with Dependency-Based Profiles](https://mlsanity.com/paper/2601.02891/) (2026)
- [ReFRAME or Remain: Unsupervised Lexical Semantic Change Detection with Frame Semantics](https://mlsanity.com/paper/2602.04514/) (2026)
- [Improving Interpretability of Lexical Semantic Change with Neurobiological Features](https://mlsanity.com/paper/2602.09760/) (2026)
- [Knowledge-driven Augmentation and Retrieval for Integrative Temporal Adaptation](https://mlsanity.com/paper/2604.22098/) (2026)

## Abstract (arXiv)

Semantic Change Detection (SCD) of words is an important task for various NLP applications that must make time-sensitive predictions. Some words are used over time in novel ways to express new meanings, and these new meanings establish themselves as novel senses of existing words. On the other hand, Word Sense Disambiguation (WSD) methods associate ambiguous words with sense ids, depending on the context in which they occur. Given this relationship between WSD and SCD, we explore the possibility of predicting whether a target word has its meaning changed between two corpora collected at different time steps, by comparing the distributions of senses of that word in each corpora. For this purpose, we use pretrained static sense embeddings to automatically annotate each occurrence of the target word in a corpus with a sense id. Next, we compute the distribution of sense ids of a target word in a given corpus. Finally, we use different divergence or distance measures to quantify the semantic change of the target word across the two given corpora. Our experimental results on SemEval 2020 Task 1 dataset show that word sense distributions can be accurately used to predict semantic changes of words in English, German, Swedish and Latin.