# Substitution-based Semantic Change Detection using Contextual Embeddings

arXiv: https://arxiv.org/abs/2309.02403 · HTML: https://mlsanity.com/paper/2309.02403/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 5 Sep 2023.

## Builds on (most cited first, 16 of 16)

- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection](https://mlsanity.com/paper/2007.11464/) (2020)
- [Diachronic word embeddings and semantic shifts: a survey](https://mlsanity.com/paper/1806.03537/) (2018)
- [Analysing Lexical Semantic Change with Contextualised Word Representations](https://mlsanity.com/paper/2004.14118/) (2020)
- [Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Change](https://mlsanity.com/paper/1606.02821/) (2016)
- [Survey of Computational Approaches to Lexical Semantic Change](https://mlsanity.com/paper/1811.06278/) (2018)
- [Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift](https://mlsanity.com/paper/1912.01072/) (2019)
- [Capturing Evolution in Word Usage: Just Add More Clusters?](https://mlsanity.com/paper/2001.06629/) (2020)
- [Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora](https://mlsanity.com/paper/2112.14330/) (2021)
- [A State-of-the-Art of Semantic Change Computation](https://mlsanity.com/paper/1801.09872/) (2018)
- [Statistically significant detection of semantic shifts using contextual word embeddings](https://mlsanity.com/paper/2104.03776/) (2021)
- [Lexical Semantic Change Discovery](https://mlsanity.com/paper/2106.03111/) (2021)
- [LSCDiscovery: A shared task on semantic change discovery and detection in Spanish](https://mlsanity.com/paper/2205.06691/) (2022)
- [CIRCE at SemEval-2020 Task 1: Ensembling Context-Free and Context-Dependent Word Representations](https://mlsanity.com/paper/2005.06602/) (2020)
- [Large Scale Substitution-based Word Sense Induction](https://mlsanity.com/paper/2110.07681/) (2021)
- [BOS at LSCDiscovery: Lexical Substitution for Interpretable Lexical Semantic Change Detection](https://mlsanity.com/paper/2206.11865/) (2022)

## Built on by (most cited first, 9 of 9)

- [Graph-based Clustering for Detecting Semantic Change Across Time and Languages](https://mlsanity.com/paper/2402.01025/) (2024)
- [Analyzing Semantic Change through Lexical Replacements](https://mlsanity.com/paper/2404.18570/) (2024)
- [A Semantic Distance Metric Learning approach for Lexical Semantic Change Detection](https://mlsanity.com/paper/2403.00226/) (2024)
- [Tracking Semantic Change in Slovene: A Novel Dataset and Optimal Transport-Based Distance](https://mlsanity.com/paper/2402.16596/) (2024)
- [Improving Interpretability of Lexical Semantic Change with Neurobiological Features](https://mlsanity.com/paper/2602.09760/) (2026)
- [DHPLT: large-scale multilingual diachronic corpora and word representations for semantic change modelling](https://mlsanity.com/paper/2602.11968/) (2026)
- [The Prediction-Measurement Gap: Toward Meaning Representations as Scientific Instruments](https://mlsanity.com/paper/2603.10130/) (2026)
- [Words and Action: Modeling Linguistic Leadership in #BlackLivesMatter Communities](https://mlsanity.com/paper/2412.02637/) (2024)
- [A Computational Framework for Modelling Organisation-Level Semantic Identity from Longitudinal Textual Data](https://mlsanity.com/paper/2609.29584/) (2026)

## Abstract (arXiv)

Measuring semantic change has thus far remained a task where methods using contextual embeddings have struggled to improve upon simpler techniques relying only on static word vectors. Moreover, many of the previously proposed approaches suffer from downsides related to scalability and ease of interpretation. We present a simplified approach to measuring semantic change using contextual embeddings, relying only on the most probable substitutes for masked terms. Not only is this approach directly interpretable, it is also far more efficient in terms of storage, achieves superior average performance across the most frequently cited datasets for this task, and allows for more nuanced investigation of change than is possible with static word vectors.