# Capturing Evolution in Word Usage: Just Add More Clusters?

arXiv: https://arxiv.org/abs/2001.06629 · HTML: https://mlsanity.com/paper/2001.06629/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jan 2020.

## Built on by (most cited first, 22 of 22)

- [Time Masking for Temporal Language Models](https://mlsanity.com/paper/2110.06366/) (2021)
- [Explaining and Improving BERT Performance on Lexical Semantic Change Detection](https://mlsanity.com/paper/2103.07259/) (2021)
- [A Survey on Contextualised Semantic Shift Detection](https://mlsanity.com/paper/2304.01666/) (2023)
- [Temporal Attention for Language Models](https://mlsanity.com/paper/2202.02093/) (2022)
- [A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change](https://mlsanity.com/paper/2402.12011/) (2024)
- [Contextualized language models for semantic change detection: lessons learned](https://mlsanity.com/paper/2209.00154/) (2022)
- [Slangvolution: A Causal Analysis of Semantic Change and Frequency Dynamics in Slang](https://mlsanity.com/paper/2203.04651/) (2022)
- [Lexical Semantic Change Discovery](https://mlsanity.com/paper/2106.03111/) (2021)
- [Graph-based Clustering for Detecting Semantic Change Across Time and Languages](https://mlsanity.com/paper/2402.01025/) (2024)
- [Substitution-based Semantic Change Detection using Contextual Embeddings](https://mlsanity.com/paper/2309.02403/) (2023)
- [Do Not Fire the Linguist: Grammatical Profiles Help Language Models Detect Semantic Change](https://mlsanity.com/paper/2204.05717/) (2022)
- [A Greek Parliament Proceedings Dataset for Computational Linguistics and Political Analysis](https://mlsanity.com/paper/2210.12883/) (2022)
- [Detection of Non-recorded Word Senses in English and Swedish](https://mlsanity.com/paper/2403.02285/) (2024)
- [Presence or Absence: Are Unknown Word Usages in Dictionaries?](https://mlsanity.com/paper/2406.00656/) (2024)
- [To Word Senses and Beyond: Inducing Concepts with Contextualized Language Models](https://mlsanity.com/paper/2406.20054/) (2024)
- [Tracing the Development of the Virtual Particle Concept Using Semantic Change Detection](https://mlsanity.com/paper/2410.16855/) (2024)
- [Large Language Models on Lexical Semantic Change Detection: An Evaluation](https://mlsanity.com/paper/2312.06002/) (2023)
- [Incremental Affinity Propagation based on Cluster Consolidation and Stratification](https://mlsanity.com/paper/2401.14439/) (2024)
- [Mindful Explanations: Prevalence and Impact of Mind Attribution in XAI Research](https://mlsanity.com/paper/2312.12119/) (2023)
- [Tracking Semantic Change in Slovene: A Novel Dataset and Optimal Transport-Based Distance](https://mlsanity.com/paper/2402.16596/) (2024)
- [Historical Ink: Semantic Shift Detection for 19th Century Spanish](https://mlsanity.com/paper/2407.12852/) (2024)
- [Improving Interpretability of Lexical Semantic Change with Neurobiological Features](https://mlsanity.com/paper/2602.09760/) (2026)

## Abstract (arXiv)

The way the words are used evolves through time, mirroring cultural or technological evolution of society. Semantic change detection is the task of detecting and analysing word evolution in textual data, even in short periods of time. In this paper we focus on a new set of methods relying on contextualised embeddings, a type of semantic modelling that revolutionised the NLP field recently. We leverage the ability of the transformer-based BERT model to generate contextualised embeddings capable of detecting semantic change of words across time. Several approaches are compared in a common setting in order to establish strengths and weaknesses for each of them. We also propose several ideas for improvements, managing to drastically improve the performance of existing approaches.