# Analyzing Continuous Semantic Shifts with Diachronic Word Similarity Matrices

arXiv: https://arxiv.org/abs/2501.09538 · HTML: https://mlsanity.com/paper/2501.09538/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 16 Jan 2025.

## Builds on (most cited first, 22 of 22)

- [Mind the Gap: Assessing Temporal Generalization in Neural Language Models](https://mlsanity.com/paper/2102.01951/) (2021)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection](https://mlsanity.com/paper/2007.11464/) (2020)
- [Diachronic word embeddings and semantic shifts: a survey](https://mlsanity.com/paper/1806.03537/) (2018)
- [Analysing Lexical Semantic Change with Contextualised Word Representations](https://mlsanity.com/paper/2004.14118/) (2020)
- [Statistically Significant Detection of Linguistic Change](https://mlsanity.com/paper/1411.3315/) (2014)
- [Time Masking for Temporal Language Models](https://mlsanity.com/paper/2110.06366/) (2021)
- [Dynamic Word Embeddings for Evolving Semantic Discovery](https://mlsanity.com/paper/1703.00607/) (2017)
- [Explaining and Improving BERT Performance on Lexical Semantic Change Detection](https://mlsanity.com/paper/2103.07259/) (2021)
- [Time is Encoded in the Weights of Finetuned Language Models](https://mlsanity.com/paper/2312.13401/) (2023)
- [A Survey on Contextualised Semantic Shift Detection](https://mlsanity.com/paper/2304.01666/) (2023)
- [A Novel Geometric Framework on Gram Matrix Trajectories for Human Behavior Understanding](https://mlsanity.com/paper/1807.00676/) (2018)
- [A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change](https://mlsanity.com/paper/2402.12011/) (2024)
- [Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change](https://mlsanity.com/paper/2210.17127/) (2022)
- [Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labels](https://mlsanity.com/paper/2309.09697/) (2023)
- [Using Synchronic Definitions and Semantic Relations to Classify Semantic Change Types](https://mlsanity.com/paper/2406.03452/) (2024)
- [Analyzing Semantic Change through Lexical Replacements](https://mlsanity.com/paper/2404.18570/) (2024)
- [Frequency patterns of semantic change: Corpus-based evidence of a near-critical dynamics in language change](https://mlsanity.com/paper/1703.00203/) (2017)
- [Survey in Characterizing Semantic Change](https://mlsanity.com/paper/2402.19088/) (2024)
- [A Semantic Distance Metric Learning approach for Lexical Semantic Change Detection](https://mlsanity.com/paper/2403.00226/) (2024)
- [A Multidimensional Framework for Evaluating Lexical Semantic Change with Social Science Applications](https://mlsanity.com/paper/2406.06052/) (2024)
- [Improving Deep Learning Sound Events Classifiers using Gram Matrix Feature-wise Correlations](https://mlsanity.com/paper/2102.11771/) (2021)

## Built on by (most cited first, 3 of 3)

- [LSC-Eval: A General Framework to Evaluate Methods for Assessing Dimensions of Lexical Semantic Change Using LLM-Generated Synthetic Data](https://mlsanity.com/paper/2503.08042/) (2025)
- [Word Sense Detection Leveraging Maximum Mean Discrepancy](https://mlsanity.com/paper/2506.01602/) (2025)
- [Coupled Usage-Sense Processes: Temporal and Attributable Lexical Semantic Change](https://mlsanity.com/paper/2609.30974/) (2026)

## Abstract (arXiv)

The meanings and relationships of words shift over time. This phenomenon is referred to as semantic shift. Research focused on understanding how semantic shifts occur over multiple time periods is essential for gaining a detailed understanding of semantic shifts. However, detecting change points only between adjacent time periods is insufficient for analyzing detailed semantic shifts, and using BERT-based methods to examine word sense proportions incurs a high computational cost. To address those issues, we propose a simple yet intuitive framework for how semantic shifts occur over multiple time periods by leveraging a similarity matrix between the embeddings of the same word through time. We compute a diachronic word similarity matrix using fast and lightweight word embeddings across arbitrary time periods, making it deeper to analyze continuous semantic shifts. Additionally, by clustering the similarity matrices for different words, we can categorize words that exhibit similar behavior of semantic shift in an unsupervised manner.