# Diachronic word embeddings and semantic shifts: a survey

arXiv: https://arxiv.org/abs/1806.03537 · HTML: https://mlsanity.com/paper/1806.03537/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2018.

## Built on by (most cited first, 30 of 92)

- [Time Masking for Temporal Language Models](https://mlsanity.com/paper/2110.06366/) (2021)
- [Explaining and Improving BERT Performance on Lexical Semantic Change Detection](https://mlsanity.com/paper/2103.07259/) (2021)
- [Dynamic Language Models for Continuously Evolving Content](https://mlsanity.com/paper/2106.06297/) (2021)
- [A Survey on Contextualised Semantic Shift Detection](https://mlsanity.com/paper/2304.01666/) (2023)
- [Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora](https://mlsanity.com/paper/2112.14330/) (2021)
- [Temporal Attention for Language Models](https://mlsanity.com/paper/2202.02093/) (2022)
- [A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change](https://mlsanity.com/paper/2402.12011/) (2024)
- [Contextualized language models for semantic change detection: lessons learned](https://mlsanity.com/paper/2209.00154/) (2022)
- [Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis](https://mlsanity.com/paper/2305.11993/) (2023)
- [COVID-19 sentiment analysis via deep learning during the rise of novel cases](https://mlsanity.com/paper/2104.10662/) (2021)
- [A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication](https://mlsanity.com/paper/2111.04424/) (2021)
- [Statistically significant detection of semantic shifts using contextual word embeddings](https://mlsanity.com/paper/2104.03776/) (2021)
- [Lexical Semantic Change Discovery](https://mlsanity.com/paper/2106.03111/) (2021)
- [Three-part diachronic semantic change dataset for Russian](https://mlsanity.com/paper/2106.08294/) (2021)
- [HistBERT: A Pre-trained Language Model for Diachronic Lexical Semantic Analysis](https://mlsanity.com/paper/2202.03612/) (2022)
- [LSCDiscovery: A shared task on semantic change discovery and detection in Spanish](https://mlsanity.com/paper/2205.06691/) (2022)
- [Grammatical Profiling for Semantic Change Detection](https://mlsanity.com/paper/2109.10397/) (2021)
- [Can Word Sense Distribution Detect Semantic Changes of Words?](https://mlsanity.com/paper/2310.10400/) (2023)
- [Substitution-based Semantic Change Detection using Contextual Embeddings](https://mlsanity.com/paper/2309.02403/) (2023)
- [Building for Tomorrow: Assessing the Temporal Persistence of Text Classifiers](https://mlsanity.com/paper/2205.05435/) (2022)
- [What company do words keep? Revisiting the distributional semantics of J.R. Firth & Zellig Harris](https://mlsanity.com/paper/2205.07750/) (2022)
- [Unsupervised Semantic Variation Prediction using the Distribution of Sibling Embeddings](https://mlsanity.com/paper/2305.08654/) (2023)
- [Distilling Semantic Concept Embeddings from Contrastively Fine-Tuned Language Models](https://mlsanity.com/paper/2305.09785/) (2023)
- [(Chat)GPT v BERT: Dawn of Justice for Semantic Change Detection](https://mlsanity.com/paper/2401.14040/) (2024)
- [NorDiaChange: Diachronic Semantic Change Dataset for Norwegian](https://mlsanity.com/paper/2201.05123/) (2022)
- [Time Machine GPT](https://mlsanity.com/paper/2404.18543/) (2024)
- [Abolitionist Networks: Modeling Language Change in Nineteenth-Century Activist Newspapers](https://mlsanity.com/paper/2103.07538/) (2021)
- [textitSwap and Predict -- Predicting the Semantic Changes in Words across Corpora by Context Swapping](https://mlsanity.com/paper/2310.10397/) (2023)
- [Theoretical foundations and limits of word embeddings: what types of meaning can they capture?](https://mlsanity.com/paper/2107.10413/) (2021)
- [Survey in Characterizing Semantic Change](https://mlsanity.com/paper/2402.19088/) (2024)

## Abstract (arXiv)

Recent years have witnessed a surge of publications aimed at tracing temporal changes in lexical semantics using distributional methods, particularly prediction-based word embedding models. However, this vein of research lacks the cohesion, common terminology and shared practices of more established areas of natural language processing. In this paper, we survey the current state of academic research related to diachronic word embeddings and semantic shifts detection. We start with discussing the notion of semantic shifts, and then continue with an overview of the existing methods for tracing such time-related shifts with word embedding models. We propose several axes along which these methods can be compared, and outline the main challenges before this emerging subfield of NLP, as well as prospects and possible applications.