# A Survey on Contextualised Semantic Shift Detection

arXiv: https://arxiv.org/abs/2304.01666 · HTML: https://mlsanity.com/paper/2304.01666/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 157 references that are in the map.
- First posted on arXiv 4 Apr 2023.

## Builds on (most cited first, 30 of 61)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations](https://mlsanity.com/paper/1808.09121/) (2018)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://mlsanity.com/paper/1908.08962/) (2019)
- [Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language](https://mlsanity.com/paper/1905.07213/) (2019)
- [SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection](https://mlsanity.com/paper/2007.11464/) (2020)
- [Debiasing Pre-trained Contextualised Embeddings](https://mlsanity.com/paper/2101.09523/) (2021)
- [Diachronic word embeddings and semantic shifts: a survey](https://mlsanity.com/paper/1806.03537/) (2018)
- [Analysing Lexical Semantic Change with Contextualised Word Representations](https://mlsanity.com/paper/2004.14118/) (2020)
- [Survey of Computational Approaches to Lexical Semantic Change](https://mlsanity.com/paper/1811.06278/) (2018)
- [Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders](https://mlsanity.com/paper/2005.02590/) (2020)
- [Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media](https://mlsanity.com/paper/2104.08116/) (2021)
- [Time Masking for Temporal Language Models](https://mlsanity.com/paper/2110.06366/) (2021)
- [Latin BERT: A Contextual Language Model for Classical Philology](https://mlsanity.com/paper/2009.10053/) (2020)
- [Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Senses](https://mlsanity.com/paper/2104.14694/) (2021)
- [Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift](https://mlsanity.com/paper/1912.01072/) (2019)
- [Diachronic Usage Relatedness (DURel): A Framework for the Annotation of Lexical Semantic Change](https://mlsanity.com/paper/1804.06517/) (2018)
- [Explaining and Improving BERT Performance on Lexical Semantic Change Detection](https://mlsanity.com/paper/2103.07259/) (2021)

## Built on by (most cited first, 22 of 22)

- [Pretraining Language Models for Diachronic Linguistic Change Discovery](https://mlsanity.com/paper/2504.05523/) (2025)
- [Concept Drift Adaptation in Text Stream Mining Settings: A Systematic Review](https://mlsanity.com/paper/2312.02901/) (2023)
- [Turkronicles: Diachronic Resources for the Fast Evolving Turkish Language](https://mlsanity.com/paper/2405.10133/) (2024)
- [Tracing the Development of the Virtual Particle Concept Using Semantic Change Detection](https://mlsanity.com/paper/2410.16855/) (2024)
- [Quantifying Lexical Semantic Shift via Unbalanced Optimal Transport](https://mlsanity.com/paper/2412.12569/) (2024)
- [Analyzing Continuous Semantic Shifts with Diachronic Word Similarity Matrices](https://mlsanity.com/paper/2501.09538/) (2025)
- [Incremental Affinity Propagation based on Cluster Consolidation and Stratification](https://mlsanity.com/paper/2401.14439/) (2024)
- [How BERT Speaks Shakespearean English? Evaluating Historical Bias in Contextual Language Models](https://mlsanity.com/paper/2402.05034/) (2024)
- [Large Language Models for History, Philosophy, and Sociology of Science: Interpretive Uses, Methodological Challenges, and Critical Perspectives](https://mlsanity.com/paper/2506.12242/) (2025)
- [TiMoE: Time-Aware Mixture of Language Experts](https://mlsanity.com/paper/2508.08827/) (2025)
- [Countermind: A Multi-Layered Security Architecture for Large Language Models](https://mlsanity.com/paper/2510.11837/) (2025)
- [Responsible LLM Deployment for High-Stake Decisions by Decentralized Technologies and Human-AI Interactions](https://mlsanity.com/paper/2512.04108/) (2025)
- [Tracking Semantic Change in Slovene: A Novel Dataset and Optimal Transport-Based Distance](https://mlsanity.com/paper/2402.16596/) (2024)
- [Identifying Emerging Concepts in Large Corpora](https://mlsanity.com/paper/2502.21315/) (2025)
- [Beyond Text: Characterizing Domain Expert Needs in Document Research](https://mlsanity.com/paper/2504.12495/) (2025)
- [From Memorization to Generalization: Fine-Tuning Large Language Models for Biomedical Term-to-Identifier Normalization](https://mlsanity.com/paper/2510.19036/) (2025)
- [Values That Are Explicitly Present in Fairy Tales: Comparing Samples from German, Italian and Portuguese Traditions](https://mlsanity.com/paper/2402.08318/) (2024)
- [Towards Universal Semantics With Large Language Models](https://mlsanity.com/paper/2505.11764/) (2025)
- [Word Sense Detection Leveraging Maximum Mean Discrepancy](https://mlsanity.com/paper/2506.01602/) (2025)
- [Computational conceptual history of scientific concepts: From early digital methods to LLMs](https://mlsanity.com/paper/2606.04118/) (2026)
- [Beyond frequency measures: Can contextual embeddings capture meaning change in scientific texts?](https://mlsanity.com/paper/2609.18804/) (2026)
- [Large Language Models, Machine-Assisted Cultural Analytics, and the Agentic Turn](https://mlsanity.com/paper/2610.12792/) (2026)

## Abstract (arXiv)

Semantic Shift Detection (SSD) is the task of identifying, interpreting, and assessing the possible change over time in the meanings of a target word. Traditionally, SSD has been addressed by linguists and social scientists through manual and time-consuming activities. In the recent years, computational approaches based on Natural Language Processing and word embeddings gained increasing attention to automate SSD as much as possible. In particular, over the past three years, significant advancements have been made almost exclusively based on word contextualised embedding models, which can handle the multiple usages/meanings of the words and better capture the related semantic shifts. In this paper, we survey the approaches based on contextualised embeddings for SSD (i.e., CSSDetection) and we propose a classification framework characterised by meaning representation, time-awareness, and learning modality dimensions. The framework is exploited i) to review the measures for shift assessment, ii) to compare the approaches on performance, and iii) to discuss the current issues in terms of scalability, interpretability, and robustness. Open challenges and future research directions about CSSDetection are finally outlined.