mlsanityData to 14 Oct 2026

A Survey on Contextualised Semantic Shift Detection

arXiv 2304.01666 ↗First posted 4 Apr 2023cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

Semantic Shift Detection (SSD) is the task of identifying, interpreting, and assessing the possible change over time in the meanings of a target word. Traditionally, SSD has been addressed by linguists and social scientists through manual and time-consuming activities. In the recent years, computational approaches based on Natural Language Processing and word embeddings gained increasing attention to automate SSD as much as possible. In particular, over the past three years, significant advancements have been made almost exclusively based on word contextualised embedding models, which can handle the multiple usages/meanings of the words and better capture the related semantic shifts. In this paper, we survey the approaches based on contextualised embeddings for SSD (i.e., CSSDetection) and we propose a classification framework characterised by meaning representation, time-awareness, and learning modality dimensions. The framework is exploited i) to review the measures for shift assessment, ii) to compare the approaches on performance, and iii) to discuss the current issues in terms of scalability, interpretability, and robustness. Open challenges and future research directions about CSSDetection are finally outlined.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 61 of its 157 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

61 of 157 references.

References · 61

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  8. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2018Deep contextualized word representationscited by 2,290
  11. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  12. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
38 more
  1. 2014Distributed Representations of Sentences and Documentscited by 678
  2. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 372
  3. 2016Diachronic Word Embeddings Reveal Statistical Laws of Semantic Changecited by 221
  4. 2019Well-Read Students Learn Better: On the Importance of Pre-training Compact Modelscited by 120
  5. 2019Adaptation of Deep Bidirectional Multilingual Transformers for Russian Languagecited by 109
  6. 2020SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detectioncited by 108
  7. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  8. 2018Diachronic word embeddings and semantic shifts: a surveycited by 92
  9. 2020Analysing Lexical Semantic Change with Contextualised Word Representationscited by 92
  10. 2018Survey of Computational Approaches to Lexical Semantic Changecited by 62
  11. 2020Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoderscited by 61
  12. 2021Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Mediacited by 51
  13. 2021Time Masking for Temporal Language Modelscited by 46
  14. 2020Latin BERT: A Contextual Language Model for Classical Philologycited by 43
  15. 2021Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Sensescited by 39
  16. 2019Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shiftcited by 37
  17. 2018Diachronic Usage Relatedness (DURel): A Framework for the Annotation of Lexical Semantic Changecited by 35
  18. 2021Explaining and Improving BERT Performance on Lexical Semantic Change Detectioncited by 34
  19. 2018Factors Influencing the Surprising Instability of Word Embeddingscited by 33
  20. 2019Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Changecited by 33
  21. 2020UiO-UvA at SemEval-2020 Task 1: Contextualised Embeddings for Lexical Semantic Change Detectioncited by 32
  22. 2021DWUG: A large Resource of Diachronic Word Usage Graphs in Four Languagescited by 26
  23. 2018Short-Term Meaning Shift: A Distributional Explorationcited by 23
  24. 2020Capturing Evolution in Word Usage: Just Add More Clusters?cited by 22
  25. 2021Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corporacited by 21
  26. 2022Temporal Attention for Language Modelscited by 21
  27. 2020Dynamic Contextualized Word Embeddingscited by 20
  28. 2021Frequency-based Distortions in Contextualized Word Embeddingscited by 20
  29. 2022Contextualized language models for semantic change detection: lessons learnedcited by 20
  30. 2022TempoWiC: An Evaluation Benchmark for Detecting Meaning Shift in Social Mediacited by 19
  31. 2022Multitasking Framework for Unsupervised Simple Definition Generationcited by 18
  32. 2021Challenges for Computational Lexical Semantic Changecited by 16
  33. 2022Slangvolution: A Causal Analysis of Semantic Change and Frequency Dynamics in Slangcited by 15
  34. 2018A State-of-the-Art of Semantic Change Computationcited by 14
  35. 2021Statistically significant detection of semantic shifts using contextual word embeddingscited by 14
  36. 2021Three-part diachronic semantic change dataset for Russiancited by 11
  37. 2022HistBERT: A Pre-trained Language Model for Diachronic Lexical Semantic Analysiscited by 11
  38. 2022LSCDiscovery: A shared task on semantic change discovery and detection in Spanishcited by 11

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 7; 2025: 11; 2026: 3.

22 papers.

Sort:

By area:

  1. 2025Pretraining Language Models for Diachronic Linguistic Change Discoverycited by 6
  2. 2023Concept Drift Adaptation in Text Stream Mining Settings: A Systematic Reviewcited by 5
  3. 2024Turkronicles: Diachronic Resources for the Fast Evolving Turkish Languagecited by 4
  4. 2024Tracing the Development of the Virtual Particle Concept Using Semantic Change Detectioncited by 3
  5. 2024Quantifying Lexical Semantic Shift via Unbalanced Optimal Transportcited by 3
  6. 2025Analyzing Continuous Semantic Shifts with Diachronic Word Similarity Matricescited by 3
  7. 2024Incremental Affinity Propagation based on Cluster Consolidation and Stratificationcited by 2
  8. 2024How BERT Speaks Shakespearean English? Evaluating Historical Bias in Contextual Language Modelscited by 2
  9. 2025Large Language Models for History, Philosophy, and Sociology of Science: Interpretive Uses, Methodological Challenges, and Critical Perspectivescited by 2
  10. 2025TiMoE: Time-Aware Mixture of Language Expertscited by 2
  11. 2025Countermind: A Multi-Layered Security Architecture for Large Language Modelscited by 2
  12. 2025Responsible LLM Deployment for High-Stake Decisions by Decentralized Technologies and Human-AI Interactionscited by 2
10 more
  1. 2024Tracking Semantic Change in Slovene: A Novel Dataset and Optimal Transport-Based Distancecited by 1
  2. 2025Identifying Emerging Concepts in Large Corporacited by 1
  3. 2025Beyond Text: Characterizing Domain Expert Needs in Document Researchcited by 1
  4. 2025From Memorization to Generalization: Fine-Tuning Large Language Models for Biomedical Term-to-Identifier Normalizationcited by 1
  5. 2024Values That Are Explicitly Present in Fairy Tales: Comparing Samples from German, Italian and Portuguese Traditionscited by 0
  6. 2025Towards Universal Semantics With Large Language Modelscited by 0
  7. 2025Word Sense Detection Leveraging Maximum Mean Discrepancycited by 0
  8. 2026Computational conceptual history of scientific concepts: From early digital methods to LLMscited by 0
  9. 2026Beyond frequency measures: Can contextual embeddings capture meaning change in scientific texts?cited by 0
  10. 2026Large Language Models, Machine-Assisted Cultural Analytics, and the Agentic Turncited by 0