mlsanityData to 7 Oct 2026

Analyzing Semantic Change through Lexical Replacements

arXiv 2404.18570 ↗First posted 29 Apr 2024cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Modern language models are capable of contextualizing words based on their surrounding context. However, this capability is often compromised due to semantic change that leads to words being used in new, unexpected contexts not encountered during pre-training. In this paper, we model textitsemantic change by studying the effect of unexpected contexts introduced by textitlexical replacements. We propose a textitreplacement schema where a target word is substituted with lexical replacements of varying relatedness, thus simulating different kinds of semantic change. Furthermore, we leverage the replacement schema as a basis for a novel textitinterpretable model for semantic change. We are also the first to evaluate the use of LLaMa for semantic change detection.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 22 of its 52 references that are in the map. First posted on arXiv 29 Apr 2024.

Builds on

22 of 52 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  3. 2019Is Attention Interpretable?cited by 387
  4. 2019Visualizing and Measuring the Geometry of BERTcited by 215
  5. 2020SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detectioncited by 105
  6. 2019SenseBERT: Driving Some Sense into BERTcited by 73
  7. 2018Survey of Computational Approaches to Lexical Semantic Changecited by 60
  8. 2021Time Masking for Temporal Language Modelscited by 46
  9. 2020Why Attentions May Not Be Interpretable?cited by 41
  10. 2021Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Sensescited by 39
  11. 2019Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shiftcited by 37
  12. 2021Explaining and Improving BERT Performance on Lexical Semantic Change Detectioncited by 33
10 more
  1. 2020UiO-UvA at SemEval-2020 Task 1: Contextualised Embeddings for Lexical Semantic Change Detectioncited by 32
  2. 2022Word Order Does Matter (And Shuffled Language Models Know It)cited by 32
  3. 2024A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Changecited by 21
  4. 2022Contextualized language models for semantic change detection: lessons learnedcited by 20
  5. 2023Learn over Past, Evolve for Future: Forecasting Temporal Trends for Fake News Detectioncited by 20
  6. 2022Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Changecited by 18
  7. 2020How Far Does BERT Look At:Distance-based Clustering and Analysis of BERT's Attentioncited by 9
  8. 2023Substitution-based Semantic Change Detection using Contextual Embeddingscited by 9
  9. 2024Survey in Characterizing Semantic Changecited by 5
  10. 2024How BERT Speaks Shakespearean English? Evaluating Historical Bias in Contextual Language Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 3.

7 papers.

  1. 2024Semantic Change Characterization with LLMs using Rhetoricscited by 4
  2. 2025Pretraining Language Models for Diachronic Linguistic Change Discoverycited by 4
  3. 2025SCDTour: Embedding Axis Ordering and Merging for Interpretable Semantic Change Detectioncited by 4
  4. 2025Analyzing Continuous Semantic Shifts with Diachronic Word Similarity Matricescited by 3
  5. 2026DHPLT: large-scale multilingual diachronic corpora and word representations for semantic change modellingcited by 1
  6. 2026"I'm happy even though it's not real": GenAI Photo Editing as a Remembering Experiencecited by 0
  7. 2026Relation Geometry in Semantic Space of Language Modelscited by 0