mlsanityData to 7 Oct 2026

Analysing Lexical Semantic Change with Contextualised Word Representations

arXiv 2004.14118 ↗First posted 29 Apr 2020cs.CL cs.CYcited by 92

Not on the map. Why?

Abstract (arXiv)

This paper presents the first unsupervised approach to lexical semantic change that makes use of contextualised word representations. We propose a novel method that exploits the BERT neural language model to obtain representations of word usages, clusters these representations into usage types, and measures change along time with three proposed metrics. We create a new evaluation dataset and show that the model representations and the detected semantic shifts are positively correlated with human judgements. Our extensive qualitative analysis demonstrates that our method captures a variety of synchronic and diachronic linguistic phenomena. We expect our work to inspire further research in this direction.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 14; 2023: 20; 2024: 21; 2025: 6; 2026: 15.

92 papers.

Sort:

By area:

  1. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  2. 2021Time Masking for Temporal Language Modelscited by 46
  3. 2021Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Sensescited by 39
  4. 2021Explaining and Improving BERT Performance on Lexical Semantic Change Detectioncited by 33
  5. 2024LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Modelscited by 33
  6. 2024Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Researchcited by 26
  7. 2021DWUG: A large Resource of Diachronic Word Usage Graphs in Four Languagescited by 25
  8. 2022Temporal Attention for Language Modelscited by 21
  9. 2023A Survey on Contextualised Semantic Shift Detectioncited by 21
  10. 2024A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Changecited by 21
  11. 2021Characterizing English Variation across Social Media Communities with BERTcited by 20
  12. 2022Contextualized language models for semantic change detection: lessons learnedcited by 20
38 more
  1. 2021Causal Distillation for Language Modelscited by 19
  2. 2023Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysiscited by 19
  3. 2022Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Changecited by 18
  4. 2022BiTimeBERT: Extending Pre-Trained Language Representations with Bi-Temporal Informationcited by 17
  5. 2021Challenges for Computational Lexical Semantic Changecited by 15
  6. 2021Statistically significant detection of semantic shifts using contextual word embeddingscited by 14
  7. 2022Slangvolution: A Causal Analysis of Semantic Change and Frequency Dynamics in Slangcited by 14
  8. 2022On the Impact of Temporal Concept Drift on Model Explanationscited by 14
  9. 2025TempRetriever: Fusion-based Temporal Dense Passage Retrieval for Time-Sensitive Questionscited by 14
  10. 2021Lexical Semantic Change Discoverycited by 13
  11. 2023A Method for Studying Semantic Construal in Grammatical Constructions with Interpretable Contextual Embedding Spacescited by 13
  12. 2022Discovering Differences in the Representation of People using Contextualized Semantic Axescited by 12
  13. 2022HistBERT: A Pre-trained Language Model for Diachronic Lexical Semantic Analysiscited by 11
  14. 2021Grammatical Profiling for Semantic Change Detectioncited by 10
  15. 2023Can Word Sense Distribution Detect Semantic Changes of Words?cited by 10
  16. 2023Once Upon a textitTime in textitGraph: Relative-Time Pretraining for Complex Temporal Reasoningcited by 10
  17. 2024Graph-based Clustering for Detecting Semantic Change Across Time and Languagescited by 10
  18. 2025A Study into Investigating Temporal Robustness of LLMscited by 10
  19. 2023Substitution-based Semantic Change Detection using Contextual Embeddingscited by 9
  20. 2023Unsupervised Semantic Variation Prediction using the Distribution of Sibling Embeddingscited by 8
  21. 2023Cross-lingual Cross-temporal Summarization: Dataset, Models, Evaluationcited by 8
  22. 2022NorDiaChange: Diachronic Semantic Change Dataset for Norwegiancited by 7
  23. 2022Do Not Fire the Linguist: Grammatical Profiles Help Language Models Detect Semantic Changecited by 7
  24. 2023The DURel Annotation Tool: Human and Computational Measurement of Semantic Proximity, Sense Clusters and Semantic Changecited by 7
  25. 2023textitSwap and Predict -- Predicting the Semantic Changes in Words across Corpora by Context Swappingcited by 6
  26. 2024The Impact of Word Splitting on the Semantic Content of Contextualized Word Representationscited by 6
  27. 2024Resolving Regular Polysemy in Named Entitiescited by 5
  28. 2024Survey in Characterizing Semantic Changecited by 5
  29. 2024Towards Effective Time-Aware Language Representation: Exploring Enhanced Temporal Understanding in Language Modelscited by 5
  30. 2021A diachronic evaluation of gender asymmetry in euphemismcited by 4
  31. 2021On the Universality of Deep Contextual Language Modelscited by 4
  32. 2022A Greek Parliament Proceedings Dataset for Computational Linguistics and Political Analysiscited by 4
  33. 2023Experimenting with UD Adaptation of an Unsupervised Rule-based Approach for Sentiment Analysis of Mexican Tourist Textscited by 4
  34. 2021Predicting emergent linguistic compositions through time: Syntactic frame extension via multimodal chainingcited by 3
  35. 2022Predicting Long-Term Citations from Short-Term Linguistic Influencecited by 3
  36. 2023Word sense extensioncited by 3
  37. 2024Syntactic Language Change in English and German: Metrics, Parsers, and Convergencescited by 3
  38. 2024Presence or Absence: Are Unknown Word Usages in Dictionaries?cited by 3

50 most cited of 92 All 92 (JSON)