mlsanityData to 7 Oct 2026

Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change

arXiv 1605.09096 ↗First posted 30 May 2016cs.CLcited by 219

Not on the map. Why?

Abstract (arXiv)

Understanding how words change their meanings over time is key to models of language and cultural evolution, but historical data on meaning is scarce, making theories hard to develop and test. Word embeddings show promise as a diachronic tool, but have not been carefully evaluated. We develop a robust methodology for quantifying semantic change by evaluating word embeddings (PPMI, SVD, word2vec) against known historical changes. We then use this methodology to reveal statistical laws of semantic evolution. Using six historical corpora spanning four languages and two centuries, we propose two quantitative laws of semantic change: (i) the law of conformity---the rate of semantic change scales with an inverse power-law of word frequency; (ii) the law of innovation---independent of frequency, words that are more polysemous have higher rates of semantic change.

Key facts. Cited by 219 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 54; 2022: 31; 2023: 32; 2024: 37; 2025: 16; 2026: 49.

219 papers.

Sort:

By area:

  1. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  2. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  3. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  4. 2022TimeLMs: Diachronic Language Models from Twittercited by 129
  5. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 128
  6. 2024Universal Neurons in GPT2 Language Modelscited by 100
  7. 2021Time Waits for No One! Analysis and Challenges of Temporal Misalignmentcited by 91
  8. 2023AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languagescited by 68
  9. 2022Entity Cloze By Date: What LMs Know About Unseen Entitiescited by 55
  10. 2021Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Mediacited by 51
  11. 2021Artificial Text Detection via Examining the Topology of Attention Mapscited by 49
  12. 2021Named Entity Recognition and Classification on Historical Documents: A Surveycited by 47
38 more
  1. 2021Time Masking for Temporal Language Modelscited by 46
  2. 2021Temporal Effects on Pre-trained Models for Language Processing Taskscited by 46
  3. 2023Meta-Learning Online Adaptation of Language Modelscited by 43
  4. 2023SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)cited by 41
  5. 2024Tracing Representation Progression: Analyzing and Enhancing Layer-Wise Similaritycited by 36
  6. 2023Time is Encoded in the Weights of Finetuned Language Modelscited by 31
  7. 2021Dynamic Language Models for Continuously Evolving Contentcited by 27
  8. 2024Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Researchcited by 26
  9. 2021Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corporacited by 21
  10. 2022Temporal Attention for Language Modelscited by 21
  11. 2023A Survey on Contextualised Semantic Shift Detectioncited by 21
  12. 2022Contextualized language models for semantic change detection: lessons learnedcited by 20
  13. 2023Evaluating Biased Attitude Associations of Language Models in an Intersectional Contextcited by 20
  14. 2022TempoWiC: An Evaluation Benchmark for Detecting Meaning Shift in Social Mediacited by 19
  15. 2021A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communicationcited by 18
  16. 2022Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Changecited by 18
  17. 2024ReSi: A Comprehensive Benchmark for Representational Similarity Measurescited by 18
  18. 2021LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyondcited by 17
  19. 2022Identifying Moments of Change from Longitudinal User Textcited by 17
  20. 2022Emblaze: Illuminating Machine Learning Representations through Interactive Comparison of Embedding Spacescited by 16
  21. 2021Challenges for Computational Lexical Semantic Changecited by 15
  22. 2021Statistically significant detection of semantic shifts using contextual word embeddingscited by 14
  23. 2021Assessing the Reliability of Word Embedding Gender Bias Measurescited by 14
  24. 2022Slangvolution: A Causal Analysis of Semantic Change and Frequency Dynamics in Slangcited by 14
  25. 2022Geographic Adaptation of Pretrained Language Modelscited by 14
  26. 2021Lexical Semantic Change Discoverycited by 13
  27. 2021Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biasescited by 13
  28. 2023Machine-assisted quantitizing designs: augmenting humanities and social sciences with artificial intelligencecited by 13
  29. 2021Hyperbolic Temporal Knowledge Graph Embeddings with Relational and Time Curvaturescited by 12
  30. 2024Time Awareness in Large Language Models: Benchmarking Fact Recall Across Timecited by 12
  31. 2025It's High Time: A Survey of Temporal Question Answeringcited by 12
  32. 2021A Computational Framework for Slang Generationcited by 11
  33. 2021Three-part diachronic semantic change dataset for Russiancited by 11
  34. 2022HistBERT: A Pre-trained Language Model for Diachronic Lexical Semantic Analysiscited by 11
  35. 2023Can Word Sense Distribution Detect Semantic Changes of Words?cited by 10
  36. 2023Once Upon a textitTime in textitGraph: Relative-Time Pretraining for Complex Temporal Reasoningcited by 10
  37. 2024Graph-based Clustering for Detecting Semantic Change Across Time and Languagescited by 10
  38. 2021Modeling Ideological Salience and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsitycited by 9

50 most cited of 219 All 219 (JSON)