# Dynamic Word Embeddings for Evolving Semantic Discovery

arXiv: https://arxiv.org/abs/1703.00607 · HTML: https://mlsanity.com/paper/1703.00607/

## Key facts

- Cited by 38 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 2 Mar 2017.

## Built on by (most cited first, 30 of 38)

- [Time Masking for Temporal Language Models](https://mlsanity.com/paper/2110.06366/) (2021)
- [Dynamic Language Models for Continuously Evolving Content](https://mlsanity.com/paper/2106.06297/) (2021)
- [Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora](https://mlsanity.com/paper/2112.14330/) (2021)
- [Temporal Attention for Language Models](https://mlsanity.com/paper/2202.02093/) (2022)
- [Learning Backward Compatible Embeddings](https://mlsanity.com/paper/2206.03040/) (2022)
- [BERTrend: Neural Topic Modeling for Emerging Trends Detection](https://mlsanity.com/paper/2411.05930/) (2024)
- [Can Word Sense Distribution Detect Semantic Changes of Words?](https://mlsanity.com/paper/2310.10400/) (2023)
- [A Study into Investigating Temporal Robustness of LLMs](https://mlsanity.com/paper/2503.17073/) (2025)
- [Unsupervised Semantic Variation Prediction using the Distribution of Sibling Embeddings](https://mlsanity.com/paper/2305.08654/) (2023)
- [Abolitionist Networks: Modeling Language Change in Nineteenth-Century Activist Newspapers](https://mlsanity.com/paper/2103.07538/) (2021)
- [textitSwap and Predict -- Predicting the Semantic Changes in Words across Corpora by Context Swapping](https://mlsanity.com/paper/2310.10397/) (2023)
- [Theoretical foundations and limits of word embeddings: what types of meaning can they capture?](https://mlsanity.com/paper/2107.10413/) (2021)
- [A Semantic Distance Metric Learning approach for Lexical Semantic Change Detection](https://mlsanity.com/paper/2403.00226/) (2024)
- [VideoMap: Supporting Video Editing Exploration, Brainstorming, and Prototyping in the Latent Space](https://mlsanity.com/paper/2211.12492/) (2022)
- [Investigating the Contextualised Word Embedding Dimensions Specified for Contextual and Temporal Semantic Changes](https://mlsanity.com/paper/2407.02820/) (2024)
- [NLP for Climate Policy: Creating a Knowledge Platform for Holistic and Effective Climate Action](https://mlsanity.com/paper/2105.05621/) (2021)
- [Letters From the Past: Modeling Historical Sound Change Through Diachronic Character Embeddings](https://mlsanity.com/paper/2205.08256/) (2022)
- [Word Embedding for Social Sciences: An Interdisciplinary Survey](https://mlsanity.com/paper/2207.03086/) (2022)
- [Analyzing Continuous Semantic Shifts with Diachronic Word Similarity Matrices](https://mlsanity.com/paper/2501.09538/) (2025)
- [Computational modeling of semantic change](https://mlsanity.com/paper/2304.06337/) (2023)
- [Sig-Networks Toolkit: Signature Networks for Longitudinal Language Modelling](https://mlsanity.com/paper/2312.03523/) (2023)
- [Jointly modelling the evolution of social structure and language in online communities](https://mlsanity.com/paper/2409.19243/) (2024)
- [Quantifying Lexical Semantic Shift via Unbalanced Optimal Transport](https://mlsanity.com/paper/2412.12569/) (2024)
- [Fake it Till You Make it: Self-Supervised Semantic Shifts for Monolingual Word Embedding Tasks](https://mlsanity.com/paper/2102.00290/) (2021)
- [When FastText Pays Attention: Efficient Estimation of Word Representations using Constrained Positional Weighting](https://mlsanity.com/paper/2104.09691/) (2021)
- [Visualizing Temporal Topic Embeddings with a Compass](https://mlsanity.com/paper/2409.10649/) (2024)
- [Understanding mobility in networks: A node embedding approach](https://mlsanity.com/paper/2111.06161/) (2021)
- [Subdiffusive semantic evolution in Indo-European languages](https://mlsanity.com/paper/2209.04701/) (2022)
- [Trust in Motion: Capturing Trust Ascendancy in Open-Source Projects using Hybrid AI](https://mlsanity.com/paper/2210.02656/) (2022)
- [Domain-Specific Word Embeddings with Structure Prediction](https://mlsanity.com/paper/2210.04962/) (2022)

## Abstract (arXiv)

Word evolution refers to the changing meanings and associations of words throughout time, as a byproduct of human language evolution. By studying word evolution, we can infer social trends and language constructs over different periods of human history. However, traditional techniques such as word representation learning do not adequately capture the evolving language structure and vocabulary. In this paper, we develop a dynamic statistical model to learn time-aware word vector representation. We propose a model that simultaneously learns time-aware embeddings and solves the resulting "alignment problem". This model is trained on a crawled NYTimes dataset. Additionally, we develop multiple intuitive evaluation strategies of temporal word embeddings. Our qualitative and quantitative tests indicate that our method not only reliably captures this evolution over time, but also consistently outperforms state-of-the-art temporal embedding approaches on both semantic accuracy and alignment quality.