# Dynamic Bernoulli Embeddings for Language Evolution

arXiv: https://arxiv.org/abs/1703.08052 · HTML: https://mlsanity.com/paper/1703.08052/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2017.

## Built on by (most cited first, 6 of 6)

- [The Geometry of Categorical and Hierarchical Concepts in Large Language Models](https://mlsanity.com/paper/2406.01506/) (2024)
- [On the Origins of Linear Representations in Large Language Models](https://mlsanity.com/paper/2403.03867/) (2024)
- [Time Machine GPT](https://mlsanity.com/paper/2404.18543/) (2024)
- [On the Universality of Deep Contextual Language Models](https://mlsanity.com/paper/2109.07140/) (2021)
- [Posterior Sampling of Probabilistic Word Embeddings](https://mlsanity.com/paper/2508.02337/) (2025)
- [Dynamics of Meaning: Towards the Evaluation of Diachronic Semantic Change in Sinhala](https://mlsanity.com/paper/2609.08609/) (2026)

## Abstract (arXiv)

Word embeddings are a powerful approach for unsupervised analysis of language. Recently, Rudolph et al. (2016) developed exponential family embeddings, which cast word embeddings in a probabilistic framework. Here, we develop dynamic embeddings, building on exponential family embeddings to capture how the meanings of words change over time. We use dynamic embeddings to analyze three large collections of historical texts: the U.S. Senate speeches from 1858 to 2009, the history of computer science ACM abstracts from 1951 to 2014, and machine learning papers on the Arxiv from 2007 to 2015. We find dynamic embeddings provide better fits than classical embeddings and capture interesting patterns about how language changes.