# Measuring diachronic sense change: new models and Monte Carlo methods for Bayesian inference

arXiv: https://arxiv.org/abs/2105.00819 · HTML: https://mlsanity.com/paper/2105.00819/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 49 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection](https://mlsanity.com/paper/2007.11464/) (2020)
- [Diachronic word embeddings and semantic shifts: a survey](https://mlsanity.com/paper/1806.03537/) (2018)
- [Statistically Significant Detection of Linguistic Change](https://mlsanity.com/paper/1411.3315/) (2014)
- [Survey of Computational Approaches to Lexical Semantic Change](https://mlsanity.com/paper/1811.06278/) (2018)
- [Dynamic Word Embeddings](https://mlsanity.com/paper/1702.08359/) (2017)
- [From Word to Sense Embeddings: A Survey on Vector Representations of Meaning](https://mlsanity.com/paper/1805.04032/) (2018)
- [The Dynamic Embedded Topic Model](https://mlsanity.com/paper/1907.05545/) (2019)
- [Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change](https://mlsanity.com/paper/1906.01688/) (2019)
- [A State-of-the-Art of Semantic Change Computation](https://mlsanity.com/paper/1801.09872/) (2018)
- [Exponential Family Embeddings](https://mlsanity.com/paper/1608.00778/) (2016)
- [That's sick dude!: Automatic identification of word sense change across different timescales](https://mlsanity.com/paper/1405.4392/) (2014)
- [Breaking Sticks and Ambiguities with Adaptive Skip-gram](https://mlsanity.com/paper/1502.07257/) (2015)
- [GASC: Genre-Aware Semantic Change for Ancient Greek](https://mlsanity.com/paper/1903.05587/) (2019)
- [Distributed representation of multi-sense words: A loss-driven approach](https://mlsanity.com/paper/1904.06725/) (2019)

## Built on by (most cited first, 1 of 1)

- [An Embedded Diachronic Sense Change Model with a Case Study from Ancient Greek](https://mlsanity.com/paper/2311.00541/) (2023)

## Abstract (arXiv)

In a bag-of-words model, the senses of a word with multiple meanings, e.g. "bank" (used either in a river-bank or an institution sense), are represented as probability distributions over context words, and sense prevalence is represented as a probability distribution over senses. Both of these may change with time. Modelling and measuring this kind of sense change is challenging due to the typically high-dimensional parameter space and sparse datasets. A recently published corpus of ancient Greek texts contains expert-annotated sense labels for selected target words. Automatic sense-annotation for the word "kosmos" (meaning decoration, order or world) has been used as a test case in recent work with related generative models and Monte Carlo methods. We adapt an existing generative sense change model to develop a simpler model for the main effects of sense and time, and give MCMC methods for Bayesian inference on all these models that are more efficient than existing methods. We carry out automatic sense-annotation of snippets containing "kosmos" using our model, and measure the time-evolution of its three senses and their prevalence. As far as we are aware, ours is the first analysis of this data, within the class of generative models we consider, that quantifies uncertainty and returns credible sets for evolving sense prevalence in good agreement with those given by expert annotation.