# Breaking Sticks and Ambiguities with Adaptive Skip-gram

arXiv: https://arxiv.org/abs/1502.07257 · HTML: https://mlsanity.com/paper/1502.07257/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2015.

## Built on by (most cited first, 11 of 11)

- [A survey of dynamic graph neural networks](https://mlsanity.com/paper/2404.18211/) (2024)
- [Multitasking Framework for Unsupervised Simple Definition Generation](https://mlsanity.com/paper/2203.12926/) (2022)
- [Driving Context into Text-to-Text Privatization](https://mlsanity.com/paper/2306.01457/) (2023)
- [PolyLM: Learning about Polysemy through Language Modeling](https://mlsanity.com/paper/2101.10448/) (2021)
- [Measuring diachronic sense change: new models and Monte Carlo methods for Bayesian inference](https://mlsanity.com/paper/2105.00819/) (2021)
- [Unsupervised Contextualized Document Representation](https://mlsanity.com/paper/2109.10509/) (2021)
- [A Comparative Study of Transformers on Word Sense Disambiguation](https://mlsanity.com/paper/2111.15417/) (2021)
- [Exploiting Correlations Between Contexts and Definitions with Multiple Definition Modeling](https://mlsanity.com/paper/2305.14717/) (2023)
- [LitMind Dictionary: An Open-Source Online Dictionary](https://mlsanity.com/paper/2204.11087/) (2022)
- [Word Sense Induction with Knowledge Distillation from BERT](https://mlsanity.com/paper/2304.10642/) (2023)
- [Sequence graphs realizations and ambiguity in language models](https://mlsanity.com/paper/2402.08830/) (2024)

## Abstract (arXiv)

Recently proposed Skip-gram model is a powerful method for learning high-dimensional word representations that capture rich semantic relationships between words. However, Skip-gram as well as most prior work on learning word representations does not take into account word ambiguity and maintain only single representation per word. Although a number of Skip-gram modifications were proposed to overcome this limitation and learn multi-prototype word representations, they either require a known number of word meanings or learn them using greedy heuristic approaches. In this paper we propose the Adaptive Skip-gram model which is a nonparametric Bayesian extension of Skip-gram capable to automatically learn the required number of representations for all words at desired semantic resolution. We derive efficient online variational learning algorithm for the model and empirically demonstrate its efficiency on word-sense induction task.