# Are Word Embedding Methods Stable and Should We Care About It?

arXiv: https://arxiv.org/abs/2104.08433 · HTML: https://mlsanity.com/paper/2104.08433/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 17 Apr 2021.

## Builds on (most cited first, 11 of 11)

- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Factors Influencing the Surprising Instability of Word Embeddings](https://mlsanity.com/paper/1804.09692/) (2018)
- [Frustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddings](https://mlsanity.com/paper/1804.05262/) (2018)
- [Portuguese Word Embeddings: Evaluating on Word Analogies and Natural Language Tasks](https://mlsanity.com/paper/1708.06025/) (2017)
- [A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding](https://mlsanity.com/paper/1511.00215/) (2015)
- [It's Only Words And Words Are All I Have](https://mlsanity.com/paper/1901.05227/) (2019)
- [Second-Order Word Embeddings from Nearest Neighbor Topological Features](https://mlsanity.com/paper/1705.08488/) (2017)
- [Understanding the Stability of Medical Concept Embeddings](https://mlsanity.com/paper/1904.09552/) (2019)

## Built on by (most cited first, 4 of 4)

- [Statistical Uncertainty in Word Embeddings: GloVe-V](https://mlsanity.com/paper/2406.12165/) (2024)
- [Geometric instability of graph neural networks on large graphs](https://mlsanity.com/paper/2308.10099/) (2023)
- [GEANN: Scalable Graph Augmentations for Multi-Horizon Time Series Forecasting](https://mlsanity.com/paper/2307.03595/) (2023)
- [The Impact of Dimensionality on the Stability of Node Embeddings](https://mlsanity.com/paper/2604.08492/) (2026)

## Abstract (arXiv)

A representation learning method is considered stable if it consistently generates similar representation of the given data across multiple runs. Word Embedding Methods (WEMs) are a class of representation learning methods that generate dense vector representation for each word in the given text data. The central idea of this paper is to explore the stability measurement of WEMs using intrinsic evaluation based on word similarity. We experiment with three popular WEMs: Word2Vec, GloVe, and fastText. For stability measurement, we investigate the effect of five parameters involved in training these models. We perform experiments using four real-world datasets from different domains: Wikipedia, News, Song lyrics, and European parliament proceedings. We also observe the effect of WEM stability on three downstream tasks: Clustering, POS tagging, and Fairness evaluation. Our experiments indicate that amongst the three WEMs, fastText is the most stable, followed by GloVe and Word2Vec.