# Statistical Uncertainty in Word Embeddings: GloVe-V

arXiv: https://arxiv.org/abs/2406.12165 · HTML: https://mlsanity.com/paper/2406.12165/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 66 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Builds on (most cited first, 19 of 19)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [What Will it Take to Fix Benchmarking in Natural Language Understanding?](https://mlsanity.com/paper/2104.02145/) (2021)
- [With Little Power Comes Great Responsibility](https://mlsanity.com/paper/2010.06595/) (2020)
- [Understanding the Origins of Bias in Word Embeddings](https://mlsanity.com/paper/1810.03611/) (2018)
- [Understanding Undesirable Word Embedding Associations](https://mlsanity.com/paper/1908.06361/) (2019)
- [Towards Understanding Linear Word Analogies](https://mlsanity.com/paper/1810.04882/) (2018)
- [Factors Influencing the Surprising Instability of Word Embeddings](https://mlsanity.com/paper/1804.09692/) (2018)
- [What are the biases in my word embedding?](https://mlsanity.com/paper/1812.08769/) (2018)
- [Issues in evaluating semantic spaces using word analogies](https://mlsanity.com/paper/1606.07736/) (2016)
- [Evolutionary forces in language change](https://mlsanity.com/paper/1608.00938/) (2016)
- [Experimental Standards for Deep Learning in Natural Language Processing Research](https://mlsanity.com/paper/2204.06251/) (2022)
- [Human Languages with Greater Information Density Increase Communication Speed, but Decrease Conversation Breadth](https://mlsanity.com/paper/2112.08491/) (2021)
- [Are Word Embedding Methods Stable and Should We Care About It?](https://mlsanity.com/paper/2104.08433/) (2021)
- [Gender and Racial Stereotype Detection in Legal Opinion Word Embeddings](https://mlsanity.com/paper/2203.13369/) (2022)
- [Context Vectors are Reflections of Word Vectors in Half the Dimensions](https://mlsanity.com/paper/1902.09859/) (2019)

## Built on by (most cited first, 5 of 5)

- [A New Pair of GloVes](https://mlsanity.com/paper/2507.18103/) (2025)
- [Posterior Sampling of Probabilistic Word Embeddings](https://mlsanity.com/paper/2508.02337/) (2025)
- [Research on a hybrid LSTM-CNN-Attention model for text-based web content classification](https://mlsanity.com/paper/2512.18475/) (2025)
- [Accurate and Efficient Statistical Testing for Word Semantic Breadth](https://mlsanity.com/paper/2605.08048/) (2026)
- [Forecasting Corpus-Sampling Disagreement in Masked Text Reconstruction](https://mlsanity.com/paper/2610.12987/) (2026)

## Abstract (arXiv)

Static word embeddings are ubiquitous in computational social science applications and contribute to practical decision-making in a variety of fields including law and healthcare. However, assessing the statistical uncertainty in downstream conclusions drawn from word embedding statistics has remained challenging. When using only point estimates for embeddings, researchers have no streamlined way of assessing the degree to which their model selection criteria or scientific conclusions are subject to noise due to sparsity in the underlying data used to generate the embeddings. We introduce a method to obtain approximate, easy-to-use, and scalable reconstruction error variance estimates for GloVe (Pennington et al., 2014), one of the most widely used word embedding models, using an analytical approximation to a multivariate normal model. To demonstrate the value of embeddings with variance (GloVe-V), we illustrate how our approach enables principled hypothesis testing in core word embedding tasks, such as comparing the similarity between different word pairs in vector space, assessing the performance of different models, and analyzing the relative degree of ethnic or gender bias in a corpus using different word lists.