# Frequency-based Distortions in Contextualized Word Embeddings

arXiv: https://arxiv.org/abs/2104.08465 · HTML: https://mlsanity.com/paper/2104.08465/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 17 Apr 2021.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [Experience Grounds Language](https://mlsanity.com/paper/2004.10151/) (2020)
- [Visualizing and Measuring the Geometry of BERT](https://mlsanity.com/paper/1906.02715/) (2019)
- [Understanding Undesirable Word Embedding Associations](https://mlsanity.com/paper/1908.06361/) (2019)
- [Towards Understanding Linear Word Analogies](https://mlsanity.com/paper/1810.04882/) (2018)
- [Word meaning in minds and machines](https://mlsanity.com/paper/2008.01766/) (2020)
- [Factors Influencing the Surprising Instability of Word Embeddings](https://mlsanity.com/paper/1804.09692/) (2018)

## Built on by (most cited first, 20 of 20)

- [Anisotropy Is Inherent to Self-Attention in Transformers](https://mlsanity.com/paper/2401.12143/) (2024)
- [LMPriors: Pre-Trained Language Models as Task-Specific Priors](https://mlsanity.com/paper/2210.12530/) (2022)
- [Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models](https://mlsanity.com/paper/2110.00672/) (2021)
- [A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedians](https://mlsanity.com/paper/2405.20956/) (2024)
- [A Survey on Contextualised Semantic Shift Detection](https://mlsanity.com/paper/2304.01666/) (2023)
- [Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck](https://mlsanity.com/paper/2404.07647/) (2024)
- [Improving Contrastive Learning of Sentence Embeddings with Case-Augmented Positives and Retrieved Negatives](https://mlsanity.com/paper/2206.02457/) (2022)
- [Slangvolution: A Causal Analysis of Semantic Change and Frequency Dynamics in Slang](https://mlsanity.com/paper/2203.04651/) (2022)
- [On the Scaling Laws of Geographical Representation in Language Models](https://mlsanity.com/paper/2402.19406/) (2024)
- [Discovering Differences in the Representation of People using Contextualized Semantic Axes](https://mlsanity.com/paper/2210.12170/) (2022)
- [Residual2Vec: Debiasing graph embedding with random graphs](https://mlsanity.com/paper/2110.07654/) (2021)
- [Solving Cosine Similarity Underestimation between High Frequency Words by L2 Norm Discounting](https://mlsanity.com/paper/2305.10610/) (2023)
- [Is Anisotropy Inherent to Transformers?](https://mlsanity.com/paper/2306.07656/) (2023)
- [Headless Language Models: Learning without Predicting with Contrastive Weight Tying](https://mlsanity.com/paper/2309.08351/) (2023)
- [Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamics](https://mlsanity.com/paper/2604.08764/) (2026)
- [Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery](https://mlsanity.com/paper/2606.06267/) (2026)
- [Norm of Mean Contextualized Embeddings Determines their Variance](https://mlsanity.com/paper/2409.11253/) (2024)
- [Comparing scalable strategies for generating numerical perspectives](https://mlsanity.com/paper/2308.01535/) (2023)
- [How much do contextualized representations encode long-range context?](https://mlsanity.com/paper/2410.12292/) (2024)
- [On the Tip of the Tongue: Why LLMs Hallucinate Answers They Can Decode](https://mlsanity.com/paper/2603.13911/) (2026)

## Abstract (arXiv)

How does word frequency in pre-training data affect the behavior of similarity metrics in contextualized BERT embeddings? Are there systematic ways in which some word relationships are exaggerated or understated? In this work, we explore the geometric characteristics of contextualized word embeddings with two novel tools: (1) an identity probe that predicts the identity of a word using its embedding; (2) the minimal bounding sphere for a word's contextualized representations. Our results reveal that words of high and low frequency differ significantly with respect to their representational geometry. Such differences introduce distortions: when compared to human judgments, point estimates of embedding similarity (e.g., cosine similarity) can over- or under-estimate the semantic similarity of two words, depending on the frequency of those words in the training data. This has downstream societal implications: BERT-Base has more trouble differentiating between South American and African countries than North American and European ones. We find that these distortions persist when using BERT-Multilingual, suggesting that they cannot be easily fixed with additional data, which in turn introduces new distortions.