# Variation and Instability in Dialect-Based Embedding Spaces

arXiv: https://arxiv.org/abs/2303.14963 · HTML: https://mlsanity.com/paper/2303.14963/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Builds on (most cited first, 15 of 15)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Mapping Languages: The Corpus of Global Language Use](https://mlsanity.com/paper/2004.00798/) (2020)
- [Factors Influencing the Surprising Instability of Word Embeddings](https://mlsanity.com/paper/1804.09692/) (2018)
- [Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora](https://mlsanity.com/paper/2112.14330/) (2021)
- [Dialectometric analysis of language variation in Twitter](https://mlsanity.com/paper/1702.06777/) (2017)
- [Modeling Global Syntactic Variation in English Using Dialect Classification](https://mlsanity.com/paper/1904.05527/) (2019)
- [A Neural Model for User Geolocation and Lexical Dialectology](https://mlsanity.com/paper/1704.04008/) (2017)
- [Finding Variants for Construction-Based Dialectometry: A Corpus-Based Approach to Regional CxGs](https://mlsanity.com/paper/2104.01299/) (2021)
- [Global Syntactic Variation in Seven Languages: Towards a Computational Dialectology](https://mlsanity.com/paper/2104.01306/) (2021)
- [Representations of Language Varieties Are Reliable Given Corpus Similarity Measures](https://mlsanity.com/paper/2104.01294/) (2021)
- [Analyzing the Surprising Variability in Word Embedding Stability Across Languages](https://mlsanity.com/paper/2004.14876/) (2020)
- [Stability of Syntactic Dialect Classification Over Space and Time](https://mlsanity.com/paper/2209.04958/) (2022)
- [The Influence of Down-Sampling Strategies on SVD Word Embedding Stability](https://mlsanity.com/paper/1808.06810/) (2018)
- [Predicting Embedding Reliability in Low-Resource Settings Using Corpus Similarity Measures](https://mlsanity.com/paper/2206.04330/) (2022)

## Built on by (most cited first, 3 of 3)

- [Time Machine GPT](https://mlsanity.com/paper/2404.18543/) (2024)
- [Syntactic Variation Across the Grammar: Modelling a Complex Adaptive System](https://mlsanity.com/paper/2309.11869/) (2023)
- [Linear Semantic Segmentation for Low-Resource Spoken Dialects](https://mlsanity.com/paper/2605.06276/) (2026)

## Abstract (arXiv)

This paper measures variation in embedding spaces which have been trained on different regional varieties of English while controlling for instability in the embeddings. While previous work has shown that it is possible to distinguish between similar varieties of a language, this paper experiments with two follow-up questions: First, does the variety represented in the training data systematically influence the resulting embedding space after training? This paper shows that differences in embeddings across varieties are significantly higher than baseline instability. Second, is such dialect-based variation spread equally throughout the lexicon? This paper shows that specific parts of the lexicon are particularly subject to variation. Taken together, these experiments confirm that embedding spaces are significantly influenced by the dialect represented in the training data. This finding implies that there is semantic variation across dialects, in addition to previously-studied lexical and syntactic variation.