# Regionalized models for Spanish language variations based on Twitter

arXiv: https://arxiv.org/abs/2110.06128 · HTML: https://mlsanity.com/paper/2110.06128/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 50 references that are in the map.
- First posted on arXiv 12 Oct 2021.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [Dialectometric analysis of language variation in Twitter](https://mlsanity.com/paper/1702.06777/) (2017)
- [Where in the World are You? Geolocation and Language Identification in Twitter](https://mlsanity.com/paper/1308.0683/) (2013)
- [Crowdsourcing Dialect Characterization through Twitter](https://mlsanity.com/paper/1407.7094/) (2014)
- [Using Word Embeddings in Twitter Election Classification](https://mlsanity.com/paper/1606.07006/) (2016)
- [Unsupervised Cross-lingual Word Embedding by Multilingual Neural Language Models](https://mlsanity.com/paper/1809.02306/) (2018)

## Built on by (most cited first, 4 of 4)

- [Entropy and type-token ratio in gigaword corpora](https://mlsanity.com/paper/2411.10227/) (2024)
- [Sequence-to-Sequence Spanish Pre-trained Language Models](https://mlsanity.com/paper/2309.11259/) (2023)
- [Designing Beyond Language: Sociotechnical Barriers in AI Health Technologies for Limited English Proficiency](https://mlsanity.com/paper/2511.07277/) (2025)
- [Dialect and Gender Bias in YouTube's Spanish Captioning System](https://mlsanity.com/paper/2602.24002/) (2026)

## Abstract (arXiv)

Spanish is one of the most spoken languages in the globe, but not necessarily Spanish is written and spoken in the same way in different countries. Understanding local language variations can help to improve model performances on regional tasks, both understanding local structures and also improving the message's content. For instance, think about a machine learning engineer who automatizes some language classification task on a particular region or a social scientist trying to understand a regional event with echoes on social media; both can take advantage of dialect-based language models to understand what is happening with more contextual information hence more precision. This manuscript presents and describes a set of regionalized resources for the Spanish language built on four-year Twitter public messages geotagged in 26 Spanish-speaking countries. We introduce word embeddings based on FastText, language models based on BERT, and per-region sample corpora. We also provide a broad comparison among regions covering lexical and semantical similarities; as well as examples of using regional resources on message classification tasks.