# Mapping Languages and Demographics with Georeferenced Corpora

arXiv: https://arxiv.org/abs/2004.00809 · HTML: https://mlsanity.com/paper/2004.00809/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2020.

## Built on by (most cited first, 6 of 6)

- [Global Syntactic Variation in Seven Languages: Towards a Computational Dialectology](https://mlsanity.com/paper/2104.01306/) (2021)
- [The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?](https://mlsanity.com/paper/2601.07220/) (2026)
- [Representations of Language Varieties Are Reliable Given Corpus Similarity Measures](https://mlsanity.com/paper/2104.01294/) (2021)
- [Towards Region-aware Bias Evaluation Metrics](https://mlsanity.com/paper/2406.16152/) (2024)
- [Comparing Measures of Linguistic Diversity Across Social Media Language Data and Census Data at Subnational Geographic Areas](https://mlsanity.com/paper/2308.10452/) (2023)
- [Measuring Linguistic Diversity During COVID-19](https://mlsanity.com/paper/2104.01290/) (2021)

## Abstract (arXiv)

This paper evaluates large georeferenced corpora, taken from both web-crawled and social media sources, against ground-truth population and language-census datasets. The goal is to determine (i) which dataset best represents population demographics; (ii) in what parts of the world the datasets are most representative of actual populations; and (iii) how to weight the datasets to provide more accurate representations of underlying populations. The paper finds that the two datasets represent very different populations and that they correlate with actual populations with values of r=0.60 (social media) and r=0.49 (web-crawled). Further, Twitter data makes better predictions about the inventory of languages used in each country.