mlsanityData to 14 Oct 2026

Pre-Trained Language Models Represent Some Geographic Populations Better Than Others

arXiv 2403.11025 ↗First posted 16 Mar 2024cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

This paper measures the skew in how well two families of LLMs represent diverse geographic populations. A spatial probing task is used with geo-referenced corpora to measure the degree to which pre-trained language models from the OPT and BLOOM series represent diverse populations around the world. Results show that these models perform much better for some populations than others. In particular, populations across the US and the UK are represented quite well while those in South and Southeast Asia are poorly represented. Analysis shows that both families of models largely share the same skew across populations. At the same time, this skew cannot be fully explained by sociolinguistic factors, economic factors, or geographic factors. The basic conclusion from this analysis is that pre-trained models do not equally represent the world's population: there is a strong skew towards specific geographic populations. This finding challenges the idea that a single model can be used for all populations.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 18 of its 29 references that are in the map. First posted on arXiv 16 Mar 2024.

Builds on

18 of 29 references.

References · 18

  1. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  3. 2020A Primer in BERTology: What we know about how BERT workscited by 815
  4. 2019How multilingual is Multilingual BERT?cited by 645
  5. 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
  6. 2020Mapping Languages: The Corpus of Global Language Usecited by 42
  7. 2021When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfercited by 42
  8. 2021Detecting Cross-Geographic Biases in Toxicity Modeling on Social Mediacited by 31
  9. 2023Counterfactual reasoning: Testing language models' understanding of hypothetical scenarioscited by 27
  10. 2022Geographic Citation Gaps in NLP Researchcited by 19
  11. 2022Geographic Adaptation of Pretrained Language Modelscited by 14
  12. 2021Representations of Language Varieties Are Reliable Given Corpus Similarity Measurescited by 10
6 more
  1. 2015Confounds and Consequences in Geotagged Twitter Datacited by 8
  2. 2022Geographical Distance Is The New Hyperparameter: A Case Study Of Finding The Optimal Pre-trained Language For English-isiZulu Machine Translationcited by 5
  3. 2022Measuring Geographic Performance Disparities of Offensive Language Classifierscited by 5
  4. 2022Corpus Similarity Measures Remain Robust Across Diverse Languagescited by 4
  5. 2022Register Variation Remains Stable Across 60 Languagescited by 3
  6. 2023cantnlp@LT-EDI-2023: Homophobia/Transphobia Detection in Social Media Comments using Spatio-Temporally Retrained Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 2.

6 papers.

  1. 2024Language Model Alignment in Multilingual Trolley Problemscited by 46
  2. 2025From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centeredcited by 21
  3. 2026XCR-Bench: Benchmarking Cross-Cultural Reasoning in LLMs via Culture-Specific Items and Hall's Triadcited by 3
  4. 2024Uncovering Regional Defaults from Photorealistic Forests in Text-to-Image Generation with DALL-E 2cited by 0
  5. 2025Performance Gains of LLMs With Humans in a World of LLMs Versus Humanscited by 0
  6. 2026Mapping the City Through the Lens of Language Modelscited by 0