mlsanityData to 7 Oct 2026

Dataset Geography: Mapping Language Data to Language Users

arXiv 2112.03497 ↗First posted 7 Dec 2021cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

As language technologies become more ubiquitous, there are increasing efforts towards expanding the language diversity and coverage of natural language processing (NLP) systems. Arguably, the most important factor influencing the quality of modern NLP systems is data availability. In this work, we study the geographical representativeness of NLP datasets, aiming to quantify if and by how much do NLP datasets match the expected needs of the language speakers. In doing so, we use entity recognition and linking systems, also making important observations about their cross-lingual consistency and giving suggestions for more robust evaluation. Last, we explore some geographical and economic factors that may explain the observed dataset distributions. Code and data are available here: https://github.com/ffaisal93/dataset_geography. Additional visualizations are available here: https://nlp.cs.gmu.edu/project/datasetmaps/.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 26 of its 58 references that are in the map. First posted on arXiv 7 Dec 2021.

Builds on

26 of 58 references.

References · 26

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  2. 2018Datasheets for Datasetscited by 1,371
  3. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  4. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  5. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  6. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  7. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  8. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  9. 2021Visually Grounded Reasoning across Languages and Culturescited by 188
  10. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  11. 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
  12. 2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
14 more
  1. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  2. 2020When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Modelscited by 131
  3. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  4. 2020X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Modelscited by 117
  5. 2018A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indicescited by 97
  6. 2018Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processingcited by 63
  7. 2021Multilingual Autoregressive Entity Linkingcited by 50
  8. 2017Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projectioncited by 36
  9. 2021SD-QA: Spoken Dialectal Question Answering for the Real Worldcited by 36
  10. 2018A Pragmatic Guide to Geoparsing Evaluationcited by 17
  11. 2019Topics to Avoid: Demoting Latent Confounds in Text Classificationcited by 14
  12. 2014Cross-lingual Annotation Projection for Semantic Rolescited by 11
  13. 2021Towards More Equitable Question Answering Systems: How Much More Data Do You Need?cited by 8
  14. 2021Language Invariant Properties in Natural Language Processingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 2; 2024: 6; 2025: 4; 2026: 3.

22 papers.

Sort:

By area:

  1. 2022Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP Worldcited by 55
  2. 2022MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognitioncited by 38
  3. 2022GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Modelscited by 30
  4. 2024The Future of Large Language Model Pre-training is Federatedcited by 30
  5. 2022Geographic and Geopolitical Biases of Language Modelscited by 29
  6. 2023A Material Lens on Coloniality in NLPcited by 19
  7. 2024On the Scaling Laws of Geographical Representation in Language Modelscited by 14
  8. 2022TaTa: A Multilingual Table-to-Text Dataset for African Languagescited by 10
  9. 2022DataLab: A Platform for Data Analysis and Interventioncited by 8
  10. 2025Do You Know About My Nation? Investigating Multilingual Language Models' Cultural Literacy Through Factual Knowledgecited by 8
  11. 2024Worldwide Federated Training of Language Modelscited by 5
  12. 2024DEPT: Decoupled Embeddings for Pre-training Language Modelscited by 4
10 more
  1. 2023SituatedGen: Incorporating Geographical and Temporal Contexts into Generative Commonsense Reasoningcited by 3
  2. 2025Modality Matching Matters: Calibrating Language Distances for Cross-Lingual Transfer in URIEL+cited by 3
  3. 2024Towards Better Inclusivity: A Diverse Tweet Corpus of English Varietiescited by 2
  4. 2025Conflicts in Texts: Data, Implications and Challengescited by 2
  5. 2026The Culture Funnel: You Can't Align What isn't in the Datacited by 2
  6. 2021Graph-based Ensemble Machine Learning for Student Performance Predictioncited by 0
  7. 2024Validating and Exploring Large Geographic Corporacited by 0
  8. 2025Bridging Cultural Distance Between Models Default and Local Classroom Demands: How Global Teachers Adopt GenAI to Support Everyday Teaching Practicescited by 0
  9. 2026Where Do Images Come From? Analyzing Captions to Geographically Profile Datasetscited by 0
  10. 2026AtlasNLP: A Country-Aware Atlas of Dataset Representation in NLPcited by 0