mlsanityData to 7 Oct 2026

GlobalBench: A Benchmark for Global Progress in Natural Language Processing

arXiv 2305.14716 ↗First posted 24 May 2023cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Despite the major advances in NLP, significant disparities in NLP system performance across languages still exist. Arguably, these are due to uneven resource allocation and sub-optimal incentives to work on less resourced languages. To track and further incentivize the global development of equitable language technology, we introduce GlobalBench. Prior multilingual benchmarks are static and have focused on a limited number of tasks and languages. In contrast, GlobalBench is an ever-expanding collection that aims to dynamically track progress on all NLP datasets in all languages. Rather than solely measuring accuracy, GlobalBench also tracks the estimated per-speaker utility and equity of technology across all languages, providing a multi-faceted view of how language technology is serving people of the world. Furthermore, GlobalBench is designed to identify the most under-served languages, and rewards research efforts directed towards those languages. At present, the most under-served languages are the ones with a relatively high population, but nonetheless overlooked by composite multilingual benchmarks (like Punjabi, Portuguese, and Wu Chinese). Currently, GlobalBench covers 966 datasets in 190 languages, and has 1,128 system submissions spanning 62 languages.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 33 of its 48 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

33 of 48 references.

References · 33

  1. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  2. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  5. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  6. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  7. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  8. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  9. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  10. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  11. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  12. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
21 more
  1. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  2. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  3. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  4. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  5. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 221
  6. 2018ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehensioncited by 198
  7. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
  8. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
  9. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  10. 2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
  11. 2018End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completioncited by 136
  12. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  13. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  14. 2020IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understandingcited by 108
  15. 2021ExplainaBoard: An Explainable Leaderboard for NLPcited by 50
  16. 2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
  17. 2021SD-QA: Spoken Dialectal Question Answering for the Real Worldcited by 36
  18. 2022Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resourcescited by 25
  19. 2022Evaluating the Diversity, Equity and Inclusion of NLP Technology: A Case Study for Indian Languagescited by 20
  20. 2022Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switchingcited by 6
  21. 2022NusaCrowd: A Call for Open and Reproducible NLP Research in Indonesian Languagescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 1; 2026: 1.

8 papers.

  1. 2023Large Language Models are Fixated by Red Herrings: Exploring Creative Problem Solving and Einstellung Effect using the Only Connect Wall Datasetcited by 17
  2. 2024LLM Whisperer: An Inconspicuous Attack to Bias LLM Responsescited by 15
  3. 2024Teaching LLMs to Abstain across Languages via Multilingual Feedbackcited by 14
  4. 2024MINERS: Multilingual Language Models as Semantic Retrieverscited by 11
  5. 2025FLUKE: A Linguistically-Driven and Task-Agnostic Framework for Robustness Evaluationcited by 3
  6. 2024Direct Punjabi to English speech translation using discrete unitscited by 1
  7. 2024Filtered not Mixed: Stochastic Filtering-Based Online Gating for Mixture of Large Language Modelscited by 0
  8. 2026Lost in the Tower of Babel: The Adverse Effects of Incidental Multilingualism in LLMscited by 0