mlsanityData to 7 Oct 2026

XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation

arXiv 2004.01401 ↗First posted 3 Apr 2020cs.CLcited by 221

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce XGLUE, a new benchmark dataset that can be used to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora and evaluate their performance across a diverse set of cross-lingual tasks. Comparing to GLUE(Wang et al., 2019), which is labeled in English for natural language understanding tasks only, XGLUE has two main advantages: (1) it provides 11 diversified tasks that cover both natural language understanding and generation scenarios; (2) for each task, it provides labeled data in multiple languages. We extend a recent cross-lingual pre-trained model Unicoder(Huang et al., 2019) to cover both understanding and generation tasks, which is evaluated on XGLUE as a strong baseline. We also evaluate the base versions (12-layer) of Multilingual BERT, XLM and XLM-R for comparison.

Key facts. Cited by 221 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 57; 2022: 49; 2023: 40; 2024: 38; 2025: 26; 2026: 11.

221 papers.

Sort:

By area:

  1. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  2. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  3. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  4. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  5. 2022Challenges and Strategies in Cross-Cultural NLPcited by 216
  6. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 203
  7. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 189
  8. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  9. 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
  10. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  11. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  12. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
38 more
  1. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  2. 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 99
  3. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  4. 2021VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluationcited by 95
  5. 2022The Geometry of Multilingual Language Model Representationscited by 94
  6. 2024ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabiccited by 87
  7. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 78
  8. 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluationcited by 73
  9. 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
  10. 2024CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Koreancited by 65
  11. 2021Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoningcited by 64
  12. 2021AraT5: Text-to-Text Transformers for Arabic Language Generationcited by 63
  13. 2021Multi-Task Learning in Natural Language Processing: An Overviewcited by 61
  14. 2021IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generationcited by 57
  15. 2022IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languagescited by 54
  16. 2023Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLUcited by 53
  17. 2021ExplainaBoard: An Explainable Leaderboard for NLPcited by 50
  18. 2021AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languagescited by 50
  19. 2021A Primer on Pretrained Multilingual Language Modelscited by 49
  20. 2024DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languagescited by 49
  21. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  22. 2021Revisiting the Primacy of English in Zero-shot Cross-lingual Transfercited by 47
  23. 2022ORCA: A Challenging Benchmark for Arabic Language Understandingcited by 46
  24. 2024AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMscited by 46
  25. 2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
  26. 2024PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Datacited by 37
  27. 2022The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challengescited by 35
  28. 2023XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languagescited by 34
  29. 2021AMMU : A Survey of Transformer-based Biomedical Pretrained Language Modelscited by 32
  30. 2021ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generationcited by 31
  31. 2021OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languagescited by 30
  32. 2022Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialogcited by 30
  33. 2022Pretrained Models for Multilingual Federated Learningcited by 30
  34. 2021Multilingual Language Models Predict Human Reading Behaviorcited by 29
  35. 2024P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMscited by 29
  36. 2023Zero-shot cross-lingual transfer language selection using linguistic similaritycited by 28
  37. 2025EuroBERT: Scaling Multilingual Encoders for European Languagescited by 28
  38. 2023Revisiting non-English Text Simplification: A Unified Multilingual Benchmarkcited by 27

50 most cited of 221 All 221 (JSON)