mlsanityData to 7 Oct 2026

XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning

arXiv 2005.00333 ↗First posted 1 May 2020cs.CLcited by 330

Not on the map. Why?

Cited by 27 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In order to simulate human language capacity, natural language processing systems must be able to reason about the dynamics of everyday situations, including their possible causes and effects. Moreover, they should be able to generalise the acquired world knowledge to new languages, modulo cultural differences. Advances in machine reasoning and cross-lingual transfer depend on the availability of challenging evaluation benchmarks. Motivated by both demands, we introduce Cross-lingual Choice of Plausible Alternatives (XCOPA), a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages, which includes resource-poor languages like Eastern Apur'imac Quechua and Haitian Creole. We evaluate a range of state-of-the-art models on this novel dataset, revealing that the performance of current methods based on multilingual pretraining and zero-shot fine-tuning falls short compared to translation-based transfer. Finally, we propose strategies to adapt multilingual models to out-of-sample resource-lean languages where only a small corpus or a bilingual dictionary is available, and report substantial improvements over the random baseline. The XCOPA dataset is freely available at github.com/cambridgeltl/xcopa.

Key facts. Cited by 330 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 33; 2023: 52; 2024: 99; 2025: 79; 2026: 41.

330 papers.

Sort:

By area:

  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2024Qwen2 Technical Reportcited by 2,076
  3. 2023PaLM 2 Technical Reportcited by 1,154
  4. 2023A Comprehensive Overview of Large Language Modelscited by 639
  5. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  6. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  7. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  8. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  9. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  10. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  11. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 233
  12. 2022Challenges and Strategies in Cross-Cultural NLPcited by 216
38 more
  1. 2021Visually Grounded Reasoning across Languages and Culturescited by 188
  2. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  3. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  4. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Promptingcited by 163
  5. 2024Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrencecited by 155
  6. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 149
  7. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 145
  8. 2021Composable Sparse Fine-Tuning for Cross-Lingual Transfercited by 144
  9. 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
  10. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 121
  11. 2023M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Modelscited by 109
  12. 2023Tokenizer Choice For LLM Training: Negligible or Crucial?cited by 105
  13. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmeticcited by 105
  14. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  15. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 96
  16. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  17. 2023Do Multilingual Language Models Think Better in English?cited by 87
  18. 2023Modular Deep Learningcited by 82
  19. 2023Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptationcited by 79
  20. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 78
  21. 2023Review of Large Vision Models and Visual Prompt Engineeringcited by 77
  22. 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluationcited by 73
  23. 2023KoBBQ: Korean Bias Benchmark for Question Answeringcited by 71
  24. 2021It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoningcited by 69
  25. 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
  26. 2022NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languagescited by 67
  27. 2024Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Modelscited by 67
  28. 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 65
  29. 2021Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoningcited by 64
  30. 2023PolyLM: An Open Source Polyglot Large Language Modelcited by 64
  31. 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
  32. 2024Why do language models perform worse for morphologically complex languages?cited by 62
  33. 2024Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectorscited by 61
  34. 2025A Systematic Survey of Automatic Prompt Optimization Techniquescited by 61
  35. 2021Broaden the Vision: Geo-Diverse Visual Commonsense Reasoningcited by 59
  36. 2022AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Modelcited by 57
  37. 2023MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Taskscited by 57
  38. 2024IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languagescited by 55

50 most cited of 330 All 330 (JSON)