mlsanityData to 14 Oct 2026

Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning

arXiv 2106.06937 ↗First posted 13 Jun 2021cs.CL cs.AIcited by 65

Not on the map. Why?

Abstract (arXiv)

Commonsense reasoning research has so far been limited to English. We aim to evaluate and improve popular multilingual language models (ML-LMs) to help advance commonsense reasoning (CSR) beyond English. We collect the Mickey Corpus, consisting of 561k sentences in 11 different languages, which can be used for analyzing and improving ML-LMs. We propose Mickey Probe, a language-agnostic probing task for fairly evaluating the common sense of popular ML-LMs across different languages. In addition, we also create two new datasets, X-CSQA and X-CODAH, by translating their English versions to 15 other languages, so that we can evaluate popular ML-LMs for cross-lingual commonsense reasoning. To improve the performance beyond English, we propose a simple yet effective method -- multilingual contrastive pre-training (MCP). It significantly enhances sentence representations, yielding a large performance gain on both benchmarks.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 13 Jun 2021.

Builds on

26 of 35 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2019Language Models as Knowledge Bases?cited by 1,936
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  8. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  9. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  10. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  11. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  12. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
14 more
  1. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  2. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
  3. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  4. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  5. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 333
  6. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  7. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 222
  8. 2019KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoningcited by 216
  9. 2018Marian: Fast Neural Machine Translation in C++cited by 203
  10. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  11. 2020Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answeringcited by 122
  12. 2020X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Modelscited by 117
  13. 2020Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Modelscited by 110
  14. 2020Towards an Atlas of Cultural Commonsense for Machine Reasoningcited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 6; 2023: 11; 2024: 15; 2025: 15; 2026: 12.

65 papers.

Sort:

By area:

  1. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  2. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  3. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 240
  4. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
  5. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 206
  6. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
  7. 2024How do Large Language Models Handle Multilingualism?cited by 150
  8. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  9. 2023KoBBQ: Korean Bias Benchmark for Question Answeringcited by 71
  10. 2021Broaden the Vision: Geo-Diverse Visual Commonsense Reasoningcited by 59
  11. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  12. 2024LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languagescited by 48
38 more
  1. 2023Benchmarks for Automated Commonsense Reasoning: A Surveycited by 40
  2. 2023Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed?cited by 39
  3. 2022Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Surveycited by 38
  4. 2023COPAL-ID: Indonesian Language Reasoning with Local Culture and Nuancescited by 35
  5. 2025Enhancing Multilingual LLM Pretraining with Model-Based Data Selectioncited by 35
  6. 2022GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Modelscited by 30
  7. 2024Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowlycited by 27
  8. 2024Position: Measure Dataset Diversity, Don't Just Claim Itcited by 26
  9. 2024Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Interventioncited by 24
  10. 2025Localized Cultural Knowledge is Conserved and Controllable in Large Language Modelscited by 20
  11. 2025Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenizationcited by 20
  12. 2024An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inferencecited by 18
  13. 2024HyperCLOVA X Technical Reportcited by 18
  14. 2021Scientia Potentia Est -- On the Role of Knowledge in Computational Argumentationcited by 17
  15. 2025Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Trainingcited by 16
  16. 2023GD-COMET: A Geo-Diverse Commonsense Inference Modelcited by 13
  17. 2024mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humanscited by 10
  18. 2025CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answeringcited by 10
  19. 2025Revisiting Multilingual Data Mixtures in Language Model Pretrainingcited by 10
  20. 2021Leveraging Knowledge in Multilingual Commonsense Reasoningcited by 9
  21. 2024The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insightscited by 7
  22. 2025Multilingual Prompt Engineering in Large Language Models: A Survey Across NLP Taskscited by 6
  23. 2025AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignmentcited by 6
  24. 2026How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Datacited by 6
  25. 2021Natural Language Processing with Commonsense Knowledge: A Surveycited by 4
  26. 2022Commonsense Knowledge Salience Evaluation with a Benchmark Dataset in E-commercecited by 4
  27. 2024Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraintscited by 4
  28. 2025The Cultural Gene of Large Language Models: A Study on the Impact of Cross-Corpus Training on Model Values and Biasescited by 4
  29. 2022Few-shot News Recommendation via Cross-lingual Transfercited by 3
  30. 20242M-BELEBELE: Highly Multilingual Speech and American Sign Language Comprehension Datasetcited by 3
  31. 2026CAGE: A Framework for Culturally Adaptive Red-Teaming Benchmark Generationcited by 3
  32. 2023ChatGPT Prompting Cannot Estimate Predictive Uncertainty in High-Resource Languagescited by 2
  33. 2024Unraveling Babel: Exploring Multilingual Activation Patterns of LLMs and Their Applicationscited by 2
  34. 2026Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilingualitycited by 2
  35. 2021Transfer Learning for Multi-lingual Tasks -- a Surveycited by 1
  36. 2023PHALM: Building a Knowledge Graph from Scratch by Prompting Humans and a Language Modelcited by 1
  37. 2024LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentationcited by 1
  38. 2025Statement-Tuning Enables Efficient Cross-lingual Generalization in Encoder-only Modelscited by 1

50 most cited of 65 All 65 (JSON)