mlsanityData to 7 Oct 2026

Language Models are Multilingual Chain-of-Thought Reasoners

arXiv 2210.03057 ↗First posted 6 Oct 2022cs.CL cs.AI cs.LGcited by 518

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

We evaluate the reasoning abilities of large language models in multilingual settings. We introduce the Multilingual Grade School Math (MGSM) benchmark, by manually translating 250 grade-school math problems from the GSM8K dataset (Cobbe et al., 2021) into ten typologically diverse languages. We find that the ability to solve MGSM problems via chain-of-thought prompting emerges with increasing model scale, and that models have strikingly strong multilingual reasoning abilities, even in underrepresented languages such as Bengali and Swahili. Finally, we show that the multilingual reasoning abilities of language models extend to other tasks such as commonsense reasoning and word-in-context semantic judgment. The MGSM benchmark is publicly available at https://github.com/google-research/url-nlp.

Key facts. Cited by 518 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 40 of its 53 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

40 of 53 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  10. 2021Program Synthesis with Large Language Modelscited by 3,509
  11. 2022Emergent Abilities of Large Language Modelscited by 2,483
  12. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
28 more
  1. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  2. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  3. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  4. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  5. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  6. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  7. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  8. 2019How multilingual is Multilingual BERT?cited by 642
  9. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  10. 2017Word Translation Without Parallel Datacited by 480
  11. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  12. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 370
  13. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  14. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  15. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  16. 2022Natural Language to Code Translation with Executioncited by 122
  17. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  18. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  19. 2019Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsingcited by 69
  20. 2021Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoningcited by 64
  21. 2021Discrete and Soft Prompting for Multilingual Modelscited by 49
  22. 2018Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numberscited by 42
  23. 2020XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualizationcited by 42
  24. 2020From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformerscited by 35
  25. 2022Multi Task Learning For Zero Shot Performance Prediction of Multilingual Modelscited by 20
  26. 2021Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignmentcited by 19
  27. 2019Polyglot Contextual Representations Improve Crosslingual Transfercited by 16
  28. 2021Substructure Distribution Projection for Zero-Shot Cross-Lingual Dependency Parsingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 13; 2023: 84; 2024: 119; 2025: 169; 2026: 133.

518 papers.

Sort:

By area:

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2023A Survey of Large Language Modelscited by 2,537
  5. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  6. 2025Gemma 3 Technical Reportcited by 1,686
  7. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  8. 2023PaLM 2 Technical Reportcited by 1,154
  9. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  10. 2022A Survey on In-context Learningcited by 717
  11. 2024Phi-4 Technical Reportcited by 675
  12. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
38 more
  1. 2023A Comprehensive Overview of Large Language Modelscited by 639
  2. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  3. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  4. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  5. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  6. 2024Automated Design of Agentic Systemscited by 360
  7. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  8. 2023Large Language Models as Tool Makerscited by 293
  9. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  10. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  11. 2023A Survey on Model Compression for Large Language Modelscited by 263
  12. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 256
  13. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  14. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  15. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
  16. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
  17. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Promptingcited by 163
  18. 2024How do Large Language Models Handle Multilingualism?cited by 148
  19. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  20. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 146
  21. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  22. 2025Phi-4-reasoning Technical Reportcited by 146
  23. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 142
  24. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  25. 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 138
  26. 2023How well do Large Language Models perform in Arithmetic tasks?cited by 130
  27. 2024Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluationscited by 130
  28. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 124
  29. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
  30. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 122
  31. 2023Teaching Arithmetic to Small Transformerscited by 121
  32. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 121
  33. 2024M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thoughtcited by 120
  34. 2024Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffoldingcited by 115
  35. 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
  36. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 106
  37. 2024Large Language Models Meet NLP: A Surveycited by 102
  38. 2024Chain of Thoughtlessness? An Analysis of CoT in Planningcited by 100

50 most cited of 518 All 518 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025MathMist: A Parallel Multilingual Benchmark Dataset for Mathematical Problem Solving and Reasoning
  2. 2024mCoT: Multilingual Instruction Tuning for Reasoning Consistency in Language Models
  3. 2026LLM Parameters for Math Across Languages: Shared or Separate?
  4. 2025Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models?
  5. 2026Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners
  6. 2025Best-of-L: Cross-Lingual Reward Modeling for Mathematical Reasoning
  7. 2026Align to the Pivot: Dual Alignment with Self-Feedback for Multilingual Math Reasoning
  8. 2025MathRobust-LV: Evaluation of Large Language Models' Robustness to Linguistic Variations in Mathematical Reasoning