mlsanityData to 7 Oct 2026

GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings

arXiv 2608.13698 ↗First posted 13 Aug 2026cs.CL cs.LGcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR), often optimized with Group Relative Policy Optimization (GRPO), has become a central recipe for improving the reasoning capabilities of pretrained language models but current studies remain heavily English-centric. We conduct a large-scale empirical study of multilingual and non-English GRPO across a wide range of base models, training languages, and different reasoning language rewards. We find that training to reason in the native language often leaves only a small gap to training for English reasoning. We further observe strong crosslingual transfer: training in one language often improves performance in many others. However, specific trends are highly model- and language-dependent. In some cases, training in a particular language induces severe regressions on out-of-domain capabilities in other languages. Our analysis shows that RLVR beyond English can provide broad crosslingual gains, but also requires broad evaluation to detect language-specific regressions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 53 references that are in the map. First posted on arXiv 13 Aug 2026.

Builds on

36 of 53 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Spurious Rewards: Rethinking Training Signals in RLVR12 Jun
  2. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  3. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  6. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov

Other references · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2017Decoupled Weight Decay Regularizationcited by 1,722
  5. 2025Gemma 3 Technical Reportcited by 1,686
  6. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  7. 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentioncited by 248
  8. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  9. 2025Skywork Open Reasoner 1 Technical Reportcited by 161
  10. 2025Retaining by Doing: The Role of On-Policy Data in Mitigating Forgettingcited by 82
  11. 2022Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generationcited by 62
  12. 2024xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoningcited by 46
18 more
  1. 2025Crosslingual Reasoning through Test-Time Scalingcited by 46
  2. 2025Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Trainingcited by 42
  3. 2024mCoT: Multilingual Instruction Tuning for Reasoning Consistency in Language Modelscited by 31
  4. 2026Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learningcited by 26
  5. 2025Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoningcited by 23
  6. 2025Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models?cited by 20
  7. 2025SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignmentcited by 17
  8. 2025Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learningcited by 17
  9. 2025Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Modelscited by 11
  10. 2025Conditions for Catastrophic Forgetting in Multilingual Translationcited by 10
  11. 2026Crosslingual On-Policy Self-Distillation for Multilingual Reasoningcited by 10
  12. 2026Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoningcited by 6
  13. 2026TAPO: Translation Augmented Policy Optimization for Multilingual Mathematical Reasoningcited by 6
  14. 2026R3S: Refining and Recovering Reinforcement Signals for Multilingual Understanding and Reasoningcited by 5
  15. 2026Multilingual Reasoning Gym: Multilingual Scaling of Procedural Reasoning Environmentscited by 5
  16. 2025Aligning Multilingual Reasoning with Verifiable Semantics from a High-Resource Expert Modelcited by 4
  17. 2026Cross-lingual Self-Consistency for Multilingual Reasoning with Language Modelscited by 3
  18. 2026Language as a Latent Variable for Reasoning Optimizationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Can GRPO Help LLMs Transcend Their Pretraining Origin?
  2. 2026Transformation-Augmented GRPO for Enhancing Exploration in Reasoning of Large Language Models
  3. 2025Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs
  4. 2026LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance
  5. 2025R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO
  6. 2025Parallel Scaling Law: Unveiling Reasoning Generalization through A Cross-Linguistic Perspective
  7. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce
  8. 2025Explore Data Left Behind in Reinforcement Learning for Reasoning Language Models