# Crosslingual On-Policy Self-Distillation for Multilingual Reasoning

arXiv: https://arxiv.org/abs/2605.09548 · HTML: https://mlsanity.com/paper/2605.09548/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 30 of its 65 references that are in the map.
- First posted on arXiv 10 May 2026.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)
- Knowledge distillation / [On-policy and self-distillation for LLMs](https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/)

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models](https://mlsanity.com/paper/2601.18734/) (2026)
- [OpenThoughts: Data Recipes for Reasoning Models](https://mlsanity.com/paper/2506.04178/) (2025)
- [Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe](https://mlsanity.com/paper/2604.13016/) (2026)
- [Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs](https://mlsanity.com/paper/2506.14245/) (2025)
- [Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation](https://mlsanity.com/paper/2602.12125/) (2026)
- [Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?](https://mlsanity.com/paper/2603.24472/) (2026)
- [LLaMA Beyond English: An Empirical Study on Language Capability Transfer](https://mlsanity.com/paper/2401.01055/) (2024)
- [CRISP: Compressed Reasoning via Iterative Self-Policy Distillation](https://mlsanity.com/paper/2603.05433/) (2026)
- [PolyMath: Evaluating Mathematical Reasoning in Multilingual Contexts](https://mlsanity.com/paper/2504.18428/) (2025)
- [Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers](https://mlsanity.com/paper/2404.04925/) (2024)
- [Crosslingual Reasoning through Test-Time Scaling](https://mlsanity.com/paper/2505.05408/) (2025)
- [OPSDL: On-Policy Self-Distillation for Long-Context Language Models](https://mlsanity.com/paper/2604.17535/) (2026)
- [Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning](https://mlsanity.com/paper/2507.05418/) (2025)
- [Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models?](https://mlsanity.com/paper/2505.17407/) (2025)
- [Long Chain-of-Thought Reasoning Across Languages](https://mlsanity.com/paper/2508.14828/) (2025)
- [Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning](https://mlsanity.com/paper/2510.07300/) (2025)
- [Systematically Analyzing Prompt Injection Vulnerabilities in Diverse LLM Architectures](https://mlsanity.com/paper/2410.23308/) (2024)
- [Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?](https://mlsanity.com/paper/2510.27269/) (2025)
- [Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners](https://mlsanity.com/paper/2601.02996/) (2026)
- [Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs](https://mlsanity.com/paper/2509.23657/) (2025)
- [Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoning](https://mlsanity.com/paper/2601.18722/) (2026)
- [Reasoning Transfer for an Extremely Low-Resource and Endangered Language: Bridging Languages Through Sample-Efficient Language Understanding](https://mlsanity.com/paper/2504.02890/) (2025)
- [Aligning Multilingual Reasoning with Verifiable Semantics from a High-Resource Expert Model](https://mlsanity.com/paper/2509.25543/) (2025)
- [What Makes Good Multilingual Reasoning? Disentangling Traces with Measurable Features](https://mlsanity.com/paper/2604.04720/) (2026)

## Built on by (most cited first, 10 of 10)

- [Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models](https://mlsanity.com/paper/2605.30251/) (2026)
- [Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories](https://mlsanity.com/paper/2606.03979/) (2026)
- [Learning When to Translate for Multilingual Reasoning](https://mlsanity.com/paper/2606.02465/) (2026)
- [Mind the Cap: Output-Budget Regimes Change the Measured Multilingual Reasoning Gap](https://mlsanity.com/paper/2608.04160/) (2026)
- [On-Policy Delta Distillation for Multilingual Math Reasoning](https://mlsanity.com/paper/2608.05802/) (2026)
- [CroCo: Cross-Lingual Contrastive Preference Tuning on Self-Generations](https://mlsanity.com/paper/2605.26293/) (2026)
- [Beyond Input Understanding: Diagnosing Multilingual Mathematical Reasoning with Directed Acyclic Trace Graphs](https://mlsanity.com/paper/2605.27715/) (2026)
- [SHARD: Safe and Helpful Alignment via Self-Reframing Distillation](https://mlsanity.com/paper/2606.15517/) (2026)
- [RP-OPSD: Reasoning-Pivot-Guided On-Policy Self-Distillation for Multilingual Reasoning Transfer](https://mlsanity.com/paper/2608.06347/) (2026)
- [GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings](https://mlsanity.com/paper/2608.13698/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have achieved remarkable progress in mathematical reasoning, but this ability is not equally accessible across languages. Especially low-resource languages exhibit much lower reasoning performance. To address this, we propose Crosslingual On-Policy Self-Distillation (COPSD), which transfers a model's own high-resource reasoning behavior to low-resource languages. COPSD uses the same model as student and teacher: the student sees only the low-resource problem, while the teacher receives privileged crosslingual context, including the problem translation and reference solution in English. Training minimizes full-distribution token-level divergence on the student's own rollouts, providing dense supervision while avoiding the sparsity and instability of outcome-only reinforcement learning (RL). Experiments on 17 low-resource African languages show that COPSD consistently improves low-resource mathematical reasoning across model sizes and substantially outperforms Group Relative Policy Optimization (GRPO). Further analyses show that COPSD improves answer-format adherence, strengthens test-time scaling, and generalizes to harder multilingual reasoning benchmarks, with especially large gains for lower-resource languages. We make our code and data available at: https://github.com/cisnlp/COPSD.