mlsanityData to 7 Oct 2026

Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought

arXiv 2510.04230 ↗First posted 5 Oct 2025cs.CLcited by 10

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Recent frontier models employ long chain-of-thought reasoning to explore solution spaces in context and achieve stonger performance. While many works study distillation to build smaller yet capable models, most focus on English and little is known about language-specific reasoning. To bridge this gap, we first introduct **Language-Mixed CoT**, a reasoning schema that switches between English and a target language, using English as an anchor to excel in reasoning while minimizing translation artificats. As a Korean case study, we curate **Yi-Sang**: 5.79M native-Korean prompts from web Q&A, exams, STEM, and code; 3.7M long reasoning traces generated from Qwen3-32B; and a targeted 260k high-yield subset. We train ninve models (4B-35B) across six families (Qwen2.5, Llama-3.1, Gemma-3, etc). Our best model, **KO-REAson-35B**, achieves state-of-the-art performance, with the highest overall average score (64.0 ± 25), ranking first on 5/9 benchmarks and second on the remainder. Samller and mid-sized models also benefit substantially, with an average improvement of +18.6 points across teh evaluated nine benchmarks. Ablations show **Language-Mixed CoT** is more effective than monolingual CoT, also resulting in cross-lingual and mult-modal performance gains. We release our data-curation pipeline, evaluation system, datasets, and models to advance research on language-specific reasoning. Data and model collection: https://huggingface.co/KOREAson.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 58 of its 77 references that are in the map. First posted on arXiv 5 Oct 2025.

Builds on

58 of 77 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Models6 Jun
  2. 2025OpenThoughts: Data Recipes for Reasoning Models4 Jun
  3. 2025When Models Reason in Your Language: Controlling Thinking Language Comes at the Cost of Accuracy28 May
  4. 2025Qwen3 Technical Report14 May
  5. 2025AM-Thinking-v1: Advancing the Frontier of Reasoning at 32B Scale13 May
  6. 2025Crosslingual Reasoning through Test-Time Scaling8 May
  7. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility9 Apr
  8. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond13 Mar
  9. 2025Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipe13 Feb
  10. 2024Question Translation Training for Better Multilingual Reasoning15 Jan

Other references · 48

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  8. 2025Gemma 3 Technical Reportcited by 1,686
  9. 2025s1: Simple test-time scalingcited by 1,311
  10. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  11. 2024OpenAI o1 System Cardcited by 1,220
  12. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
36 more
  1. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  2. 2025LIMO: Less is More for Reasoningcited by 488
  3. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  4. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  5. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 323
  6. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  7. 20242 OLMo 2 Furiouscited by 302
  8. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
  9. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 233
  10. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 199
  11. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 194
  12. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 151
  13. 2024Liger Kernel: Efficient Triton Kernels for LLM Trainingcited by 123
  14. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scalingcited by 110
  15. 2024KMMLU: Measuring Massive Multitask Language Understanding in Koreancited by 91
  16. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learningcited by 85
  17. 2024CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Koreancited by 65
  18. 2021Scaling Scaling Laws with Board Gamescited by 62
  19. 2025Magistralcited by 57
  20. 2025rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Datasetcited by 49
  21. 2023HAE-RAE Bench: Evaluation of Korean Knowledge in Language Modelscited by 47
  22. 2025Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gapcited by 41
  23. 2025Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoningcited by 33
  24. 2024mCoT: Multilingual Instruction Tuning for Reasoning Consistency in Language Modelscited by 31
  25. 2025EXAONE Deep: Reasoning Enhanced Language Modelscited by 20
  26. 2024Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaborationcited by 19
  27. 2025Kanana: Compute-efficient Bilingual Language Modelscited by 19
  28. 2024Cross-lingual Transfer of Reward Models in Multilingual Alignmentcited by 17
  29. 2025Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learningcited by 17
  30. 2025KoBALT: Korean Benchmark For Advanced Linguistic Taskscited by 12
  31. 2025Lost in Literalism: How Supervised Training Shapes Translationese in LLMscited by 11
  32. 2025Good Learners Think Their Thinking: Generative PRM Makes Large Reasoning Model More Efficient Math Learnercited by 11
  33. 2025From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for LLM Evaluationcited by 6
  34. 2024Improving Fine-grained Visual Understanding in VLMs through Text-Only Trainingcited by 5
  35. 2025Pensez: Less Data, Better Reasoning -- Rethinking French LLMcited by 5
  36. 2025KAIO: A Collection of More Challenging Korean Questionscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages
  2. 2025Long Chain-of-Thought Reasoning Across Languages
  3. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  4. 2025AdaMCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Multilingual Chain-of-Thought
  5. 2025Reasoning Transfer for an Extremely Low-Resource and Endangered Language: Bridging Languages Through Sample-Efficient Language Understanding
  6. 2025A Survey of Multilingual Reasoning in Language Models
  7. 2025MultiNRC: A Challenging and Native Multilingual Reasoning Evaluation Benchmark for LLMs
  8. 2025Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models