mlsanityData to 7 Oct 2026

Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gap

arXiv 2501.02448 ↗First posted 5 Jan 2025cs.CLcited by 41

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Large language models (LLMs) demonstrate exceptional performance on complex reasoning tasks. However, despite their strong reasoning capabilities in high-resource languages (e.g., English and Chinese), a significant performance gap persists in other languages. To investigate this gap in Korean, we introduce HRM8K, a benchmark comprising 8,011 English-Korean parallel bilingual math problems. Through systematic analysis of model behaviors, we identify a key finding: these performance disparities stem primarily from difficulties in comprehending non-English inputs, rather than limitations in reasoning capabilities. Based on these findings, we propose UST (Understand, Solve, and Translate), a method that strategically uses English as an anchor for reasoning and solution generation. By fine-tuning the model on 130k synthetically generated data points, UST achieves a 10.91% improvement on the HRM8K benchmark and reduces the multilingual performance gap from 11.6% to 0.7%. Additionally, we show that improvements from UST generalize effectively to different Korean domains, demonstrating that capabilities acquired from machine-verifiable content can be generalized to other areas. We publicly release the benchmark, training dataset, and models.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 49 of its 68 references that are in the map. First posted on arXiv 5 Jan 2025.

Builds on

49 of 68 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models10 Oct
  2. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data2 Oct
  3. 2024OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI18 Jun
  4. 2024A Careful Examination of Large Language Model Performance on Grade School Arithmetic1 May
  5. 2024ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models22 Feb
  6. 2023Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations31 Oct
  7. 2022Language Models are Multilingual Chain-of-Thought Reasoners6 Oct

Other references · 42

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2024GPT-4o System Cardcited by 4,467
  8. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  10. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  11. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  12. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
30 more
  1. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  2. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 380
  3. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 331
  4. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
  5. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 249
  6. 2024RedPajama: an Open Dataset for Training Large Language Modelscited by 178
  7. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Promptingcited by 163
  8. 2024How do Large Language Models Handle Multilingualism?cited by 148
  9. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 142
  10. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 136
  11. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 121
  12. 2024Granite Code Models: A Family of Open Foundation Models for Code Intelligencecited by 116
  13. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  14. 2024Benchmarking Benchmark Leakage in Large Language Modelscited by 102
  15. 2024OffsetBias: Leveraging Debiased Data for Tuning Evaluatorscited by 94
  16. 2024KMMLU: Measuring Massive Multitask Language Understanding in Koreancited by 91
  17. 2024CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Koreancited by 65
  18. 2023Elo Uncovered: Robustness and Best Practices in Language Model Evaluationcited by 58
  19. 2024LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedbackcited by 51
  20. 2024MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Datacited by 51
  21. 2023HAE-RAE Bench: Evaluation of Korean Knowledge in Language Modelscited by 47
  22. 2024Question Translation Training for Better Multilingual Reasoningcited by 41
  23. 2024MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmarkcited by 41
  24. 2024Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in?cited by 40
  25. 2023PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuningcited by 39
  26. 2024Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languagescited by 38
  27. 2024MM-Eval: A Multilingual Meta-Evaluation Benchmark for LLM-as-a-Judge and Reward Modelscited by 34
  28. 2024LLM-as-a-Judge & Reward Model: What They Can and Cannot Docited by 32
  29. 2024Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmarkcited by 23
  30. 2023Removing Non-Stationary Knowledge From Pre-Trained Language Models for Entity-Level Sentiment Classification in Financecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 20; 2026: 21.

41 papers.

Sort:

By area:

  1. 2025Crosslingual Reasoning through Test-Time Scalingcited by 46
  2. 2025Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoningcited by 33
  3. 2025A Survey of Multilingual Reasoning in Language Modelscited by 31
  4. 2025EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modescited by 26
  5. 2025R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learningcited by 21
  6. 2025Language-Specific Latent Process Hinders Cross-Lingual Performancecited by 20
  7. 2025When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasonerscited by 16
  8. 2025Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?cited by 11
  9. 2025Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thoughtcited by 10
  10. 2026Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steeringcited by 8
  11. 2025On Code-Induced Reasoning in LLMscited by 5
  12. 2026R3S: Refining and Recovering Reinforcement Signals for Multilingual Understanding and Reasoningcited by 5
29 more
  1. 2025Multi-Step Reasoning in Korean and the Emergent Miragecited by 4
  2. 2025Trillion 7B Technical Reportcited by 4
  3. 2025Investigating Bias: A Multilingual Pipeline for Generating, Solving, and Evaluating Math Problems with LLMscited by 4
  4. 2026Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMscited by 4
  5. 2025BenchHub: A Unified Benchmark Suite for Holistic and Customizable LLM Evaluationcited by 3
  6. 2025Simulating LLM-to-LLM Tutoring for Multilingual Math Feedbackcited by 3
  7. 2025EfficientXLang: Towards Improving Token Efficiency Through Cross-Lingual Reasoningcited by 3
  8. 2026Mi:dm 2.0 Korea-centric Bilingual Language Modelscited by 3
  9. 2026K-EXAONE Technical Reportcited by 2
  10. 2026Solar Open Technical Reportcited by 2
  11. 2026x1: Learning to Think Adaptively Across Languages and Culturescited by 2
  12. 2026ResearchMath-14K: Scaling Research-Level Mathematics via Agentscited by 2
  13. 2026Solar Open 2 Technical Reportcited by 2
  14. 2025Redefining Evaluation Standards: A Unified Framework for Evaluating the Korean Capabilities of Language Modelscited by 1
  15. 2025KAIO: A Collection of More Challenging Korean Questionscited by 1
  16. 2026Do LLMs Need Inherent Reasoning Before Reinforcement Learning? A Study in Korean Self-Correctioncited by 1
  17. 2026Learning When to Translate for Multilingual Reasoningcited by 1
  18. 2026Korean Culture into LLM Alignment: Toward Cultural Coherencecited by 1
  19. 2026Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discoverycited by 1
  20. 2026LLM Parameters for Math Across Languages: Shared or Separate?cited by 1
  21. 2026On-Policy Delta Distillation for Multilingual Math Reasoningcited by 1
  22. 2025Won: Establishing Best Practices for Korean Financial NLPcited by 0
  23. 2025Ko-MuSR: A Multistep Soft Reasoning Benchmark for LLMs Capable of Understanding Koreancited by 0
  24. 2026A.X K1 Technical Reportcited by 0
  25. 2026Mi:dm K 2.5 Procited by 0
  26. 2026The Language-Energy Divide: Measuring Energy Costs of Multilingual LLM Inferencecited by 0
  27. 2026Mind the Cap: Output-Budget Regimes Change the Measured Multilingual Reasoning Gapcited by 0
  28. 2026K-EXAONE 2.0 Technical Reportcited by 0
  29. 2026Quantifying and Mitigating Korean Jamo-Level Typographical Vulnerabilities in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MathMist: A Parallel Multilingual Benchmark Dataset for Mathematical Problem Solving and Reasoning
  2. 2025PolyMath: Evaluating Mathematical Reasoning in Multilingual Contexts
  3. 2025Large Language Models and Mathematical Reasoning Failures
  4. 2025MMATH: A Multilingual Benchmark for Mathematical Reasoning
  5. 2026Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamil
  6. 2025Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges
  7. 2026Beyond Input Understanding: Diagnosing Multilingual Mathematical Reasoning with Directed Acyclic Trace Graphs
  8. 2026MGSM-Pro: A Simple Strategy for Robust Multilingual Mathematical Reasoning Evaluation