mlsanityData to 7 Oct 2026

MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs

arXiv 2402.16352 ↗First posted 26 Feb 2024cs.CL cs.AIcited by 72

On the map: LLM reasoning → Mathematical reasoning

also in 1 more
Abstract (arXiv)

Large language models (LLMs) have exhibited great potential in mathematical reasoning. However, there remains a performance gap in this area between existing open-source models and closed-source models such as GPT-4. In this paper, we introduce MathGenie, a novel method for generating diverse and reliable math problems from a small-scale problem-solution dataset (denoted as seed data). We augment the ground-truth solutions of our seed data and train a back-translation model to translate the augmented solutions back into new questions. Subsequently, we generate code-integrated solutions for the new questions. To ensure the correctness of the code-integrated solutions, we employ rationale-based strategy for solution verification. Various pretrained models, ranging from 7B to 70B, are trained on the newly curated data to test the effectiveness of the proposed augmentation technique, resulting in a family of models known as MathGenieLM. These models consistently outperform previous open-source models across five representative mathematical reasoning datasets, achieving state-of-the-art performance. In particular, MathGenieLM-InternLM2 achieves an accuracy of 87.7% on GSM8K and 55.7% on MATH, securing the best overall score among open-source language models.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 33 references that are in the map. First posted on arXiv 26 Feb 2024.

Builds on

17 of 33 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Llemma: An Open Language Model For Mathematics16 Oct
  2. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning5 Oct
  3. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving29 Sep
  4. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct18 Aug
  5. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification15 Aug

Other references · 12

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  8. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  9. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  10. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  11. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  12. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 23; 2025: 34; 2026: 15.

72 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 265
  3. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
  4. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 160
  5. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoningcited by 90
  6. 2024Step-level Value Preference Optimization for Mathematical Reasoningcited by 71
  7. 2024OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modelingcited by 66
  8. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  9. 2025MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoningcited by 45
  10. 2025TreeRPO: Tree Relative Policy Optimizationcited by 45
  11. 2024JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Modelscited by 44
  12. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 40
38 more
  1. 2024MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Codecited by 39
  2. 2024AI-Assisted Generation of Difficult Math Questionscited by 37
  3. 2024Step-Controlled DPO: Leveraging Stepwise Error for Enhanced Mathematical Reasoningcited by 35
  4. 2024SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correctioncited by 32
  5. 2024Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpuscited by 29
  6. 2024Unleashing LLM Reasoning Capability via Scalable Question Synthesis from Scratchcited by 24
  7. 2025SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugscited by 22
  8. 2025LEMMA: Learning from Errors for MatheMatical Advancement in LLMscited by 20
  9. 2024Markov Chain of Thought for Efficient Mathematical Reasoningcited by 19
  10. 2025BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretrainingcited by 19
  11. 2025A Survey on LLM Mid-Trainingcited by 18
  12. 2024Describe-then-Reason: Improving Multimodal Mathematical Reasoning through Visual Comprehension Trainingcited by 17
  13. 2025Let's Verify Math Questions Step by Stepcited by 17
  14. 2025Superficial Self-Improved Reasoners Benefit from Model Mergingcited by 16
  15. 2025Cultural Learning-Based Culture Adaptation of Language Modelscited by 16
  16. 2026Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gaincited by 16
  17. 2024Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoningcited by 15
  18. 2024LogicPro: Improving Complex Logical Reasoning via Program-Guided Learningcited by 14
  19. 2025Enhancing Reasoning through Process Supervision with Monte Carlo Tree Searchcited by 14
  20. 2025Theorem Prover as a Judge for Synthetic Data Generationcited by 13
  21. 2024Understanding Synthetic Context Extension via Retrieval Headscited by 10
  22. 2025Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoningcited by 10
  23. 2025Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesiscited by 8
  24. 2024System-2 Mathematical Reasoning via Enriched Instruction Tuningcited by 7
  25. 2025Does Learning Mathematical Problem-Solving Generalize to Broader Reasoning?cited by 7
  26. 2025AgenticMath: Enhancing LLM Reasoning via Agentic-based Math Data Generationcited by 7
  27. 2024SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Modelscited by 6
  28. 2025Reasoning-OCR: Can Large Multimodal Models Solve Complex Logical Reasoning Problems from OCR Cues?cited by 6
  29. 2025ScaleDiff: Scaling Difficult Problems for Advanced Mathematical Reasoningcited by 6
  30. 2025The KoLMogorov Test: Compression by Code Generationcited by 5
  31. 2026Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Modelscited by 5
  32. 2025ReFeed: Multi-dimensional Summarization Refinement with Reflective Reasoning on Feedbackcited by 4
  33. 2025gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttlingcited by 4
  34. 2025Alignment with Fill-In-the-Middle for Enhancing Code Generationcited by 4
  35. 2024Mars-PO: Multi-Agent Reasoning System Preference Optimizationcited by 3
  36. 2025GiFT: Gibbs Fine-Tuning for Code Generationcited by 3
  37. 2025When Inverse Data Outperforms: Exploring the Pitfalls of Mixed Data in Multi-Stage Fine-Tuningcited by 2
  38. 2026Preventing Curriculum Collapse in Self-Evolving Reasoning Systemscited by 2

50 most cited of 72 All 72 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Reasoning Engine: Specialized Training for Enhanced Mathematical Reasoning
  2. 2025Arrows of Math Reasoning Data Synthesis for Large Language Models: Diversity, Complexity and Correctness
  3. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
  4. 2025SPARQ: Synthetic Problem Generation for Reasoning via Quality-Diversity Algorithms
  5. 2026More Accurate or More Efficient? Evaluating Locally Deployed Compact Open-Weight Language Models for Mathematical Reasoning
  6. 2025Evaluation of LLMs for mathematical problem solving
  7. 2024Augmenting Math Word Problems via Iterative Question Composing
  8. 2024Exploring Mathematical Extrapolation of Large Language Models with Synthetic Data