mlsanityData to 7 Oct 2026

Augmenting Math Word Problems via Iterative Question Composing

arXiv 2401.09003 ↗First posted 17 Jan 2024cs.CL cs.AI cs.LGcited by 58

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Despite the advancements in large language models (LLMs) for mathematical reasoning, solving competition-level math problems remains a significant challenge, especially for open-source LLMs without external tools. We introduce the MMIQC dataset, comprising a mixture of processed web data and synthetic question-response pairs, aimed at enhancing the mathematical reasoning capabilities of base language models. Models fine-tuned on MMIQC consistently surpass their counterparts in performance on the MATH benchmark across various model sizes. Notably, Qwen-72B-MMIQC achieves a 45.0% accuracy, exceeding the previous open-source state-of-the-art by 8.2% and outperforming the initial version GPT-4 released in 2023. Extensive evaluation results on Hungarian high school finals suggest that such improvement can generalize to unseen data. Our ablation study on MMIQC reveals that a large part of the improvement can be attributed to our novel augmentation method, Iterative Question Composing (IQC), which involves iteratively composing new questions from seed problems using an LLM and applying rejection sampling through another LLM.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 27 of its 39 references that are in the map. First posted on arXiv 17 Jan 2024.

Builds on

27 of 39 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023TinyGSM: achieving >80% on GSM8k with small language models14 Dec
  2. 2023Llemma: An Open Language Model For Mathematics16 Oct
  3. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text10 Oct
  4. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving29 Sep
  5. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Models3 Aug
  6. 2022Solving Quantitative Reasoning Problems with Language Models29 Jun

Other references · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2023Let's Verify Step by Stepcited by 3,598
  11. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  12. 2023Qwen Technical Reportcited by 3,273
9 more
  1. 2023Mistral 7Bcited by 3,065
  2. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  5. 2022Large Language Models Can Self-Improvecited by 656
  6. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  7. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  8. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  9. 2022Continual Pre-Training Mitigates Forgetting in Language and Visioncited by 82

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 27; 2025: 28; 2026: 3.

58 papers.

Sort:

By area:

  1. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  2. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 265
  3. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 261
  4. 2024InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoningcited by 117
  5. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  6. 2024DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solvingcited by 92
  7. 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 81
  8. 2024Evaluating Mathematical Reasoning Beyond Accuracycited by 69
  9. 2024OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modelingcited by 66
  10. 2024Building Math Agents with Multi-Turn Iterative Preference Learningcited by 66
  11. 2024Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoningcited by 58
  12. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
38 more
  1. 2024Dynamic Evaluation of Large Language Models by Meta Probing Agentscited by 54
  2. 2024Math-PUMA: Progressive Upward Multimodal Alignment to Enhance Mathematical Reasoningcited by 52
  3. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  4. 2025Exploring the Limit of Outcome Reward for Learning Mathematical Reasoningcited by 51
  5. 2025OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modelingcited by 48
  6. 2024JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Modelscited by 44
  7. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 40
  8. 2025SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoningcited by 39
  9. 2024Entropy-Regularized Process Reward Modelcited by 38
  10. 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervisioncited by 31
  11. 2025MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusioncited by 30
  12. 2024Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Textscited by 26
  13. 2024Unleashing LLM Reasoning Capability via Scalable Question Synthesis from Scratchcited by 24
  14. 2025Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contaminationcited by 22
  15. 2025LEMMA: Learning from Errors for MatheMatical Advancement in LLMscited by 20
  16. 2024TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Treescited by 19
  17. 2024S^3cMath: Spontaneous Step-level Self-correction Makes Large Language Models Better Mathematical Reasonerscited by 17
  18. 2024Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoningcited by 15
  19. 2024Progressive distillation induces an implicit curriculumcited by 15
  20. 2025Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoningcited by 14
  21. 2024Exploring the Mystery of Influential Data for Mathematical Reasoningcited by 13
  22. 2024CodePMP: Scalable Preference Model Pretraining for Large Language Model Reasoningcited by 10
  23. 2025Behavior Injection: Preparing Language Models for Reinforcement Learningcited by 8
  24. 2025Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levelscited by 8
  25. 2024Unconstrained Model Merging for Enhanced LLM Reasoningcited by 7
  26. 2024Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Explorationcited by 7
  27. 2025SPARQ: Synthetic Problem Generation for Reasoning via Quality-Diversity Algorithmscited by 7
  28. 2025MetaLadder: Ascending Mathematical Solution Quality via Analogical-Problem Reasoning Transfercited by 6
  29. 2024Automatically Generating Numerous Context-Driven SFT Data for LLMs across Diverse Granularitycited by 4
  30. 2025A Survey of LLM × DATAcited by 4
  31. 2025Sample-efficient LLM Optimization with Reset Replaycited by 4
  32. 2025Multi-Step Alignment as Markov Games: An Optimistic Online Gradient Descent Approach with Convergence Guaranteescited by 3
  33. 2025Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Trainingcited by 3
  34. 2025Beyond the First Error: Process Reward Models for Reflective Mathematical Reasoningcited by 3
  35. 2025SIGMA: Refining Large Language Model Reasoning via Sibling-Guided Monte Carlo Augmentationcited by 3
  36. 2025Bohdi: Heterogeneous LLM Fusion with Automatic Data Explorationcited by 3
  37. 2025Domain Specific Benchmarks for Evaluating Multimodal Large Language Modelscited by 2
  38. 2025When Inverse Data Outperforms: Exploring the Pitfalls of Mixed Data in Multi-Stage Fine-Tuningcited by 2

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs
  2. 2024SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Models
  3. 2024MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoning
  4. 2025Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluation
  5. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoning
  6. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
  7. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data
  8. 2025AMO-Bench: Large Language Models Still Struggle in High School Math Competitions