mlsanityData to 7 Oct 2026

InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning

arXiv 2402.06332 ↗First posted 9 Feb 2024cs.CLcited by 117

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

The math abilities of large language models can represent their abstract reasoning ability. In this paper, we introduce and open-source our math reasoning LLMs InternLM-Math which is continue pre-trained from InternLM2. We unify chain-of-thought reasoning, reward modeling, formal reasoning, data augmentation, and code interpreter in a unified seq2seq format and supervise our model to be a versatile math reasoner, verifier, prover, and augmenter. These abilities can be used to develop the next math LLMs or self-iteration. InternLM-Math obtains open-sourced state-of-the-art performance under the setting of in-context learning, supervised fine-tuning, and code-assisted reasoning in various informal and formal benchmarks including GSM8K, MATH, Hungary math exam, MathBench-ZH, and MiniF2F. Our pre-trained model achieves 30.3 on the MiniF2F test set without fine-tuning. We further explore how to use LEAN to solve math problems and study its performance under the setting of multi-task learning which shows the possibility of using LEAN as a unified platform for solving and proving in math. Our models, codes, and data are released at urlhttps://github.com/InternLM/InternLM-Math.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 43 of its 60 references that are in the map. First posted on arXiv 9 Feb 2024.

Builds on

43 of 60 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models5 Feb
  2. 2024Augmenting Math Word Problems via Iterative Question Composing17 Jan
  3. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text10 Oct
  4. 2023MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning9 Oct
  5. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning5 Oct
  6. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification15 Aug
  7. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Models3 Aug
  8. 2022Solving Math Word Problems via Cooperative Reasoning induced Language Models28 Oct
  9. 2022Solving Quantitative Reasoning Problems with Language Models29 Jun

Other references · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2023Let's Verify Step by Stepcited by 3,598
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  9. 2023Mistral 7Bcited by 3,065
  10. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  11. 2024Mixtral of Expertscited by 1,785
  12. 2023PaLM 2 Technical Reportcited by 1,154
22 more
  1. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  2. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  3. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  4. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  5. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  6. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  7. 2022Galactica: A Large Language Model for Sciencecited by 736
  8. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  9. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  10. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  11. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 414
  12. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 380
  13. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  14. 2022HyperTree Proof Search for Neural Theorem Provingcited by 192
  15. 2023Implicit Chain of Thought Reasoning via Knowledge Distillationcited by 165
  16. 2021Proof Artifact Co-training for Theorem Proving with Language Modelscited by 130
  17. 2019The Lean mathematical librarycited by 127
  18. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  19. 2023GPT Can Solve Mathematical Problems Without a Calculatorcited by 68
  20. 2023MathPile: A Billion-Token-Scale Pretraining Corpus for Mathcited by 67
  21. 2023LLMSTEP: LLM proofstep suggestions in Leancited by 32
  22. 2024Unearthing Large Scale Domain-Specific Knowledge from Public Corporacited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 41; 2025: 63; 2026: 13.

117 papers.

Sort:

By area:

  1. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  2. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  3. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  4. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 265
  5. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 255
  6. 2024DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Searchcited by 189
  7. 2024MAmmoTH2: Scaling Instructions from the Webcited by 143
  8. 2024Rho-1: Not All Tokens Are What You Needcited by 132
  9. 2025Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Provingcited by 126
  10. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  11. 2025Q-Insight: Understanding Image Quality via Visual Reinforcement Learningcited by 93
  12. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoningcited by 90
38 more
  1. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuningcited by 86
  2. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 83
  3. 2024Formal Mathematical Reasoning: A New Frontier in AIcited by 80
  4. 2024AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modelingcited by 76
  5. 2024Are Your LLMs Capable of Stable Reasoning?cited by 65
  6. 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Modelcited by 64
  7. 2024A Survey on Deep Learning for Theorem Provingcited by 55
  8. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  9. 2024Herald: A Natural Language Annotated Lean 4 Datasetcited by 54
  10. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Predictioncited by 54
  11. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 52
  12. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  13. 2025Exploring the Limit of Outcome Reward for Learning Mathematical Reasoningcited by 51
  14. 2024Process-Driven Autoformalization in Lean 4cited by 45
  15. 2025MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoningcited by 45
  16. 2024JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Modelscited by 44
  17. 2024LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedbackcited by 40
  18. 2024Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verificationcited by 40
  19. 2025JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agentcited by 39
  20. 2024Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scalecited by 37
  21. 2024InfiMM-WebMath-40B: Advancing Multimodal Pre-Training for Enhanced Mathematical Reasoningcited by 36
  22. 2024Step-Controlled DPO: Leveraging Stepwise Error for Enhanced Mathematical Reasoningcited by 35
  23. 2024miniCTX: Neural Theorem Proving with (Long-)Contextscited by 35
  24. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  25. 2024SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correctioncited by 32
  26. 2025MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusioncited by 30
  27. 2025Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspectivecited by 29
  28. 2025Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluationcited by 29
  29. 2025Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoningcited by 28
  30. 2024Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Criticcited by 27
  31. 2024MathChat: Benchmarking Mathematical Reasoning and Instruction Following in Multi-Turn Interactionscited by 26
  32. 2024Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes Oncited by 24
  33. 2025ATLAS: Autoformalizing Theorems through Lifting, Augmentation, and Synthesis of Datacited by 23
  34. 2024Can LLMs Solve longer Math Word Problems Better?cited by 22
  35. 2024MathGLM-Vision: Solving Mathematical Problems with Multi-Modal Large Language Modelcited by 20
  36. 2025BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoningcited by 19
  37. 2025video-SALMONN-o1: Reasoning-enhanced Audio-visual Large Language Modelcited by 19
  38. 2025SpiritSight Agent: Advanced GUI Agent with One Lookcited by 18

50 most cited of 117 All 117 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Can MLLMs Absorb Math Reasoning Abilities from LLMs as Free Lunch?
  2. 2025Do LLMs Overthink Basic Math Reasoning? Benchmarking the Accuracy-Efficiency Tradeoff in Language Models
  3. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  4. 2024MARIO: MAth Reasoning with code Interpreter Output -- A Reproducible Pipeline
  5. 2026More Accurate or More Efficient? Evaluating Locally Deployed Compact Open-Weight Language Models for Mathematical Reasoning
  6. 2024Enhancing Mathematical Reasoning in LLMs with Background Operators
  7. 2024Benchmarking Large Language Models for Math Reasoning Tasks
  8. 2025UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models