mlsanityData to 14 Oct 2026

DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning

arXiv 2504.11456 ↗First posted 15 Apr 2025cs.CL cs.AIcited by 280

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning (RL) with large language models shows promise in complex reasoning. However, its progress is hindered by the lack of large-scale training data that is sufficiently challenging, contamination-free and verifiable. To this end, we introduce DeepMath-103K, a large-scale mathematical dataset designed with high difficulty (primarily levels 5-9), rigorous decontamination against numerous benchmarks, and verifiable answers for rule-based RL reward. It further includes three distinct R1 solutions adaptable for diverse training paradigms such as supervised fine-tuning (SFT). Spanning a wide range of mathematical topics, DeepMath-103K fosters the development of generalizable and advancing reasoning. Notably, models trained on DeepMath-103K achieve state-of-the-art results on challenging mathematical benchmarks and demonstrate generalization beyond math such as biology, physics and chemistry, underscoring its broad efficacy. Data: https://huggingface.co/datasets/zwhe99/DeepMath-103K.

Key facts. Cited by 280 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 52 of its 68 references that are in the map. First posted on arXiv 15 Apr 2025.

Builds on

52 of 68 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  2. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  3. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  4. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't20 Mar
  5. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs3 Mar
  6. 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models24 Feb
  7. 2025Training Language Models to Reason Efficiently6 Feb
  8. 2025Process Reinforcement through Implicit Rewards3 Feb
  9. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  10. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  11. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations14 Dec

Other references · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  12. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
29 more
  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  2. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  3. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  4. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  5. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  6. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  7. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  8. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  9. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  10. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  11. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
  12. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
  13. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruningcited by 279
  14. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 200
  15. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMscited by 163
  16. 2024MAmmoTH2: Scaling Instructions from the Webcited by 145
  17. 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Datasetcited by 143
  18. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  19. 2024DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solvingcited by 95
  20. 2025An Empirical Study on Eliciting and Improving R1-like Reasoning Modelscited by 78
  21. 2023Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Modelscited by 77
  22. 2024OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AIcited by 71
  23. 2023MathPile: A Billion-Token-Scale Pretraining Corpus for Mathcited by 67
  24. 2024Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Samplingcited by 61
  25. 2024Augmenting Math Word Problems via Iterative Question Composingcited by 59
  26. 2024MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Datacited by 52
  27. 2025MegaMath: Pushing the Limits of Open Math Corporacited by 51
  28. 2023MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoningcited by 45
  29. 2024Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes Oncited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 101; 2026: 179.

280 papers.

Sort:

By area:

  1. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
  2. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipecited by 299
  3. 2026Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolationcited by 184
  4. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  5. 2026Entropy-Aware On-Policy Distillation of Language Modelscited by 175
  6. 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Librarycited by 101
  7. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 96
  8. 2026Scaling Reasoning Efficiently via Relaxed On-Policy Distillationcited by 82
  9. 2025In-the-Flow Agentic System Optimization for Effective Planning and Tool Usecited by 72
  10. 2025Are Reasoning Models More Prone to Hallucination?cited by 64
  11. 2025Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solvingcited by 62
  12. 2025MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoningcited by 59
38 more
  1. 2025Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzlescited by 56
  2. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 56
  3. 2025Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoningcited by 56
  4. 2025The Path Not Taken: RLVR Provably Learns Off the Principalscited by 56
  5. 2025Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVRcited by 54
  6. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 53
  7. 2025OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalizationcited by 50
  8. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 50
  9. 2026Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipecited by 49
  10. 2026SkillOS: Learning Skill Curation for Self-Evolving Agentscited by 49
  11. 2026RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillationcited by 45
  12. 2026SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weightingcited by 43
  13. 2025Diversity-Aware Policy Optimization for Large Language Model Reasoningcited by 41
  14. 2025On Entropy Control in LLM-RL Algorithmscited by 40
  15. 2025RePO: Replay-Enhanced Policy Optimizationcited by 39
  16. 2025StepHint: Multi-level Stepwise Hints Enhance Reinforcement Learning to Reasoncited by 39
  17. 2025Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewardscited by 38
  18. 2025ParaThinker: Native Parallel Thinking as a New Paradigm to Scale LLM Test-time Computecited by 38
  19. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Modelscited by 37
  20. 2025RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Trainingcited by 35
  21. 2026Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Levelcited by 35
  22. 2026Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillationcited by 32
  23. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 30
  24. 2025The Open Proof Corpus: A Large-Scale Study of LLM-Generated Mathematical Proofscited by 30
  25. 2025Stable Reinforcement Learning for Efficient Reasoningcited by 27
  26. 2025Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode Supervisioncited by 26
  27. 2025RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?cited by 23
  28. 2025On Predictability of Reinforcement Learning Dynamics for Large Language Modelscited by 23
  29. 2025Learning to Orchestrate Agents in Natural Language with the Conductorcited by 23
  30. 2025VerifyBench: Benchmarking Reference-based Reward Systems for Large Language Modelscited by 20
  31. 2025Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuningcited by 20
  32. 2025ReForm: Reflective Autoformalization with Prospective Bounded Sequence Optimizationcited by 20
  33. 2025Plan Then Action:High-Level Planning Guidance Reinforcement Learning for LLM Reasoningcited by 19
  34. 2025Trace Length is a Simple Uncertainty Signal in Reasoning Modelscited by 19
  35. 2026Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillationcited by 19
  36. 2025DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Trainingcited by 18
  37. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  38. 2025ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shapingcited by 17

50 most cited of 280 All 280 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning
  2. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example
  3. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  4. 2025Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning
  5. 2025DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
  6. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
  7. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
  8. 2026Understanding Reasoning from Pretraining to Post-Training