mlsanityData to 7 Oct 2026

1.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Training

arXiv 2503.19633 ↗First posted 25 Mar 2025cs.CLcited by 49

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

The AM-DeepSeek-R1-Distilled is a large-scale dataset with thinking traces for general reasoning tasks, composed of high-quality and challenging reasoning problems. These problems are collected from a multitude of open-source datasets, subjected to semantic deduplication and meticulous cleaning to eliminate test set contamination. All responses within the dataset are distilled from reasoning models (predominantly DeepSeek-R1) and have undergone rigorous verification procedures. Mathematical problems are validated by checking against reference answers, code problems are verified using test cases, and other tasks are evaluated with the aid of a reward model. The AM-Distill-Qwen-32B model, which was trained through only simple Supervised Fine-Tuning (SFT) using this batch of data, outperformed the DeepSeek-R1-Distill-Qwen-32B model on four benchmarks: AIME2024, MATH-500, GPQA-Diamond, and LiveCodeBench. Additionally, the AM-Distill-Qwen-72B model surpassed the DeepSeek-R1-Distill-Llama-70B model on all benchmarks as well. We are releasing these 1.4 million problems and their corresponding responses to the research community with the objective of fostering the development of powerful reasoning-oriented Large Language Models (LLMs). The dataset was published in hrefhttps://huggingface.co/datasets/a-m-team/AM-DeepSeek-R1-Distilled-1.4Mhttps://huggingface.co/datasets/a-m-team/AM-DeepSeek-R1-Distilled-1.4M.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 17 of its 29 references that are in the map. First posted on arXiv 25 Mar 2025.

Builds on

17 of 29 references.

References · 17

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  6. 2025s1: Simple test-time scalingcited by 1,311
  7. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  8. 2025LIMO: Less is More for Reasoningcited by 488
  9. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 282
  10. 2024Free Process Rewards without Process Labelscited by 162
  11. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  12. 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 115
5 more
  1. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitationscited by 96
  2. 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 81
  3. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Predictioncited by 54
  4. 2024Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewardscited by 31
  5. 2024Scaling Data Diversity for Fine-Tuning Language Models in Human Alignmentcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 29; 2026: 20.

49 papers.

Sort:

By area:

  1. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 165
  2. 2025Think Only When You Need with Large Hybrid-Reasoning Modelscited by 64
  3. 2025Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operationscited by 46
  4. 2025SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoningcited by 39
  5. 2025Finetuning LLMs for Human Behavior Prediction in Social Science Experimentscited by 35
  6. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 29
  7. 2025AM-Thinking-v1: Advancing the Frontier of Reasoning at 32B Scalecited by 29
  8. 2025Not All Correct Answers Are Equal: Why Your Distillation Source Matterscited by 25
  9. 2025Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode Supervisioncited by 22
  10. 2025Outcome-based Reinforcement Learning to Predict the Futurecited by 21
  11. 2025Not All Tokens Are What You Need In Thinkingcited by 20
  12. 2025ChemDFM-R: A Chemical Reasoning LLM Enhanced with Atomized Chemical Knowledgecited by 20
37 more
  1. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  2. 2025DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Trainingcited by 17
  3. 2025How Difficulty-Aware Staged Reinforcement Learning Enhances LLMs' Reasoning Capabilities: A Preliminary Experimental Studycited by 14
  4. 2025Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Promptingcited by 13
  5. 2026Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoningcited by 13
  6. 2026Train Less, Learn More: Adaptive Efficient Rollout Optimization for Group-Based Reinforcement Learningcited by 13
  7. 2025MiroMind-M1: An Open-Source Advancement in Mathematical Reasoning via Context-Aware Multi-Stage Policy Optimizationcited by 12
  8. 2025CrossWordBench: Evaluating the Reasoning Capabilities of LLMs and LVLMs with Controllable Puzzle Generationcited by 9
  9. 2025Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillationcited by 6
  10. 2025OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Modelscited by 5
  11. 2026Timely Machine: Awareness of Time Makes Test-Time Scaling Agenticcited by 5
  12. 2026Weak-Driven Learning: How Weak Agents make Strong Agents Strongercited by 5
  13. 2025Mitigating Jailbreaks with Intent-Aware LLMscited by 4
  14. 2025StatEval: A Comprehensive Benchmark for Large Language Models in Statisticscited by 4
  15. 2026The Molecular Structure of Thought: Mapping the Topology of Long Chain-of-Thought Reasoningcited by 4
  16. 2025More Than One Teacher: Adaptive Multi-Guidance Policy Optimization for Diverse Explorationcited by 3
  17. 2025CircuitSeer: Mining High-Quality Data by Probing Mathematical Reasoning Circuits in LLMscited by 3
  18. 2025Expanding Reasoning Potential in Foundation Model by Learning Diverse Chains of Thought Patternscited by 2
  19. 2025Rethinking Expert Trajectory Utilization in LLM Post-training for Mathematical Reasoningcited by 2
  20. 2026Echoes as Anchors: Probabilistic Costs and Attention Refocusing in LLM Reasoningcited by 2
  21. 2026Pedagogically-Inspired Data Synthesis for Language Model Knowledge Distillationcited by 2
  22. 2026EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillationcited by 2
  23. 2025The Challenge of Teaching Reasoning to LLMs Without RL or Distillationcited by 1
  24. 2025InfiAlign: A Scalable and Sample-Efficient Framework for Aligning LLMs to Enhance Reasoning Capabilitiescited by 1
  25. 2026Mixing Expert Knowledge: Bring Human Thoughts Back To the Game of Gocited by 1
  26. 2026Asking the Right Questions: Improving Reasoning with Generated Stepping Stonescited by 1
  27. 2026Unlocking Cognitive Capabilities and Analyzing the Perception-Logic Trade-offcited by 1
  28. 2026LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Modelscited by 1
  29. 2026Deliberative Alignment is Deep, but Uncertainty Remains: Inference time safety improvement in reasoning via attribution of unsafe behavior to base modelcited by 1
  30. 2026POINTS-Seeker: An Open Recipe for Multimodal Search Agents with Visual Memory Managementcited by 1
  31. 2025E3-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Modelscited by 0
  32. 2026SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructionscited by 0
  33. 2026POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMscited by 0
  34. 2026GRC: Unifying Reasoning-Driven Generation, Retrieval and Compressioncited by 0
  35. 2026Towards Resource-Efficient LLMs: End-to-End Energy Accounting of Distillation Pipelinescited by 0
  36. 2026Reasoning Fine-Tuning Induces Persistent Latent Policy Statescited by 0
  37. 2026Mitigating Reasoning-Induced Misalignment via Safety-Direction Penaltycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Data
  2. 2025Skywork Open Reasoner 1 Technical Report
  3. 2025OpenThoughts: Data Recipes for Reasoning Models
  4. 2025Quantifying the Capability Boundary of DeepSeek Models: An Application-Driven Performance Analysis
  5. 2025An Empirical Study on Eliciting and Improving R1-like Reasoning Models
  6. 2025Learning to Reason: Training LLMs with GPT-OSS or DeepSeek R1 Reasoning Traces
  7. 2025Token-Hungry, Yet Precise: DeepSeek R1 Highlights the Need for Multi-Step Reasoning Over Speed in MATH
  8. 2025DeepSeek in Healthcare: A Survey of Capabilities, Risks, and Clinical Applications of Open-Source Large Language Models