mlsanityData to 14 Oct 2026

Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode Supervision

arXiv 2512.15489 ↗First posted 17 Dec 2025cs.AIcited by 26

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

High-quality mathematical reasoning supervision requires diverse reasoning styles, long-form traces, and effective tool integration, capabilities that existing datasets provide only in limited form. Leveraging the multi-mode generation ability of gpt-oss-120b, we introduce Nemotron-Math, a large-scale mathematical reasoning dataset containing 7.5M solution traces across high, medium, and low reasoning modes, each available both with and without Python tool-integrated reasoning (TIR). The dataset integrates 85K curated AoPS problems with 262K community-sourced StackExchange-Math problems, combining structured competition tasks with diverse real-world mathematical queries. We conduct controlled evaluations to assess the dataset quality. Nemotron-Math consistently outperforms the original OpenMathReasoning on matched AoPS problems. Incorporating StackExchange-Math substantially improves robustness and generalization, especially on HLE-Math, while preserving accuracy on math competition benchmarks. To support efficient long-context training, we develop a sequential bucketed strategy that accelerates 128K context-length fine-tuning by 2--3× without significant accuracy loss. Overall, Nemotron-Math enables state-of-the-art performance, including 100% maj@16 accuracy on AIME 2024 and 2025 with Python TIR.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 24 references that are in the map. First posted on arXiv 17 Dec 2025.

Builds on

15 of 24 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset23 Apr
  2. 2024MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code10 Oct
  3. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data2 Oct
  4. 2024Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On11 Jul
  5. 2024MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoning27 Feb
  6. 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset15 Feb
  7. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving29 Sep

Other references · 8

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  4. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  5. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  6. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 280
  7. 2025ToRL: Scaling Tool-Integrated RLcited by 169
  8. 20251.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Trainingcited by 49

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 25.

26 papers.

Sort:

By area:

  1. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  2. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 42
  3. 2026Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 38
  4. 2026Are Full Rollouts Necessary for On-Policy Distillation?cited by 17
  5. 2026Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillationcited by 11
  6. 2026Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Expertscited by 9
  7. 2026Learning, Fast and Slow: Towards LLMs That Adapt Continuallycited by 6
  8. 2026SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoningcited by 6
  9. 2025KV Admission: Learning What to Write for Efficient Long-Context LLM Inferencecited by 4
  10. 2026Data Science and Technology Towards AGI Part I: Tiered Data Managementcited by 4
  11. 2026Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoningcited by 2
  12. 2026OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modelingcited by 2
14 more
  1. 2026Thomson: Continual Learning of Frontier Models for SovereignAIcited by 2
  2. 2026JoyAI-LLM Flash: Advancing Mid-Scale LLMs with Token Efficiencycited by 1
  3. 2026On Subquadratic Architectures: From Applications to Principlescited by 1
  4. 2026Building Multi-Task Agentic LLMs via Two-Phase Distillationcited by 1
  5. 2026On-Policy Delta Distillation for Multilingual Math Reasoningcited by 1
  6. 2026Leum-VL Technical Reportcited by 0
  7. 2026Post-training is (Massive) Supervised Learningcited by 0
  8. 2026NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Researchcited by 0
  9. 2026Diffusion-Guided Uncertainty-Aware Delayed Policy Optimizationcited by 0
  10. 2026CantoneseLLM v2: Reasoning in a Low-Resource Languagecited by 0
  11. 2026Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Taskscited by 0
  12. 2026Flattening Every Memory Peak in Long-Context Mixture-of-Experts Trainingcited by 0
  13. 2026Off-Policy Merging Beats On-Policy Self-Distillation for Continual Learningcited by 0
  14. 2026Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning
  2. 2025Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset
  3. 2026MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval
  4. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning
  5. 2025Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Models
  6. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoning
  7. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  8. 2026Unmasking Reasoning Processes: A Process-aware Benchmark for Evaluating Structural Mathematical Reasoning in LLMs