Data to 7 Oct 2026

AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling

arXiv 2412.15084 ↗First posted 19 Dec 2024cs.CL cs.AI cs.LGcited by 76

On the map: LLM reasoning → Mathematical reasoning

also in 1 more

LLM alignment / RLHF / preference optimisation → Process reward models for reasoning

Abstract (arXiv)

In this paper, we introduce AceMath, a suite of frontier math models that excel in solving complex math problems, along with highly effective reward models capable of evaluating generated solutions and reliably identifying the correct ones. To develop the instruction-tuned math models, we propose a supervised fine-tuning (SFT) process that first achieves competitive performance across general domains, followed by targeted fine-tuning for the math domain using a carefully curated set of prompts and synthetically generated responses. The resulting model, AceMath-72B-Instruct greatly outperforms Qwen2.5-Math-72B-Instruct, GPT-4o and Claude-3.5 Sonnet. To develop math-specialized reward model, we first construct AceMath-RewardBench, a comprehensive and robust benchmark for evaluating math reward models across diverse problems and difficulty levels. After that, we present a systematic approach to build our math reward models. The resulting model, AceMath-72B-RM, consistently outperforms state-of-the-art reward models. Furthermore, when combining AceMath-72B-Instruct with AceMath-72B-RM, we achieve the highest average rm@8 score across the math reasoning benchmarks. We release model weights, training data, and evaluation benchmarks at: https://research.nvidia.com/labs/adlr/acemath

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 47 of its 80 references that are in the map. First posted on arXiv 19 Dec 2024.

Builds on

47 of 80 references.

Closest ancestors · 7

Same direction, newest first

Other references · 40

2014Adam: A Method for Stochastic Optimizationcited by 36,960
2024The Llama 3 Herd of Modelscited by 14,192
2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
2021Training Verifiers to Solve Math Word Problemscited by 9,743
2021Evaluating Large Language Models Trained on Codecited by 8,053
2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
2023Let's Verify Step by Stepcited by 3,598
2021Program Synthesis with Large Language Modelscited by 3,509
2023Mistral 7Bcited by 3,065
2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
2024Qwen2 Technical Reportcited by 2,076
2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928

28 more

2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
2017Decoupled Weight Decay Regularizationcited by 1,722
2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
2024Qwen2.5-Coder Technical Reportcited by 1,151
2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
2022Solving math word problems with process- and outcome-based feedbackcited by 871
2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
2023Instruction Tuning with GPT-4cited by 676
2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
2024InternLM2 Technical Reportcited by 391
2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 322
2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 277
2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 267
2024Advancing LLM Reasoning Generalists with Preference Treescited by 191
2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Stylecited by 168
2024Nemotron-4 340B Technical Reportcited by 143
2024HelpSteer2-Preference: Complementing Ratings with Preferencescited by 129
2024NVLM: Open Frontier-Class Multimodal LLMscited by 124
2024Self-Generated Critiques Boost Reward Modeling for Language Modelscited by 69
2023MathPile: A Billion-Token-Scale Pretraining Corpus for Mathcited by 67
2024AutoPSV: Automated Process-Supervised Verifiercited by 28
2024Evaluating Robustness of Reward Models for Mathematical Reasoningcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 57; 2026: 19.

76 papers.

Sort:

By area:

2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
2025Process Reinforcement through Implicit Rewardscited by 478
2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 288
2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 159
2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learningcited by 85
2025AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergycited by 75
2025Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Modelscited by 65
2025AgentRxiv: Towards Collaborative Autonomous Researchcited by 64
2025Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitatecited by 61

38 more

2025e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMscited by 56
2025MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoningcited by 56
2025Reinforcement Learning from Human Feedbackcited by 53
2025RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformationcited by 51
2025The Majority is not always right: RL training for solution aggregationcited by 46
2025Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoningcited by 33
2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
2025ICL-Router: In-Context Learned Model Representations for LLM Routingcited by 26
2025OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critiquecited by 24
2025Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Densecited by 19
2025CURE: Critical-Token-Guided Re-Concatenation for Entropy-Collapse Preventioncited by 17
2025RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learningcited by 16
2025Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewardscited by 14
2025Scalpel vs. Hammer: GRPO Amplifies Existing Capabilities, SFT Replaces Themcited by 14
2025GenSelect: A Generative Approach to Best-of-Ncited by 14
2025From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Modelscited by 13
2025φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitationcited by 12
2025Aligning Multimodal LLM with Human Preference: A Surveycited by 12
2025How Post-Training Reshapes LLMs: A Mechanistic View on Knowledge, Truthfulness, Refusal, and Confidencecited by 12
2025Examining False Positives under Inference Scaling for Mathematical Reasoningcited by 11
2025Through the Valley: Path to Effective Long CoT Training for Small Language Modelscited by 10
2025AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Playcited by 10
2025UALM: Unified Audio Language Model for Understanding, Generation and Reasoningcited by 10
2025Artificial Hippocampus Networks for Efficient Long-Context Modelingcited by 9
2025Best-of-Majority: Minimax-Optimal Strategy for Pass@k Inference Scalingcited by 8
2025ReliableMath: Benchmark of Reliable Mathematical Reasoning on Large Language Modelscited by 7
2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Surveycited by 7
2025Instructor-Worker Large Language Model System for Policy Recommendation: a Case Study on Air Quality Analysis of the January 2025 Los Angeles Wildfirescited by 6
2025Beyond Correctness: Evaluating Subjective Writing Preferences Across Culturescited by 6
2025Population-Evolve: a Parallel Sampling and Evolutionary Method for LLM Math Reasoningcited by 6
2026Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluationcited by 5
2026A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Modelscited by 5
2025Digital Twin Buildings: 3D Modeling, GIS Integration, and Visual Descriptions Using Gaussian Splatting, ChatGPT/Deepseek, and Google Maps Platformcited by 4
2025Executable Functional Abstractions: Inferring Generative Programs for Advanced Math Problemscited by 4
2025Making, not Taking, the Best of Ncited by 4
2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
2025Learning When to Stop: Adaptive Latent Reasoning via Reinforcement Learningcited by 4
2026FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use Environmentscited by 4

50 most cited of 76 All 76 (JSON)

Similar, not cited

Close in wording, no citation either way