mlsanityData to 7 Oct 2026

Dynamic and Generalizable Process Reward Modeling

arXiv 2507.17849 ↗First posted 23 Jul 2025cs.CLcited by 16

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Process Reward Models (PRMs) are crucial for guiding Large Language Models (LLMs) in complex scenarios by providing dense reward signals. However, existing PRMs primarily rely on heuristic approaches, which struggle with cross-domain generalization. While LLM-as-judge has been proposed to provide generalized rewards, current research has focused mainly on feedback results, overlooking the meaningful guidance embedded within the text. Additionally, static and coarse-grained evaluation criteria struggle to adapt to complex process supervision. To tackle these challenges, we propose Dynamic and Generalizable Process Reward Modeling (DG-PRM), which features a reward tree to capture and store fine-grained, multi-dimensional reward criteria. DG-PRM dynamically selects reward signals for step-wise reward scoring. To handle multifaceted reward signals, we pioneeringly adopt Pareto dominance estimation to identify discriminative positive and negative pairs. Experimental results show that DG-PRM achieves stunning performance on prevailing benchmarks, significantly boosting model performance across tasks with dense rewards. Further analysis reveals that DG-PRM adapts well to out-of-distribution scenarios, demonstrating exceptional generalizability.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 82 references that are in the map. First posted on arXiv 23 Jul 2025.

Builds on

36 of 82 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Process Reinforcement through Implicit Rewards3 Feb
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  3. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoning13 Jan
  4. 2025PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models6 Jan
  5. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoning9 Dec
  6. 2024Free Process Rewards without Process Labels2 Dec
  7. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs24 Oct
  8. 2024Process Reward Model with Q-Value Rankings15 Oct
  9. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervision5 Jun
  10. 2022Constitutional AI: Harmlessness from AI Feedback15 Dec
  11. 2022Solving math word problems with process- and outcome-based feedback25 Nov

Other references · 25

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  6. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  7. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  8. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  9. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
  10. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  11. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  12. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
13 more
  1. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  2. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  3. 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 232
  4. 2024Generative Reward Modelscited by 148
  5. 2024LLM Critics Help Catch LLM Bugscited by 138
  6. 2023Human-like Summarization Evaluation with ChatGPTcited by 126
  7. 2024Making Text Embedders Few-Shot Learnerscited by 78
  8. 2024Large Language Models for Disease Diagnosis: A Scoping Reviewcited by 51
  9. 2024LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedbackcited by 40
  10. 2024A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learningcited by 29
  11. 2024Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trialscited by 24
  12. 2024A Survey of Neural Code Intelligence: Paradigms, Advances and Beyondcited by 11
  13. 2024Exploring Large Language Models for Climate Forecastingcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 12.

16 papers.

Sort:

By area:

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  2. 2026DGPO: Distribution Guided Policy Optimization for Fine Grained Credit Assignmentcited by 7
  3. 2026PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimizationcited by 5
  4. 2026OpenRCA 2.0: From Outcome Labels to Causal Process Supervisioncited by 5
  5. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
  6. 2026Grounding the Score: Explicit Visual Premise Verification for Reliable Vision-Language Process Reward Modelscited by 4
  7. 2026SCALE: Self-uncertainty Conditioned Adaptive Looking and Execution for Vision-Language-Action Modelscited by 3
  8. 2026Let's Reward Step-by-Step: Step-Aware Contrastive Alignment for Vision-Language Navigation in Continuous Environmentscited by 3
  9. 2026STAMP: Provenance-Guided Credit Assignment for Deep Search Agentscited by 3
  10. 2026From Failure to Feedback: Group Revision Unlocks Hard Cases in Object-Level Groundingcited by 2
  11. 2025ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Modelscited by 1
  12. 2025MUSIC: MUlti-Step Instruction Contrast for Multi-Turn Reward Modelscited by 0
4 more
  1. 2026PARM: Pipeline-Adapted Reward Modelcited by 0
  2. 2026Unsupervised Process Reward Modelscited by 0
  3. 2026How Can Recommendation Feedback Evolve Agent Memory?cited by 0
  4. 2026Learning Process Rewards via Reasoning State Propagationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling
  2. 2025A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Models
  3. 2025R-PRM: Reasoning-Driven Process Reward Modeling
  4. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoning
  5. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
  6. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning
  7. 2025Error Typing for Smarter Rewards: Improving Process Reward Models with Error-Aware Hierarchical Supervision
  8. 2025GroundedPRM: Tree-Guided and Fidelity-Aware Process Reward Modeling for Step-Level Reasoning