mlsanityData to 7 Oct 2026

PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models

arXiv 2501.03124 ↗First posted 6 Jan 2025cs.CL cs.AI cs.LGcited by 98

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Process-level Reward Models (PRMs) are crucial for complex reasoning and decision-making tasks, where each intermediate step plays an important role in the reasoning process. Since language models are prone to various types of errors during the reasoning process, PRMs are required to possess nuanced capabilities for detecting various implicit error types in real-world scenarios. However, current benchmarks primarily focus on step correctness, failing to evaluate PRMs' performance systematically. To address this gap, we introduce PRMBench, a process-level benchmark specifically designed to assess the fine-grained error detection capabilities of PRMs. PRMBench comprises 6,216 carefully designed problems and 83,456 step-level labels, evaluating models across multiple dimensions, including simplicity, soundness, and sensitivity. In our experiments on 15 models, spanning both open-source PRMs and closed-source large language models prompted as critic models, we uncover significant weaknesses in current PRMs. These findings underscore the challenges inherent in process-level evaluation and highlight key directions for future research. We hope PRMBench can be a robust bench for advancing research on PRM evaluation and development.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 53 references that are in the map. First posted on arXiv 6 Jan 2025.

Builds on

30 of 53 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoning9 Dec
  2. 2024On Designing Effective RL Reward at Training Time for LLM Reasoning19 Oct
  3. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations14 Dec
  4. 2023Let's Verify Step by Step31 May
  5. 2022Solving math word problems with process- and outcome-based feedback25 Nov

Other references · 25

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  5. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  7. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  8. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  9. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 175
  10. 2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Stylecited by 168
  11. 2024O1 Replication Journey: A Strategic Progress Report -- Part 1cited by 140
  12. 2024LLM Critics Help Catch LLM Bugscited by 138
13 more
  1. 2024Twin-Merging: Dynamic Integration of Modular Expertise in Model Mergingcited by 114
  2. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoningcited by 90
  3. 2024CriticBench: Benchmarking LLMs for Critique-Correct Reasoningcited by 88
  4. 2024ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLMcited by 73
  5. 2023MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluationcited by 57
  6. 2024Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoningcited by 50
  7. 2024Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklistcited by 48
  8. 2024Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correctioncited by 45
  9. 2024LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedbackcited by 40
  10. 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
  11. 2024MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMscited by 30
  12. 2024Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?cited by 25
  13. 2022Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Changecited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 62; 2026: 36.

98 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  3. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  4. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  5. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
  6. 2025RewardBench 2: Advancing Reward Model Evaluationcited by 136
  7. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalingcited by 135
  8. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoningcited by 134
  9. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  10. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoningcited by 94
  11. 2025Evaluating Step-by-step Reasoning Traces: A Surveycited by 62
  12. 2025SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoningcited by 60
38 more
  1. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  2. 2025Stop Summation: Min-Form Credit Assignment Is All Process Reward Model Needs for Reasoningcited by 49
  3. 2025R-PRM: Reasoning-Driven Process Reward Modelingcited by 38
  4. 2025A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Modelscited by 37
  5. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  6. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 29
  7. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoningcited by 28
  8. 2025Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learningcited by 28
  9. 2026AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agentscited by 28
  10. 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detectioncited by 24
  11. 2025System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcutscited by 23
  12. 2025Shrinking the Generation-Verification Gap with Weak Verifierscited by 21
  13. 2025VLRMBench: A Comprehensive and Challenging Benchmark for Vision-Language Reward Modelscited by 19
  14. 2025Know What You Don't Know: Uncertainty Calibration of Process Reward Modelscited by 19
  15. 2025Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Modelscited by 19
  16. 2025RAFFLES: Reasoning-based Attribution of Faults for LLM Systemscited by 19
  17. 2025Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Mathcited by 19
  18. 2025ViLBench: A Suite for Vision-Language Process Reward Modelingcited by 16
  19. 2025Dynamic and Generalizable Process Reward Modelingcited by 16
  20. 2025Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scalingcited by 15
  21. 2025CodeCriticBench: A Holistic Code Critique Benchmark for Large Language Modelscited by 14
  22. 2025Efficient Process Reward Model Training via Active Learningcited by 14
  23. 2025Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewardscited by 14
  24. 2025Scaling Evaluation-time Compute with Reasoning Models as Evaluatorscited by 13
  25. 2025GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoningcited by 13
  26. 2025Better Process Supervision with Bi-directional Rewarding Signalscited by 12
  27. 2025MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process Errors Identificationcited by 12
  28. 2025A Survey on Feedback-based Multi-step Reasoning for Large Language Models on Mathematicscited by 11
  29. 2025Good Learners Think Their Thinking: Generative PRM Makes Large Reasoning Model More Efficient Math Learnercited by 11
  30. 2025Benchmarking Multimodal CoT Reward Model Stepwise by Visual Programcited by 10
  31. 2025Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Modelscited by 10
  32. 2026Save the Good Prefix: Precise Error Penalization via Process-Supervised RL to Enhance LLM Reasoningcited by 10
  33. 2025Error Typing for Smarter Rewards: Improving Process Reward Models with Error-Aware Hierarchical Supervisioncited by 8
  34. 2026IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluationcited by 8
  35. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 8
  36. 2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Surveycited by 7
  37. 2026PRPO: Aligning Process Reward with Outcome Reward in Policy Optimizationcited by 7
  38. 2026What, Whether and How? Unveiling Process Reward Models for Thinking with Images Reasoningcited by 7

50 most cited of 98 All 98 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026GR-Ben: A General Reasoning Benchmark for Evaluating Process Reward Models
  2. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoning
  3. 2025From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling
  4. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoning
  5. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
  6. 2025Linking Process to Outcome: Conditional Reward Modeling for LLM Reasoning
  7. 2026Process Reward Models Meet Planning: Generating Precise and Scalable Datasets for Step-Level Rewards
  8. 2025GroundedPRM: Tree-Guided and Fidelity-Aware Process Reward Modeling for Step-Level Reasoning