# Process reward models for reasoning

Area: [LLM alignment / RLHF / preference optimisation](https://mlsanity.com/alignment/index.md). HTML: https://mlsanity.com/alignment/process-reward-models-for-reasoning/

## Key facts

- Process reward models for reasoning: 576 papers on arXiv since 2021 (data through 14 Oct 2026).
- 153 of its 576 mapped papers (27%) cite “Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations” (2023, arXiv 2312.08935).
- 98 of its 576 mapped papers (17%) cite “The Lessons of Developing Process Reward Models in Mathematical Reasoning” (2025, arXiv 2501.07301).
- 20 new papers 17 Sep–14 Oct.

## Most cited here

- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://mlsanity.com/paper/2312.08935/) (2023), cited by 153 papers here
- [The Lessons of Developing Process Reward Models in Mathematical Reasoning](https://mlsanity.com/paper/2501.07301/) (2025), cited by 98 papers here
- [Solving math word problems with process- and outcome-based feedback](https://mlsanity.com/paper/2211.14275/) (2022), cited by 86 papers here
- [Generative Verifiers: Reward Modeling as Next-Token Prediction](https://mlsanity.com/paper/2408.15240/) (2024), cited by 55 papers here
- [Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning](https://mlsanity.com/paper/2410.08146/) (2024), cited by 55 papers here
- [Process Reinforcement through Implicit Rewards](https://mlsanity.com/paper/2502.01456/) (2025), cited by 48 papers here
- [ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search](https://mlsanity.com/paper/2406.03816/) (2024), cited by 45 papers here
- [Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs](https://mlsanity.com/paper/2406.18629/) (2024), cited by 35 papers here
- [Process Reward Models That Think](https://mlsanity.com/paper/2504.16828/) (2025), cited by 35 papers here
- [GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning](https://mlsanity.com/paper/2504.00891/) (2025), cited by 33 papers here

## New (17 Sep–14 Oct)

- [Decoupled Marginal Sharpening for Training-Free Inference-Time Scaling](https://mlsanity.com/paper/2610.20483/) (2026-10-14)
- [Lifted State Policy Optimization: Internalizing Self-Correction through Transition Geometry](https://mlsanity.com/paper/2610.20350/) (2026-10-14)
- [Tail Competition Explains Scaling in Best-of-N Sampling for Verifiable Problems](https://mlsanity.com/paper/2610.19362/) (2026-10-14)
- [EPOCH: Coverage-Aware Evolution with Experiential Preference Optimization for LLM-based Automated Heuristic Design](https://mlsanity.com/paper/2610.17379/) (2026-10-12)
- [BoT-GRPO: Efficient Process-Reward RL for Reasoning via Bag-of-Token Aggregation](https://mlsanity.com/paper/2610.09804/) (2026-10-07)
- [Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression](https://mlsanity.com/paper/2610.09788/) (2026-10-07)
- [Reinforcement Learning for Hierarchical Reasoning Rewards: Minimax-Optimal Rates with Transformers](https://mlsanity.com/paper/2610.08561/) (2026-10-06)
- [SIGMA: Self-Improving Alignment Generalization from a Model Spec](https://mlsanity.com/paper/2610.07935/) (2026-10-06)
- [Learning Process Rewards via Reasoning State Propagation](https://mlsanity.com/paper/2609.39220/) (2026-09-30)
- [LexReward: A Taxonomy-Driven Reward Framework for Legal Language Models](https://mlsanity.com/paper/2609.39071/) (2026-09-30)
- [SemOPT: Fixing Semantic Errors in LLM-based Optimization Modeling via Reward-Guided Search](https://mlsanity.com/paper/2609.37361/) (2026-09-29)
- [Inducing Process Supervision from Outcome-Only Reinforcement Learning](https://mlsanity.com/paper/2609.36641/) (2026-09-29)
- [FastGuide: Accelerating Reward Guidance for Diffusion Large Language Models](https://mlsanity.com/paper/2609.36202/) (2026-09-28)
- [Why Deterministic PRM Guidance Underperforms in Discrete Diffusion Reasoning](https://mlsanity.com/paper/2609.35472/) (2026-09-28)
- [ORPG: Reconciling Multiple Reward Objectives through Objective-wise Policy Gradients](https://mlsanity.com/paper/2609.34985/) (2026-09-28)
- [Agentic Multi-Turn Reasoning: A Fairness Approach](https://mlsanity.com/paper/2609.33323/) (2026-09-27)
- [From Trajectories to Grounded Preferences: Process Preference Synthesis via Interaction Element Graphs for Web PRMs](https://mlsanity.com/paper/2609.32351/) (2026-09-26)
- [ScopeIF: Improving Scope-Aware Precise Instruction-Following in Large Language Models via Graded Reward Modeling](https://mlsanity.com/paper/2609.32189/) (2026-09-26)
- [Rufus-Air: An Open LLM Post-Training Recipe](https://mlsanity.com/paper/2609.29421/) (2026-09-24)
- [FLARE: A Full-Lifecycle Dense Supervision Paradigm for Long-Horizon Coding Agents via Generative Reward Model](https://mlsanity.com/paper/2609.23808/) (2026-09-20)