# Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewards

arXiv: https://arxiv.org/abs/2506.11474 · HTML: https://mlsanity.com/paper/2506.11474/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 20 of its 45 references that are in the map.
- First posted on arXiv 13 Jun 2025.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)
- LLM alignment / RLHF / preference optimisation / [Medical LLM alignment](https://mlsanity.com/alignment/medical-llm-alignment/)

## Builds on (most cited first, 20 of 20)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Solving math word problems with process- and outcome-based feedback](https://mlsanity.com/paper/2211.14275/) (2022)
- [Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning](https://mlsanity.com/paper/2410.08146/) (2024)
- [MEDITRON-70B: Scaling Medical Pretraining for Large Language Models](https://mlsanity.com/paper/2311.16079/) (2023)
- [HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs](https://mlsanity.com/paper/2412.18925/) (2024)
- [Making Large Language Models Better Reasoners with Step-Aware Verifier](https://mlsanity.com/paper/2206.02336/) (2022)
- [AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments](https://mlsanity.com/paper/2405.07960/) (2024)
- [Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions](https://mlsanity.com/paper/2411.14405/) (2024)
- [LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!](https://mlsanity.com/paper/2502.07374/) (2025)
- [LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models](https://mlsanity.com/paper/2404.05221/) (2024)
- [TxGemma: Efficient and Agentic LLMs for Therapeutics](https://mlsanity.com/paper/2504.06196/) (2025)
- [MedicalAgentsBench for Complex Medical Reasoning: Comparing Internalized Reasoning Models versus Externalized Agent-based Frameworks](https://mlsanity.com/paper/2503.07459/) (2025)
- [Entropy-Regularized Process Reward Model](https://mlsanity.com/paper/2412.11006/) (2024)
- [MedS3: Towards Medical Slow Thinking with Self-Evolved Soft Dual-sided Process Supervision](https://mlsanity.com/paper/2501.12051/) (2025)
- [Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning](https://mlsanity.com/paper/2502.14361/) (2025)
- [Assessing and Enhancing Large Language Models in Rare Disease Question-answering](https://mlsanity.com/paper/2408.08422/) (2024)
- [MIRIAD: Augmenting LLMs with millions of medical query-response pairs](https://mlsanity.com/paper/2506.06091/) (2025)

## Built on by (most cited first, 24 of 24)

- [Doctor-R1: Mastering Clinical Inquiry with Experiential Agentic Reinforcement Learning](https://mlsanity.com/paper/2510.04284/) (2025)
- [Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Need](https://mlsanity.com/paper/2507.13966/) (2025)
- [Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications](https://mlsanity.com/paper/2508.00669/) (2025)
- [Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation](https://mlsanity.com/paper/2602.09305/) (2026)
- [Guideline-Grounded Evidence Accumulation for High-Stakes Agent Verification](https://mlsanity.com/paper/2603.02798/) (2026)
- [Training and Evaluation of Guideline-Based Medical Reasoning in LLMs](https://mlsanity.com/paper/2512.03838/) (2025)
- [RadThinking: A Dataset for Longitudinal Clinical Reasoning in Radiology](https://mlsanity.com/paper/2605.10761/) (2026)
- [Scaling Medical Reasoning Verification via Tool-Integrated Reinforcement Learning](https://mlsanity.com/paper/2601.20221/) (2026)
- [SSPO: Self-traced Step-wise Preference Optimization for Process Supervision and Reasoning Compression](https://mlsanity.com/paper/2508.12604/) (2025)
- [Quark Medical Alignment: A Holistic Multi-Dimensional Alignment and Collaborative Optimization Paradigm](https://mlsanity.com/paper/2602.11661/) (2026)
- [MedPRMBench: A Fine-grained Benchmark for Process Reward Models in Medical Reasoning](https://mlsanity.com/paper/2604.17282/) (2026)
- [Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning](https://mlsanity.com/paper/2606.31825/) (2026)
- [Incentivizing Cardiologist-Like Reasoning in MLLMs for Interpretable Echocardiographic Diagnosis](https://mlsanity.com/paper/2601.08440/) (2026)
- [Process Reward Agents for Steering Knowledge-Intensive Reasoning](https://mlsanity.com/paper/2604.09482/) (2026)
- [From Answers to Arguments: Toward Trustworthy Clinical Diagnostic Reasoning with Toulmin-Guided Curriculum Goal-Conditioned Learning](https://mlsanity.com/paper/2604.11137/) (2026)
- [MAPLE: Elevating Medical Reasoning from Statistical Consensus to Process-Led Alignment](https://mlsanity.com/paper/2603.08987/) (2026)
- [Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branches](https://mlsanity.com/paper/2606.01145/) (2026)
- [MARD: Mirror-Augmented Reasoning Distillation for Mechanism-Level Drug-Drug Interaction Prediction](https://mlsanity.com/paper/2606.12578/) (2026)
- [Answer Engineering: Local Trajectory Editing for Protocol-Constrained Decision Making in Large Language Models](https://mlsanity.com/paper/2606.21121/) (2026)
- [FaithMed: Training LLMs For Faithful Evidence-Based Medical Reasoning](https://mlsanity.com/paper/2607.01440/) (2026)
- [Rethinking LLM Verification: Evidence Structure, Uncertainty, and Selective Refinement](https://mlsanity.com/paper/2608.10725/) (2026)
- [CARE: Causally-Aligned Reasoning Exploration for Medical Large Language Models](https://mlsanity.com/paper/2608.26147/) (2026)
- [MedAgent-R1: Faithfulness-Aware Reinforcement Learning for Evidence-Grounded Medical Reasoning](https://mlsanity.com/paper/2608.30676/) (2026)
- [Rewarding Reasoning, Not Answers: Fixing and Bounding Test-Time Reinforcement Learning on Medical QA](https://mlsanity.com/paper/2609.16660/) (2026)

## Abstract (arXiv)

Large language models have shown promise in clinical decision making, but current approaches struggle to localize and correct errors at specific steps of the reasoning process. This limitation is critical in medicine, where identifying and addressing reasoning errors is essential for accurate diagnosis and effective patient care. We introduce Med-PRM, a process reward modeling framework that leverages retrieval-augmented generation to verify each reasoning step against established medical knowledge bases. By verifying intermediate reasoning steps with evidence retrieved from clinical guidelines and literature, our model can precisely assess the reasoning quality in a fine-grained manner. Evaluations on five medical QA benchmarks and two open-ended diagnostic tasks demonstrate that Med-PRM achieves state-of-the-art performance, with improving the performance of base models by up to 13.50% using Med-PRM. Moreover, we demonstrate the generality of Med-PRM by integrating it in a plug-and-play fashion with strong policy models such as Meerkat, achieving over 80% accuracy on MedQA for the first time using small-scale models of 8 billion parameters. Our code and data are available at: https://med-prm.github.io/