mlsanityData to 14 Oct 2026

Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning

arXiv 2502.14361 ↗First posted 20 Feb 2025cs.AI cs.IRcited by 23

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 2 more
Abstract (arXiv)

While large language models (LLMs) have significantly advanced mathematical reasoning, Process Reward Models (PRMs) have been developed to evaluate the logical validity of reasoning steps. However, PRMs still struggle with out-of-distribution (OOD) challenges. This paper identifies key OOD issues, including step OOD, caused by differences in reasoning patterns across model types and sizes, and question OOD, which arises from dataset shifts between training data and real-world problems. To address these issues, we introduce Retrieval-Augmented Process Reward Model (RetrievalPRM), a novel framework designed to tackle these OOD issues. By utilizing a two-stage retrieval-enhanced mechanism, RetrievalPRM retrieves semantically similar questions and steps as a warmup, enhancing PRM's ability to evaluate target steps and improving generalization and reasoning consistency across different models and problem types. Our extensive experiments demonstrate that RetrievalPRM outperforms existing baselines across multiple real-world datasets. Our open-source contributions include a retrieval-enhanced dataset, a tuning framework for PRM training, and the RetrievalPRM model, establishing a new standard for PRM performance.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 34 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

28 of 34 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoning9 Dec
  2. 2024Generative Verifiers: Reward Modeling as Next-Token Prediction27 Aug
  3. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervision5 Jun
  4. 2023Let's Verify Step by Step31 May
  5. 2022Solving math word problems with process- and outcome-based feedback25 Nov

Other references · 23

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2024GPT-4o System Cardcited by 4,588
  9. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  11. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  12. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
11 more
  1. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  2. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  3. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  4. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 331
  5. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 284
  6. 2024LLM Critics Help Catch LLM Bugscited by 147
  7. 2024O1 Replication Journey: A Strategic Progress Report -- Part 1cited by 143
  8. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 96
  9. 2024OpenR: An Open Source Framework for Advanced Reasoning with Large Language Modelscited by 94
  10. 2024Evaluating Mathematical Reasoning Beyond Accuracycited by 69
  11. 2024LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedbackcited by 41

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 15; 2026: 8.

23 papers.

Sort:

By area:

  1. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoningcited by 97
  2. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  3. 2025A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Modelscited by 38
  4. 2025Synergizing RAG and Reasoning: A Systematic Reviewcited by 27
  5. 2025StepORLM: A Self-Evolving Framework With Generative Process Supervision For Operations Research Language Modelscited by 26
  6. 2025Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewardscited by 24
  7. 2025Is PRM Necessary? Problem-Solving RL Implicitly Induces PRM Capability in LLMscited by 13
  8. 2026Adaptive Milestone Reward for GUI Agentscited by 12
  9. 2025Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need?cited by 8
  10. 2026Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluationcited by 7
  11. 2025CoLD: Counterfactually-Guided Length Debiasing for Process Reward Models in Mathematical Reasoningcited by 5
  12. 2025NL-Debugging: Exploiting Natural Language as an Intermediate Representation for Code Debuggingcited by 4
11 more
  1. 2025ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Callingcited by 4
  2. 2025MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Modelscited by 3
  3. 2025Arbitrage: Efficient Reasoning via Advantage-Aware Speculationcited by 2
  4. 2025When Distance Distracts: Representation Distance Bias in BT-Loss for Reward Modelscited by 1
  5. 2026Training Data Efficiency in Multimodal Process Reward Modelscited by 1
  6. 2026Process Reward Agents for Steering Knowledge-Intensive Reasoningcited by 1
  7. 2025Fusing Bidirectional Chains of Thought and Reward Mechanisms A Method for Enhancing Question-Answering Capabilities of Large Language Models for Chinese Intangible Cultural Heritagecited by 0
  8. 2026Constraint-Rectified Training for Efficient Chain-of-Thoughtcited by 0
  9. 2026EvoOptiGraph: Weakness-Driven Coevolution via Graph-Based Structural Generation for Optimization Modelingcited by 0
  10. 2026ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scalingcited by 0
  11. 2026Learning Process Rewards via Reasoning State Propagationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoning
  2. 2025R-PRM: Reasoning-Driven Process Reward Modeling
  3. 2025From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling
  4. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoning
  5. 2025Graph-Augmented Reasoning: Evolving Step-by-Step Knowledge Graph Retrieval for LLM Reasoning
  6. 2025ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding
  7. 2025An Efficient and Precise Training Data Construction Framework for Process-supervised Reward Model in Mathematical Reasoning
  8. 2025Limits of PRM-Guided Tree Search for Mathematical Reasoning with LLMs