# PRM-BAS: Enhancing Multimodal Reasoning through PRM-guided Beam Annealing Search

arXiv: https://arxiv.org/abs/2504.10222 · HTML: https://mlsanity.com/paper/2504.10222/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- First posted on arXiv 14 Apr 2025.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Built on by (most cited first, 12 of 12)

- [A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Models](https://mlsanity.com/paper/2510.08049/) (2025)
- [GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning](https://mlsanity.com/paper/2508.04088/) (2025)
- [THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning](https://mlsanity.com/paper/2509.13761/) (2025)
- [CoLD: Counterfactually-Guided Length Debiasing for Process Reward Models in Mathematical Reasoning](https://mlsanity.com/paper/2507.15698/) (2025)
- [PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment](https://mlsanity.com/paper/2603.06652/) (2026)
- [Self-Improvement of Large Language Models: A Technical Overview and Future Outlook](https://mlsanity.com/paper/2603.25681/) (2026)
- [Accelerating LLM Reasoning via Early Rejection with Partial Reward Modeling](https://mlsanity.com/paper/2508.01969/) (2025)
- [Training Data Efficiency in Multimodal Process Reward Models](https://mlsanity.com/paper/2602.04145/) (2026)
- [VeriBound: PAC-Bayesian Generalization Bounds for Process Reward Models Trained with Formal Verification Tools](https://mlsanity.com/paper/2606.20740/) (2026)
- [Test-Time Scaling for Small VLMs on Multilingual Visual MCQ](https://mlsanity.com/paper/2607.09438/) (2026)
- [Process Rewards with Learned Reliability](https://mlsanity.com/paper/2605.15529/) (2026)
- [Mitigating Over-Optimization in PRM-Guided Search in Mathematical Reasoning by Optimizing the Guide](https://mlsanity.com/paper/2608.30051/) (2026)

## Abstract (arXiv)

Recent work increasingly focuses on improving the reasoning capabilities of Multimodal Large Language Models (MLLMs). Among existing methods, Process Reward Models (PRMs) stand out for offering dense, step-wise supervision to guide intermediate reasoning. However, how to effectively integrate PRMs into search strategies remains an open question. In this paper, we introduce PRM-BAS (PRM-Guided Beam Annealing Search), a lightweight approach for PRM-guided reasoning that dynamically adjusts beam size -- starting with a broader search space and gradually narrowing it as contextual information accumulates, thereby balancing performance and efficiency. We further propose a unified framework for data construction and PRM training. Specifically, we construct the PRM-BAS-300k dataset by selecting 300k questions from existing datasets and performing rollouts at each step to estimate the probability of reaching a correct final answer. The PRM is then trained using a combination of value loss for absolute action quality and rank loss for relative action quality. Extensive experiments on challenging multimodal reasoning benchmarks demonstrate that PRM-BAS significantly improves reasoning performance while maintaining low computational cost. Moreover, it generalizes well across different model scales and architectures, showcasing strong robustness and plug-and-play capability.