# MASPRM: Multi-Agent System Process Reward Model

arXiv: https://arxiv.org/abs/2510.24803 · HTML: https://mlsanity.com/paper/2510.24803/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 1 of its 35 references that are in the map.
- First posted on arXiv 28 Oct 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)
- LLM alignment / RLHF / preference optimisation / [Process reward models for reasoning](https://mlsanity.com/alignment/process-reward-models-for-reasoning/)

## Builds on (most cited first, 1 of 1)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)

## Built on by (most cited first, 1 of 1)

- [KV-PRM: Efficient Process Reward Modeling via KV-Cache Transfer for Multi-Agent Test-Time Scaling](https://mlsanity.com/paper/2607.09153/) (2026)

## Abstract (arXiv)

Inference-time search over multi-agent systems (MAS) wastes compute when it cannot identify which agent's intermediate message advanced progress. We present the Multi-Agent System Process Reward Model (MASPRM), which scores routed transcripts (ordered sequences of messages between agents) and acts as an inference controller for step-level beam search (SBS) and Monte Carlo Tree Search (MCTS). MASPRM is trained from multi-agent MCTS rollouts labeled only with terminal outcome rewards, without human step-level annotations. We evaluate on GSM8K, MATH, MMLU, and LogiQA. Under matched scorer size and comparable MCTS budget, MASPRM exceeds a size-matched ORM by +2.0 to +3.0 points at 1.5B and +4.1 to +14.5 at 7B across all four benchmarks, with additional scorer-scaling gains over policy likelihood at 7B (avg +13.4 under MCTS). MASPRM also improves ranking quality, reducing Hit@1 to Hit@5 gaps by up to 10.3 points, with the largest gains under stepwise search that uses intermediate decisions. Code: https://github.com/milad1378yz/MASPRM