mlsanityData to 7 Oct 2026

MASPRM: Multi-Agent System Process Reward Model

arXiv 2510.24803 ↗First posted 28 Oct 2025cs.MA cs.AIcited by 1

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Inference-time search over multi-agent systems (MAS) wastes compute when it cannot identify which agent's intermediate message advanced progress. We present the Multi-Agent System Process Reward Model (MASPRM), which scores routed transcripts (ordered sequences of messages between agents) and acts as an inference controller for step-level beam search (SBS) and Monte Carlo Tree Search (MCTS). MASPRM is trained from multi-agent MCTS rollouts labeled only with terminal outcome rewards, without human step-level annotations. We evaluate on GSM8K, MATH, MMLU, and LogiQA. Under matched scorer size and comparable MCTS budget, MASPRM exceeds a size-matched ORM by +2.0 to +3.0 points at 1.5B and +4.1 to +14.5 at 7B across all four benchmarks, with additional scorer-scaling gains over policy likelihood at 7B (avg +13.4 under MCTS). MASPRM also improves ranking quality, reducing Hit@1 to Hit@5 gaps by up to 10.3 points, with the largest gains under stepwise search that uses intermediate decisions. Code: https://github.com/milad1378yz/MASPRM

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 1 of its 35 references that are in the map. First posted on arXiv 28 Oct 2025.

Builds on

1 of 35 references.

References · 1

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026KV-PRM: Efficient Process Reward Modeling via KV-Cache Transfer for Multi-Agent Test-Time Scalingcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2026M2-PALE: A Framework for Explaining Multi-Agent MCTS--Minimax Hybrids via Process Mining and LLMs
  2. 2025Process Reward Models for LLM Agents: Practical Framework and Directions
  3. 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress
  4. 2024Multi-Agent Sampling: Scaling Inference Compute for Data Synthesis with Tree Search-Based Agentic Collaboration
  5. 2026Skill-MAS: Evolving Meta-Skill for Automatic Multi-Agent Systems
  6. 2026Arbor: Tree Search as a Cognition Layer for Autonomous Agents
  7. 2026MAS-ProVe: Understanding the Process Verification of Multi-Agent Systems
  8. 2025Who Gets the Reward & Who Gets the Blame? Evaluation-Aligned Training Signals for Multi-LLM Agents