mlsanityData to 14 Oct 2026

ContextPRM: Leveraging Contextual Coherence for multi-domain Test-Time Scaling

arXiv 2509.24460 ↗First posted 29 Sep 2025cs.AIcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Process reward models (PRMs) have demonstrated significant efficacy in enhancing the mathematical reasoning capabilities of large language models (LLMs) by leveraging test-time scaling (TTS). However, while most PRMs exhibit substantial gains in mathematical domains, the scarcity of domain-specific training data and knowledge-based learning patterns limits their generalization ability when faced with other domains. To address this limitation, we shift the learning objective from verifying domain-specific knowledge to modeling domain-agnostic logical flow. Centering on contextual coherence between chain-of-thought (CoT) steps, our approach is realized through a novel data annotation and training framework, which enhances the model's generalization capabilities across diverse domains. For instance, our resulting model, ContextPRM, achieves a notable 6.5% average accuracy improvement over the majority voting baseline via weighted majority voting across nine non-mathematical domains in MMLU-Pro, including law, history, and philosophy, significantly surpassing the 2.2% improvement from VersaPRM and 0.5% gains from other mathematics-focused PRMs, demonstrating consistent performance across both mathematical and non-mathematical domains.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 40 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

30 of 40 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Error Typing for Smarter Rewards: Improving Process Reward Models with Error-Aware Hierarchical Supervision26 May
  2. 2025Process Reward Models That Think23 Apr
  3. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning1 Apr
  4. 2025VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data10 Feb
  5. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoning13 Jan
  6. 2025PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models6 Jan
  7. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoning9 Dec
  8. 2024Free Process Rewards without Process Labels2 Dec
  9. 2024Generative Verifiers: Reward Modeling as Next-Token Prediction27 Aug
  10. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations14 Dec
  11. 2023Let's Verify Step by Step31 May
  12. 2022Solving math word problems with process- and outcome-based feedback25 Nov

Other references · 18

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  10. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  11. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  12. 2023Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraintcited by 350
6 more
  1. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
  2. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 290
  3. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 263
  4. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 233
  5. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 96
  6. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 86

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling
  2. 2025Rethinking Reward Models for Multi-Domain Test-Time Scaling
  3. 2025Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoning
  4. 2025R-PRM: Reasoning-Driven Process Reward Modeling
  5. 2025Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning
  6. 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Model
  7. 2026Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning
  8. 2025Accelerating LLM Reasoning via Early Rejection with Partial Reward Modeling