mlsanityData to 7 Oct 2026

DC-W2S: Dual-Consensus Weak-to-Strong Training for Reliable Process Reward Modeling in Biological Reasoning

arXiv 2603.08095 ↗First posted 9 Mar 2026cs.CL cs.AI cs.LGcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

In scientific reasoning tasks, the veracity of the reasoning process is as critical as the final outcome. While Process Reward Models (PRMs) offer a solution to the coarse-grained supervision problems inherent in Outcome Reward Models (ORMs), their deployment is hindered by the prohibitive cost of obtaining expert-verified step-wise labels. This paper addresses the challenge of training reliable PRMs using abundant but noisy "weak" supervision. We argue that existing Weak-to-Strong Generalization (W2SG) theories lack prescriptive guidelines for selecting high-quality training signals from noisy data. To bridge this gap, we introduce the Dual-Consensus Weak-to-Strong (DC-W2S) framework. By intersecting Self-Consensus (SC) metrics among weak supervisors with Neighborhood-Consensus (NC) metrics in the embedding space, we stratify supervision signals into distinct reliability regimes. We then employ a curriculum of instance-level balanced sampling and label-level reliability-aware masking to guide the training process. We demonstrate that DC-W2S enables the training of robust PRMs for complex reasoning without exhaustive expert annotation, proving that strategic data curation is more effective than indiscriminate training on large-scale noisy datasets.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 56 references that are in the map. First posted on arXiv 9 Mar 2026.

Builds on

29 of 56 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data10 Feb
  2. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoning13 Jan
  3. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervision5 Jun
  4. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations14 Dec
  5. 2023Let's Verify Step by Step31 May
  6. 2022Constitutional AI: Harmlessness from AI Feedback15 Dec

Other references · 23

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  7. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  8. 2025s1: Simple test-time scalingcited by 1,311
  9. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  10. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  11. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
  12. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
11 more
  1. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
  2. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  3. 2023Faithful Chain-of-Thought Reasoningcited by 285
  4. 2023Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systemscited by 118
  5. 2024Theoretical Analysis of Weak-to-Strong Generalizationcited by 46
  6. 2025TxGemma: Efficient and Agentic LLMs for Therapeuticscited by 40
  7. 2025BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Modelcited by 35
  8. 2024Most Influential Subset Selection: Challenges, Promises, and Beyondcited by 31
  9. 2022Training Subset Selection for Weak Supervisioncited by 19
  10. 2025Contextualizing biological perturbation experiments through languagecited by 10
  11. 2025Representations Shape Weak-to-Strong Generalization: Theoretical Insights and Empirical Predictionscited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ScalePRM: Training Process Reward Models by Scaling Verification Compute Without Ground Truth
  2. 2026When Can LLMs Learn to Reason with Weak Supervision?
  3. 2026Towards Robust Process Reward Modeling via Noise-aware Learning
  4. 2026The Weakest Link Tells It All: Outcome-Supervised Process Reward Modeling via Learnable Credit Assignment
  5. 2026SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verification
  6. 2024PSPO*: An Effective Process-supervised Policy Optimization for Reasoning Alignment
  7. 2026The Hidden Bias of Process Reward Models:PRISM for Rewarding the Right Reasoning
  8. 2026Reward Granularity in RLVR: Comparing Process and Outcome Reward Structures for Mathematical Reasoning in Small Language Models