mlsanityData to 7 Oct 2026

What If Consensus Lies? Selective-Complementary Reinforcement Learning at Test Time

arXiv 2603.19880 ↗First posted 20 Mar 2026cs.LG cs.AIcited by 10

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Test-Time Reinforcement Learning (TTRL) enables Large Language Models (LLMs) to enhance reasoning capabilities on unlabeled test streams by deriving pseudo-rewards from majority voting consensus. However, existing TTRL methods rely exclusively on positive pseudo-labeling strategies. Such reliance becomes vulnerable under challenging scenarios where answer distributions are highly dispersed, resulting in weak consensus that inadvertently reinforces incorrect trajectories as supervision signals. In this paper, we propose SCRL (Selective-Complementary Reinforcement Learning), a robust test-time reinforcement learning framework that effectively mitigates label noise amplification. SCRL develops Selective Positive Pseudo-Labeling, which enforces strict consensus criteria to filter unreliable majorities. Complementarily, SCRL introduces Entropy-Gated Negative Pseudo-Labeling, the first negative supervision mechanism in TTRL, to reliably prune incorrect trajectories based on generation uncertainty. Extensive experiments on multiple reasoning benchmarks demonstrate that SCRL achieves substantial improvements over baselines, while maintaining robust generalization and training stability under constrained rollout budgets. Our code is available at https://github.com/Jasper-Yan/SCRL.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 47 references that are in the map. First posted on arXiv 20 Mar 2026.

Builds on

33 of 47 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025SPINE: Token-Selective Test-Time Reinforcement Learning with Entropy-Band Regularization22 Nov
  2. 2025Don't Waste Mistakes: Leveraging Negative RL-Groups via Confidence Reweighting9 Oct
  3. 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning10 Jun
  4. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning2 Jun
  5. 2025Can Large Reasoning Models Self-Train?27 May
  6. 2025Learning to Reason without External Rewards26 May
  7. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Data6 May
  8. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  9. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization8 Apr
  10. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  11. 2025Process Reinforcement through Implicit Rewards3 Feb
  12. 2024On Designing Effective RL Reward at Training Time for LLM Reasoning19 Oct
2 more
  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  2. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 19

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  6. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  7. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  8. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  9. 2025Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variationcited by 44
  10. 2025ETTRL: Balancing Exploration and Exploitation in LLM Test-Time Reinforcement Learning Via Entropy Mechanismcited by 35
  11. 2025SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Modelscited by 27
  12. 2025Heimdall: test-time scaling on the generative verificationcited by 26
7 more
  1. 2024Mirror-Consistency: Harnessing Inconsistency in Majority Votingcited by 16
  2. 2025Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Rankingcited by 10
  3. 2022Uncertainty Determines the Adequacy of the Mode and the Tractability of Decoding in Sequence-to-Sequence Modelscited by 9
  4. 2025Self-Harmony: Learning to Harmonize Self-Supervision and Self-Play in Test-Time Reinforcement Learningcited by 9
  5. 2025DeepResearch-Slice: Bridging the Retrieval-Utilization Gap via Explicit Text Slicingcited by 7
  6. 2025Rewarding the Journey, Not Just the Destination: A Composite Path and Answer Self-Scoring Reward Mechanism for Test-Time Reinforcement Learningcited by 3
  7. 2025Mission Impossible: Feedback-Guided Dynamic Interactive Planning for Improving Reasoning on LLMscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderationcited by 3
  2. 2026Consensus as Privileged Context for Label-Free Self-Distillationcited by 3
  3. 2026AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?cited by 2
  4. 2026Exploiting Verification-Generation Gap: Test-Time Reinforcement Learning with Confidence-Conditioned Verificationcited by 1
  5. 2026Hi-TTRL: Regulating Consensus with Hints for Test-Time Reinforcement Learningcited by 1
  6. 2026TTPO: Test-Time Policy Optimizationcited by 1
  7. 2026Unsupervised Post-Training of Foundation Models: A Surveycited by 0
  8. 2026Bridging the Confidence Gap: Temperature Scaling for Calibrating Test-Time Prompt Tuningcited by 0
  9. 2026Learning via Self-Consistency for Diffusion-based Video Reasoningcited by 0
  10. 2026Sharpen Before You Adapt: Data-Free Entry-State Sharpening for Test-Time Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025TTRL: Test-Time Reinforcement Learning
  2. 2025Beyond Majority Voting: Towards Fine-grained and More Reliable Reward Signal for Test-Time Reinforcement Learning
  3. 2026Tool Verification for Test-Time Reinforcement Learning
  4. 2026CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution
  5. 2026Dual Consensus: Escaping from Spurious Majority in Unsupervised RLVR via Two-Stage Vote Mechanism
  6. 2026Detecting and Mitigating the Correct-Answer Extinction Window in Test-Time Reinforcement Learning with Majority Voting
  7. 2026Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoning
  8. 2025RoiRL: Efficient, Self-Supervised Reasoning with Offline Iterative Reinforcement Learning