mlsanityData to 7 Oct 2026

Miner:Mining Intrinsic Mastery for Data-Efficient RL in Large Reasoning Models

arXiv 2601.04731 ↗First posted 8 Jan 2026cs.AI cs.CLcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Current critic-free RL methods for large reasoning models suffer from severe inefficiency when training on positive homogeneous prompts (where all rollouts are correct), resulting in waste of rollouts due to zero advantage estimates. We introduce a radically simple yet powerful solution to ulineMine ulineintrinsic mastulineery (Miner), that repurposes the policy's intrinsic uncertainty as a self-supervised reward signal, with no external supervision, auxiliary models, or additional inference cost. Our method pioneers two key innovations: (1) a token-level focal credit assignment mechanism that dynamically amplifies gradients on critical uncertain tokens while suppressing overconfident ones, and (2) adaptive advantage calibration to seamlessly integrate intrinsic and verifiable rewards. Evaluated across six reasoning benchmarks on Qwen3-4B and Qwen3-8B base models, Miner achieves state-of-the-art performance among the other four algorithms, yielding up to 4.58 absolute gains in Pass@1 and 6.66 gains in Pass@K compared to GRPO. Comparison with other methods targeted at exploration enhancement further discloses the superiority of the two newly proposed innovations. This demonstrates that latent uncertainty exploitation is both necessary and sufficient for efficient and scalable RL training of reasoning models. Code is available at https://github.com/pixas/Miner.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 57 references that are in the map. First posted on arXiv 8 Jan 2026.

Builds on

30 of 57 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025ICPO: Intrinsic Confidence-Driven Group Relative Preference Optimization for Efficient Reinforcement Learning26 Nov
  2. 2025Unlocking Exploration in RLVR: Uncertainty-aware Advantage Shaping for Deeper Reasoning12 Oct
  3. 2025Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Dense8 Oct
  4. 2025XRPO: Pushing the limits of GRPO with Targeted Exploration and Exploitation8 Oct
  5. 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Models24 Sep
  6. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty22 Jul
  7. 2025Self-Guided Process Reward Optimization with Redefined Step-wise Advantage for Process Reinforcement Learning2 Jul
  8. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling25 Jun
  9. 2025Reasoning with Exploration: An Entropy Perspective17 Jun
  10. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts2 Jun
  11. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models28 May
  12. 2025Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learning18 Apr
5 more
  1. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  3. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs3 Mar
  4. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization4 Jan
  5. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 13

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2025Group Sequence Policy Optimizationcited by 788
  6. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  7. 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphscited by 105
  8. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 52
  9. 2025Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Modelscited by 48
  10. 2025BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clippingcited by 47
  11. 2025GHPO: Adaptive Guidance for Stable and Efficient LLM Reinforcement Learningcited by 33
  12. 2025APRIL: Active Partial Rollouts in Reinforcement Learning to Tame Long-tail Generationcited by 27
1 more
  1. 2025Mid-Training of Large Language Models: A Surveycited by 18

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Rethinking Reasoning Quality in Large Language Models through Enhanced Chain-of-Thought via RL
  2. 2025Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning
  3. 2026Know What You Know: Metacognitive Entropy Calibration for Verifiable RL Reasoning
  4. 2026Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learning
  5. 2026On-Policy Supervised Fine-Tuning for Efficient Reasoning
  6. 2026Step Potential Advantage Estimation: Harnessing Intermediate Confidence and Correctness for Efficient Mathematical Reasoning
  7. 2025TinyV: Reducing False Negatives in Verification Improves RL for LLM Reasoning
  8. 2026Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement