Data to 14 Oct 2026

Maximizing Confidence Alone Improves Reasoning

arXiv 2505.22660 ↗First posted 28 May 2025cs.LG cs.AIcited by 100

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning (RL) has enabled machine learning models to achieve significant advances in many fields. Most recently, RL has empowered frontier language models to solve challenging math, science, and coding problems. However, central to any RL algorithm is the reward function, and reward engineering is a notoriously difficult problem in any domain. In this paper, we propose RENT: Reinforcement Learning via Entropy Minimization -- a fully unsupervised RL method that requires no external reward or ground-truth answers, and instead uses the model's entropy of its underlying distribution as an intrinsic reward. We find that by reinforcing the chains of thought that yield high model confidence on its generated answers, the model improves its reasoning ability. In our experiments, we showcase these improvements on an extensive suite of commonly-used reasoning benchmarks, including GSM8K, MATH500, AMC, AIME, and GPQA, and models of varying sizes from the Qwen, Mistral, and Llama families. The generality of our unsupervised learning method lends itself to applicability in a wide range of domains where external supervision is unavailable.

Key facts. Cited by 100 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 52 of its 61 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

52 of 61 references.

Closest ancestors · 6

Same direction, newest first

2025Learning to Reason without External Rewards26 May
2025TTRL: Test-Time Reinforcement Learning22 Apr
2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
2023Let's Verify Step by Step31 May
2022Solving math word problems with process- and outcome-based feedback25 Nov
2017Proximal Policy Optimization Algorithms20 Jul

Other references · 46

2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
2021Training Verifiers to Solve Math Word Problemscited by 10,109
2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
2017Deep reinforcement learning from human preferencescited by 3,912
2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
2022Language Models (Mostly) Know What They Knowcited by 1,856
2022Competition-Level Code Generation with AlphaCodecited by 1,462
2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194

34 more

2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
2018Exploration by Random Network Distillationcited by 993
2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
2023Active Retrieval Augmented Generationcited by 516
2020Improving robustness against common corruptions by covariate shift adaptationcited by 431
2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
2019Semi-supervised Domain Adaptation via Minimax Entropycited by 335
2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
2020Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shiftcited by 236
2018A DIRT-T Approach to Unsupervised Domain Adaptationcited by 215
2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 206
2021Reinforcement Learning with Prototypical Representationscited by 179
2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
2017Deep Learning is Robust to Massive Label Noisecited by 165
2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 165
2016Correlation Alignment for Unsupervised Domain Adaptationcited by 147
2024Language Model Cascades: Token-level uncertainty and beyondcited by 121
2023Calibrated Language Models Must Hallucinatecited by 115
2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 114
2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 105
2024On Verbalized Confidence Scores for LLMscited by 83
2017AutoDIAL: Automatic DomaIn Alignment Layerscited by 75
2025Reasoning Models Better Express Their Confidencecited by 65
2024Calibration and Correctness of Language Models for Codecited by 49
2024Towards Uncertainty-Aware Language Agentcited by 32
2024Calibration of Large Language Models on Code Summarizationcited by 12
2023Variational Curriculum Reinforcement Learning for Unsupervised Discovery of Skillscited by 9
2024A Survey of Calibration Process for Black-Box LLMscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 47; 2026: 53.

100 papers.

Sort:

By area:

2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 228
2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 209
2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
2025Reasoning with Sampling: Your Base Model is Smarter Than You Thinkcited by 89
2025Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contaminationcited by 82
2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 75
2025e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMscited by 59
2025Self-Questioning Language Modelscited by 50
2025SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?cited by 47
2025Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variationcited by 45
2026Embarrassingly Simple Self-Distillation Improves Code Generationcited by 40

38 more

2025The Choice of Divergence: A Neglected Key to Mitigating Diversity Collapse in Reinforcement Learning with Verifiable Rewardcited by 37
2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
2025ETTRL: Balancing Exploration and Exploitation in LLM Test-Time Reinforcement Learning Via Entropy Mechanismcited by 36
2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 35
2025RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMscited by 34
2025Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Modelscited by 33
2025CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewardscited by 29
2025Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Rewardcited by 27
2025Nudging the Boundaries of LLM Reasoningcited by 26
2025Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Modelscited by 23
2025Compute as Teacher: Turning Inference Compute Into Reference-Free Supervisioncited by 18
2025From Trial-and-Error to Improvement: A Systematic Analysis of LLM Exploration Mechanisms in RLVRcited by 17
2026On-Policy Self-Distillation without Any Supervisioncited by 17
2025RESTRAIN: From Spurious Votes to Signals -- Self-Driven RL with Self-Penalizationcited by 15
2026Teaching Models to Teach Themselves: Reasoning at the Edge of Learnabilitycited by 13
2025Don't Waste Mistakes: Leveraging Negative RL-Groups via Confidence Reweightingcited by 12
2025Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMscited by 12
2026CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolutioncited by 12
2025Let it Calm: Exploratory Annealed Decoding for Verifiable Reinforcement Learningcited by 10
2025Self-Harmony: Learning to Harmonize Self-Supervision and Self-Play in Test-Time Reinforcement Learningcited by 10
2026EDIS: Diagnosing LLM Reasoning via Entropy Dynamicscited by 10
2026VI-CuRL: Stabilizing Verifier-Independent RL Reasoning via Confidence-Guided Variance Reductioncited by 10
2025Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learningcited by 9
2025Better LLM Reasoning via Dual-Playcited by 8
2025Semantic Voting: A Self-Evaluation-Free Approach for Efficient LLM Self-Improvement on Unverifiable Open-ended Taskscited by 7
2025Achieving Olympiad-Level Geometry Large Language Model Agent via Complexity Boosting Reinforcement Learningcited by 7
2026Likelihood-Based Reward Designs for General LLM Reasoningcited by 7
2026Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigationcited by 7
2025LogicReward: Incentivizing LLM Reasoning via Step-Wise Logical Supervisioncited by 6
2026DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolutioncited by 6
2026When Self-Belief Misleads: Active Label Acquisition for Reinforcement Learning with Verifiable Rewardscited by 6
2026Self-Supervised Visual On-Policy Distillationcited by 6
2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMscited by 5
2025PEAR: Phase Entropy Aware Reward for Efficient Reasoningcited by 5
2025Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Modelscited by 5
2025BrowseConf: Confidence-Guided Test-Time Scaling for Web Agentscited by 5
2025PRInTS: Reward Modeling for Long-Horizon Information Seekingcited by 5
2026MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learningcited by 5

50 most cited of 100 All 100 (JSON)

Similar, not cited

Close in wording, no citation either way