mlsanityData to 14 Oct 2026

Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models

arXiv 2506.06395 ↗First posted 5 Jun 2025cs.CL cs.LGcited by 62

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Large language models (LLMs) excel at reasoning, yet post-training remains critical for aligning their behavior with task goals. Existing reinforcement learning (RL) methods often depend on costly human annotations or external reward models. We propose Reinforcement Learning via Self-Confidence (RLSC), which uses the model's own confidence as reward signals-eliminating the need for labels, preference models, or reward engineering. Applied to Qwen2.5-Math-7B with only 16 samples per question and 10 or 20 training steps, RLSC improves accuracy by +13.4% on AIME2024, +21.2% on MATH500, +21.7% on Minerva Math, +20.8% on Olympiadbench, and +9.7% on AMC23. RLSC provides a simple, scalable post-training method for inference models, requiring only a small number of samples and unlabelled supervision.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 19 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

17 of 19 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Data6 May
  2. 2025TTRL: Test-Time Reinforcement Learning22 Apr
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  4. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  6. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 11

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2023Qwen Technical Reportcited by 3,344
  7. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  8. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  9. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  10. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  11. 2025SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoningcited by 42

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 27; 2026: 35.

62 papers.

Sort:

By area:

  1. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 228
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  3. 2025Self-Rewarding Vision-Language Model via Reasoning Decompositioncited by 97
  4. 2025Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contaminationcited by 82
  5. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 75
  6. 2025No Free Lunch: Rethinking Internal Feedback for LLM Reasoningcited by 37
  7. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 35
  8. 2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 35
  9. 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoningcited by 31
  10. 2025SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Modelscited by 29
  11. 2025Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Modelscited by 23
  12. 2025Revisiting LLM Reasoning via Information Bottleneckcited by 19
38 more
  1. 2025Compute as Teacher: Turning Inference Compute Into Reference-Free Supervisioncited by 18
  2. 2025Semantic-Space Exploration and Exploitation in RLVR for LLM Reasoningcited by 17
  3. 2026On-Policy Self-Distillation without Any Supervisioncited by 17
  4. 2025RESTRAIN: From Spurious Votes to Signals -- Self-Driven RL with Self-Penalizationcited by 15
  5. 2026Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewardscited by 14
  6. 2025Don't Waste Mistakes: Leveraging Negative RL-Groups via Confidence Reweightingcited by 12
  7. 2026CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolutioncited by 12
  8. 2025Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Rankingcited by 10
  9. 2025Better LLM Reasoning via Dual-Playcited by 8
  10. 2025TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM Reasoningcited by 7
  11. 2026Likelihood-Based Reward Designs for General LLM Reasoningcited by 7
  12. 2025LogicReward: Incentivizing LLM Reasoning via Step-Wise Logical Supervisioncited by 6
  13. 2026Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoningcited by 6
  14. 2026LLMs Should Express Uncertainty Explicitlycited by 6
  15. 2025Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patternscited by 5
  16. 2025SERL: Self-Examining Reinforcement Learning on Open-Domaincited by 5
  17. 2026Rethinking the Role of Entropy in Optimizing Tool-Use Behaviors for Large Language Model Agentscited by 5
  18. 2026Hallucinations Undermine Trust; Metacognition is a Way Forwardcited by 5
  19. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 5
  20. 2025Maximizing Prefix-Confidence at Test-Time Efficiently Improves Mathematical Reasoningcited by 3
  21. 2025Breaking the Self-Confirming Loop: Diagnosing and Mitigating Systemic Reward Bias in Self-Rewarding RLcited by 3
  22. 2025Rewarding the Journey, Not Just the Destination: A Composite Path and Answer Self-Scoring Reward Mechanism for Test-Time Reinforcement Learningcited by 3
  23. 2026DR2Seg: Decomposed Two-Stage Rollouts for Efficient Reasoning Segmentation in Multimodal Large Language Modelscited by 3
  24. 2026PDCR: Perception-Decomposed Confidence Reward for Vision-Language Reasoningcited by 3
  25. 2026How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMscited by 3
  26. 2025Conf-Profile: A Confidence-Driven Reasoning Paradigm for Label-Free User Profilingcited by 2
  27. 2026Back to Basics: Revisiting Exploration in Reinforcement Learning for LLM Reasoning via Generative Probabilitiescited by 2
  28. 2026PowerFlow: Unlocking the Dual Nature of LLMs via Principled Distribution Matchingcited by 2
  29. 2026Can LLMs Learn to Reason Robustly under Noisy Supervision?cited by 2
  30. 2026Negative Self-Distillation: Learning to Reason by Avoiding Flawscited by 2
  31. 2026Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Modelscited by 1
  32. 2026Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivotscited by 1
  33. 2026GeoMin: Data-Efficient Semi-Supervised RLVR via Geometric Distribution Modelingcited by 1
  34. 2026From Sequence to Structure: Relational Uncertainty Propagation for LLM Agentscited by 1
  35. 2026Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RLcited by 1
  36. 2026Learning What to Practice: Diagnosis-Guided Self-Evolution for Language Modelscited by 1
  37. 2025Enhancing LLM Reasoning via Non-Human-Like Reasoning Path Preference Optimizationcited by 0
  38. 2026CPMobius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learningcited by 0

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning
  2. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales
  3. 2026Process Supervision of Confidence Margin for Calibrated LLM Reasoning
  4. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  5. 2025TinyV: Reducing False Negatives in Verification Improves RL for LLM Reasoning
  6. 2026Scaling with Confidence: Calibrating Confidence of LLMs for Adaptive Test Time Scaling
  7. 2026ConSteer-RL: Steering Reasoning Capabilities in Large Language Models via Confidence-Aware Reinforcement Learning
  8. 2026On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models