Learning from Self-Generated Responses: Reliability--Uncertainty Self-Distillation for Reasoning
On the map: LLM reasoning → Reasoning benchmarks and evaluation also fits: Test-time compute and efficient thinking
also in 2 more
Abstract (arXiv)
Reference solutions and question-specific rubrics are not always available to guide the teacher in self-distillation. A large language model's own solution attempts offer an alternative, but their quality is difficult to assess in open-ended tasks without automatic correctness checks, and even correct attempts can involve uncertain token predictions. We propose Reliability--Uncertainty Self-Distillation (method) to learn from these attempts. The same model serves as teacher and student: the teacher receives the question and completed attempt, and the student matches the teacher's token-level predictions using a question-only prompt. method weights each attempt's distillation loss by additively combining judged reliability, estimated from a self-judge's score and confidence, with student predictive uncertainty. Across eleven reasoning benchmarks, method outperforms all five baselines in average accuracy, reaching 70.47% on Qwen3-4B and 76.69% on Qwen3-8B (8.90 and 2.10 percentage points above the base models). Data and code are available at https://github.com/JP-25/RUSD.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 45 of its 58 references that are in the map. First posted on arXiv 13 Oct 2026.
Builds on
45 of 58 references.
Closest ancestors · 1
Same direction, newest first
Other references · 44
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
- 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
- 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
- 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
32 more
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
- 2016Sequence-Level Knowledge Distillationcited by 801
- 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
- 2022Large Language Models Can Self-Improvecited by 664
- 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
- 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
- 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 326
- 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 262
- 2015Unifying distillation and privileged informationcited by 233
- 2025Learning to Reason without External Rewardscited by 210
- 2026On-Policy Context Distillation for Language Modelscited by 186
- 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 166
- 2022Learning by Distilling Contextcited by 138
- 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 84
- 2026When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoningcited by 30
- 2026Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoningcited by 23
- 2026On-Policy Self-Distillation without Any Supervisioncited by 17
- 2026Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifierscited by 15
- 2026Rethinking Reward Supervision: Rubric-Conditioned Self-Distillationcited by 13
- 2026Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillationcited by 12
- 2026On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversitycited by 10
- 2026Rethinking Privileged Information in On-Policy Self-Distillationcited by 9
- 2024Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Datacited by 8
- 2026PAST: Privileged Adaptation from Complete Student Trajectories for On-Policy Self-Distillationcited by 6
- 2026Consensus as Privileged Context for Label-Free Self-Distillationcited by 5
- 2026LLM-as-a-Coach: Experiential Learning for Non-Verifiable Taskscited by 4
- 2026Small Language Models as Judges for Rubric-Based Reinforcement Learningcited by 2
- 2026Negative Self-Distillation: Learning to Reason by Avoiding Flawscited by 2
- 2026Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoningcited by 1
- 2026Activation-Conditioned Self-Distillationcited by 1
- 2026When Clipping Reverses Correction: Failure Dynamics of Pointwise Forward-KL On-Policy Self-Distillationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Semantic Self-Distillation for Language Model Uncertainty
- 2026Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillation
- 2026Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline
- 2025A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
- 2026LLMs Should Express Uncertainty Explicitly
- 2026Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?
- 2026The Role of Feedback Alignment in Self-Distillation
- 2022Language Models (Mostly) Know What They Know