mlsanityData to 14 Oct 2026

Rebellious Student: Reversing Teacher Signals for Reasoning Exploration with Self-Distilled RLVR

arXiv 2605.10781 ↗First posted 11 May 2026cs.LG cs.CLcited by 15

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Self-distillation has emerged as a powerful framework for post-training LLMs, where a teacher conditioned on extra information guides a student without it, both from the same model. While this guidance is useful when the student has failed, on successful rollouts, the same mechanism instead overwrites the student's choices and suppresses it's own reasoning. Therefore, we propose reading the original self-distillation signal in reverse: when the student succeeds along a path the teacher would not have predicted, these tokens reflect its self-driven reasoning. Building on this, we propose RLRT (RLVR with Reversed Teacher), which augments GRPO by reinforcing these tokens on correct rollouts. We interpret this as a new form of exploration in RLVR: not uniform diversity, but valuable exploration grounded in the student's own success. Across base, instruction-tuned, and thinking-tuned Qwen3 checkpoints, RLRT substantially outperforms self-distillation and exploration-based baselines, establishing information asymmetry as a new, principled design axis for RLVR.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 32 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

30 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?25 Mar
  2. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models26 Jan

Other references · 28

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  4. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  5. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  6. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 485
  7. 2026Reinforcement Learning via Self-Distillationcited by 332
  8. 2025Reasoning with Exploration: An Entropy Perspectivecited by 281
  9. 2026Self-Distillation Enables Continual Learningcited by 247
  10. 2026On-Policy Context Distillation for Language Modelscited by 186
  11. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Modelscited by 138
  12. 2026Self-Distilled RLVRcited by 131
16 more
  1. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 129
  2. 2025Outcome-based Exploration for LLM Reasoningcited by 83
  3. 2026Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routingcited by 76
  4. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelscited by 69
  5. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 53
  6. 2025Rethinking Entropy Interventions in RLVR: An Entropy Change Perspectivecited by 52
  7. 2025Diversity-Incentivized Exploration for Versatile Reasoningcited by 28
  8. 2025Revisiting Entropy in Reinforcement Learning for Large Reasoning Modelscited by 21
  9. 2026Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMscited by 20
  10. 2025Improving RL Exploration for LLM Reasoning through Retrospective Replaycited by 19
  11. 2026Exploratory Memory-Augmented LLM Agent via Hybrid On- and Off-Policy Optimizationcited by 19
  12. 2025The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Modelscited by 18
  13. 2025The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic Viewcited by 13
  14. 2025Clip-Low Increases Entropy and Clip-High Decreases Entropy in Reinforcement Learning of Large Language Modelscited by 12
  15. 2025SCOPE-RL: Stable and Quantitative Control of Policy Entropy in RL Post-Trainingcited by 12
  16. 2026DSDR: Dual-Scale Diversity Regularization for Exploration in LLM Reasoningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 15.

15 papers.

Sort:

By area:

  1. 2026RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillationcited by 45
  2. 2026UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillationcited by 12
  3. 2026Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVRcited by 11
  4. 2026Diagnosing and Mitigating Thinking Collapse in On-Policy Self-Distillationcited by 5
  5. 2026CriPO: Enhancing Rubric-based RL via Self-Distillationcited by 5
  6. 2026LLM-as-a-Coach: Experiential Learning for Non-Verifiable Taskscited by 4
  7. 2026Learn from Whoever Is Right: Answer-Verified Multi-Teacher Distillation for Multi-Domain LLMscited by 4
  8. 2026CAST: Non-Privileged Clipped Asymmetric Self-Teaching with Advantage Flipping for GRPOcited by 3
  9. 2026Negative Self-Distillation: Learning to Reason by Avoiding Flawscited by 2
  10. 2026Be My Tutor: On-Policy Co-Distillation for Mutual LLM Improvement via Peer Feedbackcited by 1
  11. 2026On Repulsive and Attractive Teachers: Separating Correctness from Behavior in Self-Distillationcited by 1
  12. 2026FlowBalance: Verifier-Grounded Self-Improvement from On-Policy Reasoning Experiencecited by 0
3 more
  1. 2026Surprising Success, Repeated Failure: Entropy-Guided Credit Assignment for Exploration in LLM Reasoningcited by 0
  2. 2026A Privileged Teacher That Only Scores Cannot Search: Critique-Conditioned On-Policy Self-Distillation for Reasoningcited by 0
  3. 2026RELAY: Selective Self-Distillation with Adaptive Hand-off for Rubric-Based Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning
  2. 2025Harnessing Negative Signals: Reinforcement Distillation from Teacher Data for LLM Reasoning
  3. 2026Tailoring Teaching to Aptitude: Direction-Adaptive Self-Distillation for LLM Reasoning
  4. 2026OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoning
  5. 2026Skill-Conditioned Gated Self-Distillation for LLM Reasoning
  6. 2026Reward-Gated On-Policy Distillation
  7. 2026H2SD: Hybrid Hindsight Self-Distillation
  8. 2026OPSD Compresses What RLVR Teaches: A Post-RL Compaction Stage for Reasoning Models