mlsanityData to 14 Oct 2026

VISTA: Verifier-Informed Student-to-Teacher Adaptation for On-Policy Self-Distillation

arXiv 2608.28306 ↗First posted 28 Aug 2026cs.LG cs.AI cs.CLcited by 0

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

On-policy self-distillation (OPSD) improves reasoning by training a problem-only student on its own rollouts using dense token-level supervision from a privileged teacher that also sees a reference solution. However, standard OPSD treats the teacher distribution as a fixed target along the student's rollout and updates only the student %, although -- even though privileged conditioning does not guarantee that the teacher always provides the most appropriate target for problem-only reasoning. This one-way supervision can therefore misdirect the student when the teacher distribution is misaligned with valid student reasoning. We therefore introduce Verifier-Informed Student-to-Teacher Adaptation (VISTA), which preserves the standard OPSD student update while using outcome-verified rollouts to adapt the teacher toward the student distribution. Within each verified rollout, VISTA further restricts this adaptation to the top-k positions with the largest teacher--student KL divergence. Notably, VISTA reuses the rollout and loss function from standard OPSD, introducing no additional sampling or separate reward objective. Across AIME24, AIME25, and HMMT25 with Qwen3 models at 1.7B, 4B, and 8B, VISTA achieves the highest Avg@12 at every scale, improving over OPSD by 0.6, 0.7, and 2.1 points, respectively. These results demonstrate the value of student supervision from outcome-verified rollouts and highlight student-to-teacher adaptation as a promising direction for OPSD.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 21 of its 33 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

21 of 33 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Diagnosing and Mitigating Thinking Collapse in On-Policy Self-Distillation12 Jul
  2. 2026DemoPSD: Disagreement-Modulated Policy Self-Distillation2 Jul
  3. 2026Purified OPSD: On-Policy Self-Distillation Without Losing How to Think2 Jul
  4. 2026Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information12 May
  5. 2026STEPS: Selective On-Policy Self-Distillation for Reasoning11 May
  6. 2026On-Policy Context Distillation for Language Models12 Feb

Other references · 15

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2020Knowledge Distillation: A Surveycited by 1,603
  7. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  8. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  9. 2017Deep Mutual Learningcited by 628
  10. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
  11. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
  12. 2015Unifying distillation and privileged informationcited by 233
3 more
  1. 2018Large scale distributed neural network training through online distillationcited by 146
  2. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforcecited by 145
  3. 2024Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Samplingcited by 90

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Reward-Gated On-Policy Distillation
  2. 2026DualOPSD: Adaptive Privileged Teachers for On-Policy Self-Distillation
  3. 2026Matching Supervision to the Student's Learning Capacity: A Unified Framework for On-Policy Self-Distillation
  4. 2026DTDD: Divergence-Triggered Dynamic Distillation for Reliable On-Policy Supervision
  5. 2026On-Policy Distillation with Best-of-N Teacher Rollout Selection
  6. 2026PAST: Privileged Adaptation from Complete Student Trajectories for On-Policy Self-Distillation
  7. 2026Recursive Self-Improvement via On-Policy Distillation for Reasoning
  8. 2026Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning