# When Should the Teacher Move? Temporal Coupling and Stability in Self On-Policy Distillation

arXiv: https://arxiv.org/abs/2606.03532 · HTML: https://mlsanity.com/paper/2606.03532/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 2 Jun 2026.

## Placed in

- Knowledge distillation / [On-policy and self-distillation for LLMs](https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/)

## Builds on (most cited first, 12 of 12)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases](https://mlsanity.com/paper/2306.05301/) (2023)
- [Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe](https://mlsanity.com/paper/2604.13016/) (2026)
- [CRISP: Compressed Reasoning via Iterative Self-Policy Distillation](https://mlsanity.com/paper/2603.05433/) (2026)
- [KVPO: ODE-Native GRPO for Autoregressive Video Alignment via KV Semantic Exploration](https://mlsanity.com/paper/2605.14278/) (2026)
- [RoboStereo: Dual-Tower 4D Embodied World Models for Unified Policy Optimization](https://mlsanity.com/paper/2603.12639/) (2026)

## Built on by (most cited first, 7 of 7)

- [Learning Visual Spatial Planning from Symbolic State via Modality-Gap-Aware Self-Distillation](https://mlsanity.com/paper/2606.06076/) (2026)
- [SR-OPSD: Self-Referenced On-Policy Self-Distillation](https://mlsanity.com/paper/2608.09745/) (2026)
- [One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation](https://mlsanity.com/paper/2608.25936/) (2026)
- [CLOOPD: Closing the Learner Loop in On-Policy Distillation](https://mlsanity.com/paper/2609.24141/) (2026)
- [Disentangling Self-Distillation: Measuring and Modeling Acquisition and Retention](https://mlsanity.com/paper/2609.39494/) (2026)
- [RADC: Risk-Aware Dual Caching for Vision-Language Test-Time Adaptation](https://mlsanity.com/paper/2610.06932/) (2026)
- [ContourVLA: A Closed-Loop Perception-Action Contour Policy for Generalized Referring Expression Segmentation](https://mlsanity.com/paper/2610.12107/) (2026)

## Abstract (arXiv)

Self on-policy distillation trains a student policy against a teacher derived from its own parameter history, yet the teacher's update schedule -- which governs the temporal coupling between teacher and student -- has not been systematically studied as a stability variable. Through a controlled schedule sweep on Qwen3-8B, we establish that isolation periods, defined as complete teacher freezing between updates, are the key structural property enabling stable learning, not teacher age. To characterize these underlying training dynamics, we introduce a diagnostic framework of temporal KL structure, refresh shock, and length-tail risk. This framework further uncovers state-oblivious collapse: optimal short-horizon fixed schedules catastrophically fail under long-horizon training because a clock-driven refresh can copy a transiently drifting student into the teacher in a single, irreversible step. This failure mode is invisible under short-horizon evaluation and mechanistically distinct from EMA's chronic contamination. To address this, we propose Consolidation-Gated Teacher Refresh (CGTR), which preserves isolation periods while gating each refresh on joint evidence of reward improvement and length-tail safety, ensuring every teacher movement responds to genuine student consolidation rather than a clock signal. With a single shared parameter set and no per-dataset retuning, CGTR achieves zero collapse and the best final score on all four tasks (Chemistry, Biology, Physics, ToolUse), self-regulating its refresh frequency to each task's learning dynamics.