# PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence

arXiv: https://arxiv.org/abs/2603.11178 · HTML: https://mlsanity.com/paper/2603.11178/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 34 of its 46 references that are in the map.
- First posted on arXiv 11 Mar 2026.

## Placed in

- Knowledge distillation / [On-policy and self-distillation for LLMs](https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/)

## Builds on (most cited first, 30 of 34)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://mlsanity.com/paper/2409.12122/) (2024)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Do Deep Nets Really Need to be Deep?](https://mlsanity.com/paper/1312.6184/) (2013)
- [Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models](https://mlsanity.com/paper/2601.18734/) (2026)
- [Not All Samples Are Created Equal: Deep Learning with Importance Sampling](https://mlsanity.com/paper/1803.00942/) (2018)
- [An Empirical Model of Large-Batch Training](https://mlsanity.com/paper/1812.06162/) (2018)
- [Automated Curriculum Learning for Neural Networks](https://mlsanity.com/paper/1704.03003/) (2017)
- [Self-Distillation Enables Continual Learning](https://mlsanity.com/paper/2601.19897/) (2026)
- [On-Policy Context Distillation for Language Models](https://mlsanity.com/paper/2602.12275/) (2026)
- [MiniLLM: On-Policy Distillation of Large Language Models](https://mlsanity.com/paper/2306.08543/) (2023)
- [Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation](https://mlsanity.com/paper/2105.08919/) (2021)
- [Estimating Example Difficulty Using Variance of Gradients](https://mlsanity.com/paper/2008.11600/) (2020)
- [Understanding and Improving Knowledge Distillation](https://mlsanity.com/paper/2002.03532/) (2020)
- [Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models](https://mlsanity.com/paper/2404.02657/) (2024)

## Built on by (most cited first, 28 of 28)

- [A Survey of On-Policy Distillation for Large Language Models](https://mlsanity.com/paper/2604.00626/) (2026)
- [Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation](https://mlsanity.com/paper/2606.02684/) (2026)
- [On the Position Bias of On-Policy Distillation](https://mlsanity.com/paper/2606.22600/) (2026)
- [Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoning](https://mlsanity.com/paper/2605.13255/) (2026)
- [Trajectory-Refined Distillation](https://mlsanity.com/paper/2606.08432/) (2026)
- [Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation](https://mlsanity.com/paper/2605.27115/) (2026)
- [Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information](https://mlsanity.com/paper/2605.11609/) (2026)
- [Rubric-based On-policy Distillation](https://mlsanity.com/paper/2605.07396/) (2026)
- [Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training](https://mlsanity.com/paper/2605.12483/) (2026)
- [Reinforcement Learning from Rich Feedback with Distributional DAgger](https://mlsanity.com/paper/2606.05152/) (2026)
- [Breaking the Tokenizer Barrier: On-Policy Distillation across Model Families](https://mlsanity.com/paper/2606.09456/) (2026)
- [A Formula-Driven Survey and Research Agenda for On-Policy Distillation](https://mlsanity.com/paper/2606.22793/) (2026)
- [DemoPSD: Disagreement-Modulated Policy Self-Distillation](https://mlsanity.com/paper/2607.02502/) (2026)
- [DriftGuard: Safety-Aware Multi-Monitor Detection and Selective Adaptation for Evolving Toxicity Moderation](https://mlsanity.com/paper/2606.28725/) (2026)
- [LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks](https://mlsanity.com/paper/2607.18110/) (2026)
- [Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable AI Access](https://mlsanity.com/paper/2608.08942/) (2026)
- [Restoring the Sweet Spot: Pass-Rate Weighted Self-Distillation for LLM Reasoning](https://mlsanity.com/paper/2605.27765/) (2026)
- [Learning from Consensus and Disagreement: Unsupervised On-Policy Self-Distillation with Minority-Trajectory Contrast](https://mlsanity.com/paper/2608.08764/) (2026)
- [OptiAgent: End-to-End Optimization Modeling via Multi-Agent Iterative Refinement](https://mlsanity.com/paper/2607.05346/) (2026)
- [AdaStop: Cost-Aware Early Stopping for DNN Test Selection](https://mlsanity.com/paper/2607.05461/) (2026)
- [Scope3Trace: Evidence-Based Identification and Extraction of Scope 3 GHG Emissions from Sustainability Reports](https://mlsanity.com/paper/2607.17122/) (2026)
- [Recovering General Capabilities via Uncertainty-Calibrated Multi-Teacher On-Policy Distillation](https://mlsanity.com/paper/2608.26735/) (2026)
- [Aha-Flow Distillation: Flow Markers Matter in LLM Reasoning](https://mlsanity.com/paper/2609.07036/) (2026)
- [Distillation as Probability Transport: Routed On-Policy Distillation](https://mlsanity.com/paper/2609.08337/) (2026)
- [Improving OCR Faithfulness via Gated and Attenuated On-Policy Distillation](https://mlsanity.com/paper/2609.38282/) (2026)
- [Transfer-Stratified On-Policy Distillation for RL-Improved Reasoning Teachers](https://mlsanity.com/paper/2610.05974/) (2026)
- [Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts](https://mlsanity.com/paper/2610.10460/) (2026)
- [PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation](https://mlsanity.com/paper/2610.11167/) (2026)

## Abstract (arXiv)

Standard LLM distillation treats all training problems equally -- wasting compute on problems the student has already mastered or cannot yet solve. We empirically show that this inefficiency has a precise gradient-level signature: the cross-problem gradient signal-to-noise ratio (SNR) follows a bell curve over student pass rate, collapsing at both extremes. We propose PACED, which weights each problem by w(p) = p(1-p) where p is the student's empirical pass rate -- concentrating training on the zone of proximal development. This requires only student rollouts, no architectural changes, and no hyperparameters. We prove the Beta kernel w(p) = pα(1-p)β is the leading-order optimal weight family arising from the SNR boundary-collapse structure, and is minimax-robust under misspecification (worst-case efficiency loss O(δ2)). Across Qwen3, Qwen2.5, and Llama-3 families, PACED sets a new state of the art in our experimental setting on MATH-500, AIME~2024, and AIME~2025, improving over unweighted distillation by up to +8.2 and over the strong AKL baseline by up to +3.6, while reducing forgetting to 1.4% and 0.6% in distillation and self-distillation. A two-stage forward-then-reverse KL schedule pushes gains further to +5.8 over standard forward KL on the hardest benchmark.