mlsanityData to 14 Oct 2026

Beyond Prompt Count: How Data Shapes Transfer in On-Policy Distillation

arXiv 2609.37377 ↗First posted 29 Sep 2026cs.AIcited by 0

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

On-policy distillation (OPD) trains students using teacher feedback on their own sampled responses, yet how prompt choice shapes transfer across teacher-student pairs remains poorly understood. We systematically study prompt quantity, source, and selection across RL- and SFT-continuation pairs and cross-model settings. We find that OPD can be highly prompt-efficient: a few prompts can approach large-pool performance, with four DAPO prompts matching the observed mathematics score of 3,840 DeepMath prompts. However, prompt utility is relational rather than intrinsic: changing only the teacher can reverse the relative effectiveness of mathematics and code prompts. To characterize these transfer differences, we analyze parameter and functional changes across prompt supports and model pairs. Functional alignment with the teacher varies across supports and target tasks; in continuation pairs, teacher-aligned prediction changes can coexist with weak parameter alignment. Continued OPD on effective supports can restore performance after unfavorable transfer. Finally, targeted selection does not consistently outperform uniform random sampling, and filtering out a source that performs poorly alone yields no consistent gain across three paired support draws. Overall, our results distinguish prompt efficiency from prompt interchangeability and show that effective data choice depends on the teacher-student pair and target capability, with random sampling providing a competitive baseline in the studied settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 33 of its 40 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

33 of 40 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Rethinking On-Policy Distillation of Large Language Models II: One Training Example3 Sep
  2. 2026Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models17 Aug
  3. 2026STEPS: Selective On-Policy Self-Distillation for Reasoning11 May
  4. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe14 Apr
  5. 2026Scaling Reasoning Efficiently via Relaxed On-Policy Distillation11 Mar

Other references · 28

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  7. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  8. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  9. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  10. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  11. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  12. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
16 more
  1. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  2. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
  3. 2025Process Reinforcement through Implicit Rewardscited by 504
  4. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  5. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 280
  6. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 194
  7. 2025Generalizing Verifiable Instruction Followingcited by 194
  8. 2026MiMo-V2-Flash Technical Reportcited by 191
  9. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  10. 2025DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMscited by 84
  11. 2020Autoregressive Knowledge Distillation through Imitation Learningcited by 74
  12. 2025JustRL: Scaling a 1.5B LLM with a Simple RL Recipecited by 64
  13. 2025MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoningcited by 59
  14. 2026KAT-Coder-V2 Technical Reportcited by 8
  15. 2026Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigmscited by 4
  16. 2025HY-MT1.5 Technical Reportcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Teaching a Moving Student: Rethinking the Curriculum of On-Policy Distillation
  2. 2026ReOrder-OPD:Reliability-Aware Prompt Ordering for On-Policy Distillation
  3. 2026Demystifying On-Policy Distillation: Where Does the Teacher's Advantage Live?
  4. 2026Understanding Off- vs On-Policy Distillation: A Tale of Distinct Training Objectives
  5. 2026When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
  6. 2026Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts
  7. 2026Reward-Aligned Reweighting for On-Policy Distillation
  8. 2026DuoOPD: Learning from Joint Teacher-Student Outcomes for Multi-Task On-Policy Distillation