mlsanityData to 14 Oct 2026

OPMD: On-Policy Mechanism Distillation Beyond Output Supervision

arXiv 2610.14022 ↗First posted 29 Sep 2026cs.LG cs.AIcited by 0

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

On-policy distillation (OPD) improves language-model by supervising a student on prefixes sampled from its own policy, but its supervision remains confined to the output space. Consequently, OPD transfers what a teacher predicts without supervising the intermediate computation that produces those predictions. Simply distilling intermediate representations like representation matching is insufficient since teacher and student representations may differ geometrically, and even well-aligned hidden states need not be usable by the student's subsequent computation. We introduce On-Policy Mechanism Distillation (OPMD), which transfers teacher-derived intermediate computation while explicitly accounting for its compatibility with the student. OPMD first constructs a student-compatible teacher that transforms teacher representations into intermediate states usable by the student's downstream computation, then it internalizes these states from on-policy trajectories. OPMD achieves the highest average accuracy among the evaluated distillation baselines in all three teacher--student settings, and generalizes to multi-teacher distillation and cross-family supervised fine-tuning. Further analyses show that mechanism supervision yields more effective reasoning and meaningfully alters the student's intermediate computation and solution strategies, supporting that it transfers information beyond output supervision alone. Together, these results establish mechanism-level supervision as an effective way to move on-policy distillation beyond output supervision, enabling students to learn inner mechanisms they can use and internalize.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 26 of its 29 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

26 of 29 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026PHF: Privileged Hidden Flow for On-Policy Self-Distillation28 Jun
  2. 2026TIP: Token Importance in On-Policy Distillation15 Apr
  3. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe14 Apr
  4. 2026On-Policy Context Distillation for Language Models12 Feb
  5. 2026Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation12 Feb
  6. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models26 Jan

Other references · 20

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  6. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  7. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  8. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  9. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  10. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  11. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  12. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 508
8 more
  1. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 297
  2. 2020MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformerscited by 245
  3. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  4. 2024How to use and interpret activation patchingcited by 214
  5. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 149
  6. 2022Less is More: Task-aware Layer-wise Distillation for Language Model Compressioncited by 93
  7. 2025Activation Space Interventions Can Be Transferred Between Large Language Modelscited by 19
  8. 2026Extremely Sparse Supervision Incentivizes Reasoning Abilitycited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes
  2. 2026Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe
  3. 2026OPRD: On-Policy Representation Distillation
  4. 2026Breaking the Tokenizer Barrier: On-Policy Distillation across Model Families
  5. 2026Fast and Effective On-policy Distillation from Reasoning Prefixes
  6. 2026Train Ahead, Distill Back: Bootstrapping On-Policy Self-Distillation for Large Language Models
  7. 2026Divergence-Routed On-Policy Distillation for Language Models
  8. 2026Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillation