mlsanityData to 14 Oct 2026

Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation

arXiv 2605.27115 ↗First posted 26 May 2026cs.AIcited by 15

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

Domain specialization can improve LLM behavior, but often weakens the general capabilities inherited from the original model. Recent Multi-Teacher On-Policy Distillation (MOPD) pipelines recover model capabilities by supervising student-generated trajectories with teacher feedback, but typically assume teacher-aligned prompt coverage, requiring prompts to match the teachers' training distributions. This assumption is difficult to satisfy when the general teacher is an open-source model whose post-training data are unknown. Instead of attempting to reconstruct this hidden distribution, we study general capability recovery with readily available proxy general prompts. We identify two failure modes of vanilla MOPD in this incomplete-coverage situation: recovery-preservation counteraction from mixing conflicting recovery and preservation gradients, and weak-signal flattening from uniformly averaging samples with unequal correction demand. We propose Counteraction-Aware Multi-Teacher On-Policy Distillation (CaMOPD), which addresses these issues with decoupled alternating training and gap-based sample selection. CaMOPD allocates dedicated updates to general recovery, periodically performs domain-preservation updates, and selects samples with larger averaged token-level teacher-student log-probability gaps to concentrate correction signals. Across role-play dialogue and medical reasoning QA scenarios, CaMOPD outperforms all baselines in general capability recovery while maintaining domain-specific behavior. Gradient coherence analyses further support the intended effect of CaMOPD in producing more coherent correction signals.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 35 of its 44 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

35 of 44 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe14 Apr
  2. 2026SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting12 Apr
  3. 2026Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes26 Mar
  4. 2026Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation19 Mar
  5. 2026PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence11 Mar
  6. 2026Scaling Reasoning Efficiently via Relaxed On-Policy Distillation11 Mar
  7. 2026Fast and Effective On-policy Distillation from Reasoning Prefixes16 Feb

Other references · 28

  1. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  5. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  6. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  7. 2018Experience Replay for Continual Learningcited by 1,095
  8. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  9. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  10. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
  11. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  12. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
16 more
  1. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 297
  2. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  3. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 262
  4. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 224
  5. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 223
  6. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 201
  7. 2024Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsalcited by 118
  8. 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphscited by 110
  9. 2015Multi-task Sequence to Sequence Learningcited by 94
  10. 2024Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Samplingcited by 90
  11. 2025Scaling Laws for Forgetting during Finetuning with Pretraining Data Injectioncited by 35
  12. 2025SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMscited by 22
  13. 2023Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perceptioncited by 19
  14. 2020Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processescited by 17
  15. 2026Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillationcited by 16
  16. 2025LLM-Oriented Token-Adaptive Knowledge Distillationcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 15.

15 papers.

Sort:

By area:

  1. 2026OneReason Technical Reportcited by 8
  2. 2026Weak-to-Strong On-Policy Distillationcited by 8
  3. 2026When Top-K Misses the Decision: Tool-Call Drift in Multi-Teacher On-Policy Distillationcited by 4
  4. 2026Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Modelscited by 4
  5. 2026Who Teaches Which Token? Verifier-Gated Multi-Expert On-Policy Distillation for Scientific Reasoningcited by 3
  6. 2026D3-MOPD: Dynamic Domain ScheDuling for Efficient Multi-Teacher Distillationcited by 2
  7. 2026Data-Free On-Policy Distillation: How Far Can We Go Without External Data?cited by 1
  8. 2026One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillationcited by 0
  9. 2026Recovering General Capabilities via Uncertainty-Calibrated Multi-Teacher On-Policy Distillationcited by 0
  10. 2026Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Compositioncited by 0
  11. 2026Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models?cited by 0
  12. 2026ChemOPD: Multi-Teacher On-Policy Distillation for Multi-Task Chemical Reasoningcited by 0
3 more
  1. 2026Look Before You Select: Rethinking Vocabulary Sparsification in On-Policy Distillationcited by 0
  2. 2026Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shiftscited by 0
  3. 2026Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Open-MOPD: Diagnosing and Fixing Capability Imbalance in Multi-Teacher On-Policy Distillation
  2. 2026Slow-Fast Multi-Teacher On-Policy Distillation for Capability Preservation
  3. 2026MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
  4. 2026Rethinking Self-Distillation for Multi-Teacher Capability Merging
  5. 2026Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals
  6. 2026MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation
  7. 2026DOPD: Dual On-policy Distillation
  8. 2026Not Every Divergence Should Be Suppressed: Counterfactual Recoverability in On-Policy Distillation