mlsanityData to 14 Oct 2026

Be My Tutor: On-Policy Co-Distillation for Mutual LLM Improvement via Peer Feedback

arXiv 2606.14368 ↗First posted 12 Jun 2026cs.LG cs.CLcited by 1

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

We study multi-domain LLM training in which two models, each stronger in a different domain, co-evolve by tutoring each other through on-policy feedback. Unlike one-way distillation or single-model fine-tuning, our goal is mutual Pareto improvement: each model improves across domains without losing its original strength. To this end, we propose On-Policy Co-Distillation (OPCoD), where each student's self-distillation is conditioned on its own correct rollout and feedback from its peer. To make feedback exchange effective, OPCoD uses cognizance-based gating to decide when to give feedback and feedback anchoring to ground feedback in the problem. On Science Q&A tasks, OPCoD consistently outperforms baselines and achieves Pareto improvement across all evaluated domain pairs and students.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 17 of its 30 references that are in the map. First posted on arXiv 12 Jun 2026.

Builds on

17 of 30 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Rebellious Student: Reversing Teacher Signals for Reasoning Exploration with Self-Distilled RLVR11 May
  2. 2026On-Policy Context Distillation for Language Models12 Feb
  3. 2026Reinforcement Learning via Self-Distillation28 Jan

Other references · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  4. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  5. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  6. 2022Galactica: A Large Language Model for Sciencecited by 744
  7. 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 95
  8. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 86
  9. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 53
  10. 2025MARFT: Multi-Agent Reinforcement Fine-Tuningcited by 52
  11. 2025Fine-tuning Large Language Models for Improving Factuality in Legal Question Answeringcited by 28
  12. 2024Teaching Models to Balance Resisting and Accepting Persuasioncited by 19
2 more
  1. 2026Advancing General-Purpose Reasoning Models with Modular Gradient Surgerycited by 9
  2. 2026Disentangling Task Conflicts in Multi-Task LoRA via Orthogonal Gradient Projectioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Look Before You Select: Rethinking Vocabulary Sparsification in On-Policy Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Recursive Self-Improvement via On-Policy Distillation for Reasoning
  2. 2026Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
  3. 2026Weak-to-Strong On-Policy Distillation
  4. 2026Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations
  5. 2026Learn from Whoever Is Right: Answer-Verified Multi-Teacher Distillation for Multi-Domain LLMs
  6. 2026Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling
  7. 2026Latent-MOPD: Latent Multi-Teacher On-Policy Distillation
  8. 2026Matching Supervision to the Student's Learning Capacity: A Unified Framework for On-Policy Self-Distillation