# Who Teaches Which Token? Verifier-Gated Multi-Expert On-Policy Distillation for Scientific Reasoning

arXiv: https://arxiv.org/abs/2609.15404 · HTML: https://mlsanity.com/paper/2609.15404/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 35 of its 37 references that are in the map.
- First posted on arXiv 14 Sep 2026.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)
- Knowledge distillation / [On-policy and self-distillation for LLMs](https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/)

## Builds on (most cited first, 30 of 35)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains](https://mlsanity.com/paper/2507.17746/) (2025)
- [Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe](https://mlsanity.com/paper/2604.13016/) (2026)
- [General-Reasoner: Advancing LLM Reasoning Across All Domains](https://mlsanity.com/paper/2505.14652/) (2025)
- [TIP: Token Importance in On-Policy Distillation](https://mlsanity.com/paper/2604.14084/) (2026)
- [NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions](https://mlsanity.com/paper/2502.13124/) (2025)
- [Scaling Reasoning Efficiently via Relaxed On-Policy Distillation](https://mlsanity.com/paper/2603.11137/) (2026)
- [MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training](https://mlsanity.com/paper/2606.30406/) (2026)
- [MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning](https://mlsanity.com/paper/2507.16812/) (2025)
- [Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation](https://mlsanity.com/paper/2605.26844/) (2026)
- [Trust Region On-Policy Distillation](https://mlsanity.com/paper/2606.01249/) (2026)
- [Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation](https://mlsanity.com/paper/2605.27115/) (2026)
- [Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifiers](https://mlsanity.com/paper/2606.12507/) (2026)
- [Rubric-based On-policy Distillation](https://mlsanity.com/paper/2605.07396/) (2026)
- [SG-OPD: Sign-Gated On-Policy Distillation via Sign-Consistency Gating and Phased Teacher Sampling](https://mlsanity.com/paper/2606.09304/) (2026)
- [Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation](https://mlsanity.com/paper/2606.19327/) (2026)
- [Reward-Gated On-Policy Distillation](https://mlsanity.com/paper/2607.04037/) (2026)
- [DOPD: Dual On-policy Distillation](https://mlsanity.com/paper/2606.30626/) (2026)
- [EvoRubric: Self-Evolving Rubric-Driven RL for Open-Ended Generation](https://mlsanity.com/paper/2605.29847/) (2026)
- [Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance](https://mlsanity.com/paper/2608.00782/) (2026)
- [H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation](https://mlsanity.com/paper/2607.02592/) (2026)
- [Skill-Conditioned Gated Self-Distillation for LLM Reasoning](https://mlsanity.com/paper/2605.28791/) (2026)
- [CriPO: Enhancing Rubric-based RL via Self-Distillation](https://mlsanity.com/paper/2607.18082/) (2026)
- [CROP: Task Relevance via Counterfactuals for Selective On-Policy Distillation](https://mlsanity.com/paper/2608.13387/) (2026)
- [Improving Data and Reward Design for Scientific Reasoning in Large Language Models](https://mlsanity.com/paper/2602.08321/) (2026)
- [ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning](https://mlsanity.com/paper/2605.23454/) (2026)
- [Reinforcement Learning with Robust Rubric Rewards](https://mlsanity.com/paper/2605.30244/) (2026)
- [When Top-K Misses the Decision: Tool-Call Drift in Multi-Teacher On-Policy Distillation](https://mlsanity.com/paper/2607.07050/) (2026)

## Built on by (most cited first, 3 of 3)

- [Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation](https://mlsanity.com/paper/2609.23697/) (2026)
- [Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models](https://mlsanity.com/paper/2609.38025/) (2026)
- [MeSD: Multi-Evidence Self-Distillation for VideoLLM](https://mlsanity.com/paper/2610.06342/) (2026)

## Abstract (arXiv)

Multi-teacher on-policy distillation (OPD) is becoming the standard way to integrate specialist capabilities into one model: train experts with RL, then distill them into the student on its own rollouts. Existing recipes assign supervision at the sequence level - each prompt goes to one domain teacher and every token receives the same weight - which implicitly assumes that a teacher is uniformly useful across a response. We find instead that useful teacher signal is sparse and heterogeneous along a reasoning trajectory, which raises a finer question: who should teach which token? Verifier-Gated Multi-Expert On-Policy Distillation (VG-OPD) answers it by verification: the counterfactual gain of an expert on a specific answer criterion licenses that expert to teach, its disagreement with the student localizes the supervision, and criterion importance sets its weight; the gated KL enters GRPO as an additive token-level advantage. Instantiated for scientific reasoning with RL-trained capability experts, VG-OPD attains the best overall performance on seven benchmarks for 4B and 8B students, ranking first on five at both scales, with the largest gains on knowledge-intensive scientific reasoning tasks. Further analysis shows that the gains come from localizing verified supervision rather than from adding teachers or distillation loss: misplacing the same supervision budget is the single most damaging change, and indiscriminate distillation drags RL below its own floor where gated distillation lifts it.