# When Top-K Misses the Decision: Tool-Call Drift in Multi-Teacher On-Policy Distillation

arXiv: https://arxiv.org/abs/2607.07050 · HTML: https://mlsanity.com/paper/2607.07050/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 30 of its 33 references that are in the map.
- First posted on arXiv 8 Jul 2026.

## Placed in

- Knowledge distillation / [On-policy and self-distillation for LLMs](https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/)

## Builds on (most cited first, 30 of 30)

- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models](https://mlsanity.com/paper/2601.18734/) (2026)
- [Entropy-Aware On-Policy Distillation of Language Models](https://mlsanity.com/paper/2603.07079/) (2026)
- [APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay](https://mlsanity.com/paper/2504.03601/) (2025)
- [A Survey of On-Policy Distillation for Large Language Models](https://mlsanity.com/paper/2604.00626/) (2026)
- [SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning](https://mlsanity.com/paper/2509.02479/) (2025)
- [MiniLLM: On-Policy Distillation of Large Language Models](https://mlsanity.com/paper/2306.08543/) (2023)
- [TIP: Token Importance in On-Policy Distillation](https://mlsanity.com/paper/2604.14084/) (2026)
- [DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs](https://mlsanity.com/paper/2503.07067/) (2025)
- [MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training](https://mlsanity.com/paper/2606.30406/) (2026)
- [Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation](https://mlsanity.com/paper/2605.26844/) (2026)
- [SMART: Self-Aware Agent for Tool Overuse Mitigation](https://mlsanity.com/paper/2502.11435/) (2025)
- [When2Call: When (not) to Call Tools](https://mlsanity.com/paper/2504.18851/) (2025)
- [Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation](https://mlsanity.com/paper/2606.02684/) (2026)
- [ToDi: Token-wise Distillation via Fine-Grained Divergence Control](https://mlsanity.com/paper/2505.16297/) (2025)
- [MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate](https://mlsanity.com/paper/2605.01347/) (2026)
- [Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMs](https://mlsanity.com/paper/2503.16870/) (2025)
- [On Group Relative Policy Optimization Collapse in Agent Search: The Lazy Likelihood-Displacement](https://mlsanity.com/paper/2512.04220/) (2025)
- [Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation](https://mlsanity.com/paper/2605.27115/) (2026)
- [LLM-Oriented Token-Adaptive Knowledge Distillation](https://mlsanity.com/paper/2510.11615/) (2025)
- [Don't Ignore the Tail: Decoupling top-K Probabilities for Efficient Language Model Distillation](https://mlsanity.com/paper/2602.20816/) (2026)
- [Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix It](https://mlsanity.com/paper/2606.26027/) (2026)
- [The Tool-Overuse Illusion: Why Does LLM Prefer External Tools over Internal Knowledge?](https://mlsanity.com/paper/2604.19749/) (2026)
- [Communication-Aware Knowledge Distillation for Federated LLM Fine-Tuning over Wireless Networks](https://mlsanity.com/paper/2509.01750/) (2025)
- [Stabilizing On-Policy Distillation for MLLM Reasoning with Global Normalization](https://mlsanity.com/paper/2606.09091/) (2026)
- [Reliability Gated Multi-Teacher Distillation for Low Resource Abstractive Summarization](https://mlsanity.com/paper/2604.03192/) (2026)
- [Identified-Set Geometry of Distributional Model Extraction under Top-K Censored API Access](https://mlsanity.com/paper/2605.10407/) (2026)

## Built on by (most cited first, 4 of 4)

- [Who Teaches Which Token? Verifier-Gated Multi-Expert On-Policy Distillation for Scientific Reasoning](https://mlsanity.com/paper/2609.15404/) (2026)
- [UP-MOPD: Update Projection in Multi-Teacher On-Policy Distillation](https://mlsanity.com/paper/2610.08398/) (2026)
- [Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts](https://mlsanity.com/paper/2610.10460/) (2026)
- [The Gradient That Top-k Distillation Never Computes](https://mlsanity.com/paper/2610.15589/) (2026)

## Abstract (arXiv)

Top-K teacher logits make on-policy distillation tractable, but retained teacher mass does not certify student-relative gradient fidelity. We study routed, two-teacher tool-use distillation. In a frozen Qwen3.5-9B audit, the tool teacher ranks the entry token first on 500 tool prompts, and top-32 reinforces it in every matched pair. The response teacher's top-32 instead retains 99.99% mass yet contains the same token on only 0.4% of 500 response prompts; student-aware support restores the coordinate in all matched pairs and nearly matches full-vocabulary descent. On the tool route, top-32 preserves the reinforcing direction but substantially attenuates its full-vocabulary magnitude despite mass displayed as 1.000000. Matched restoration connects the response-side omission to behavior: restoring the entry coordinate at every supervised response position lowers full-generation over-calling from 14.2 ± 2.1% to 3.7 ± 0.5% across three seeds, but also lowers call recall by 12.4 points. A non-tool placebo changes over-calling by only 0.95 points. A synthetic output-logit-gradient-matched reroute also fails to reproduce exact restoration. We compare three optimization layers: student-aware support changes distilled coordinates, loss shaping changes retained-signal strength, and validation-tuned entry bias shifts inference without retraining. They expose distinct trade-offs in correction scope, system access, and required-call retention. Llama-3.1-8B reproduces the directional support asymmetry under its native JSON protocol. The results causally implicate decision-critical support omission as one contributor in the primary Qwen setting and show why both support fidelity and deployment costs must be audited. Code and aggregate artifacts are available at https://github.com/shen-jiabin/decision-support-opd.