mlsanityData to 7 Oct 2026

When Top-K Misses the Decision: Tool-Call Drift in Multi-Teacher On-Policy Distillation

arXiv 2607.07050 ↗First posted 8 Jul 2026cs.CL cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Top-K teacher logits make on-policy distillation tractable, but retained teacher mass does not certify student-relative gradient fidelity. We study routed, two-teacher tool-use distillation. In a frozen Qwen3.5-9B audit, the tool teacher ranks the entry token first on 500 tool prompts, and top-32 reinforces it in every matched pair. The response teacher's top-32 instead retains 99.99% mass yet contains the same token on only 0.4% of 500 response prompts; student-aware support restores the coordinate in all matched pairs and nearly matches full-vocabulary descent. On the tool route, top-32 preserves the reinforcing direction but substantially attenuates its full-vocabulary magnitude despite mass displayed as 1.000000. Matched restoration connects the response-side omission to behavior: restoring the entry coordinate at every supervised response position lowers full-generation over-calling from 14.2 ± 2.1% to 3.7 ± 0.5% across three seeds, but also lowers call recall by 12.4 points. A non-tool placebo changes over-calling by only 0.95 points. A synthetic output-logit-gradient-matched reroute also fails to reproduce exact restoration. We compare three optimization layers: student-aware support changes distilled coordinates, loss shaping changes retained-signal strength, and validation-tuned entry bias shifts inference without retraining. They expose distinct trade-offs in correction scope, system access, and required-call retention. Llama-3.1-8B reproduces the directional support asymmetry under its native JSON protocol. The results causally implicate decision-critical support omission as one contributor in the primary Qwen setting and show why both support fidelity and deployment costs must be audited. Code and aggregate artifacts are available at https://github.com/shen-jiabin/decision-support-opd.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 33 references that are in the map. First posted on arXiv 8 Jul 2026.

Builds on

30 of 33 references.

References · 30

  1. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  2. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  3. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  4. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  5. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
  6. 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 158
  7. 2026Entropy-Aware On-Policy Distillation of Language Modelscited by 151
  8. 2026A Survey of On-Policy Distillation for Large Language Modelscited by 148
  9. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoningcited by 145
  10. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 129
  11. 2026TIP: Token Importance in On-Policy Distillationcited by 90
  12. 2025DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMscited by 73
18 more
  1. 2026MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Trainingcited by 60
  2. 2025SMART: Self-Aware Agent for Tool Overuse Mitigationcited by 52
  3. 2026Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillationcited by 42
  4. 2025When2Call: When (not) to Call Toolscited by 36
  5. 2026Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillationcited by 31
  6. 2025ToDi: Token-wise Distillation via Fine-Grained Divergence Controlcited by 23
  7. 2025On Group Relative Policy Optimization Collapse in Agent Search: The Lazy Likelihood-Displacementcited by 15
  8. 2025Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMscited by 14
  9. 2026Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservationcited by 14
  10. 2025LLM-Oriented Token-Adaptive Knowledge Distillationcited by 12
  11. 2026MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debatecited by 12
  12. 2026Don't Ignore the Tail: Decoupling top-K Probabilities for Efficient Language Model Distillationcited by 7
  13. 2026The Tool-Overuse Illusion: Why Does LLM Prefer External Tools over Internal Knowledge?cited by 6
  14. 2026Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix Itcited by 6
  15. 2025Communication-Aware Knowledge Distillation for Federated LLM Fine-Tuning over Wireless Networkscited by 2
  16. 2026Reliability Gated Multi-Teacher Distillation for Low Resource Abstractive Summarizationcited by 2
  17. 2026Stabilizing On-Policy Distillation for MLLM Reasoning with Global Normalizationcited by 2
  18. 2026Identified-Set Geometry of Distributional Model Extraction under Top-K Censored API Accesscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Who Teaches Which Token? Verifier-Gated Multi-Expert On-Policy Distillation for Scientific Reasoningcited by 2
  2. 2026Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shiftscited by 0