A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients
On the map: Knowledge distillation → On-policy and self-distillation for LLMs
Abstract (arXiv)
Per-token gating of forward/reverse KL losses has become a standard technique for on-policy knowledge distillation (OPD), but existing methods such as EOPD (Jin et al., 2026) and ToDi (Jung et al., 2025) each fix a single gating signal and a single gating direction, and the two have never been compared directly. We introduce a four-coefficient parameterization lambda_t = sigma(a * h_t + b * u(x) + c + d * gap_t) in which direction-aligned proxies of EOPD and ToDi appear as one-dimensional (1D) restrictions, and which adds multi-channel composition and an explicit bias as further degrees of freedom. On TweetEval (Barbieri et al., 2020) emotion and hate, with a Qwen3-32B teacher and a Qwen3-4B student, configurations in the full family reach higher accuracy than the matched-magnitude single-channel (entropy-only / gap-only) 1D restrictions in 33 of 36 comparable cells, and a 26-cell mean-match isolation experiment places dynamic gating ahead of effective-KL-matched static baselines in 19 of 26 cells. Because cells share training data, models, and parameter substructure, we report both counts as exploratory aggregate directional evidence rather than as independent hypothesis tests. Targeted three-seed paired replications of the nine headline comparisons singled out by that sweep -- including a third task, offensive -- are directionally consistent, but individually smaller than the single-seed estimates and not significant at n=3. We therefore present the parameterization primarily as a shared coordinate system for comparing per-token gating designs in short-output classification OPD.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 8 of its 16 references that are in the map. First posted on arXiv 10 Sep 2026.
Builds on
8 of 16 references.
Closest ancestors · 1
Same direction, newest first
Other references · 7
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
- 2016Sequence-Level Knowledge Distillationcited by 801
- 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
- 2025ToDi: Token-wise Distillation via Fine-Grained Divergence Controlcited by 27
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Token Utility for On-Policy Distillation via Local Trust-Region Optimization
- 2026A Token-Level Analysis of Sampled-Token Reverse-KL On-Policy Distillation
- 2026SMOPD: Selective Token-Entropy Masking for Dirty-History Multi-Turn On-Policy Self-Distillation
- 2026ReTaCo: Residual-Target Control for On-Policy Distillation
- 2026PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformation
- 2026Influence-Directed Distillation: Solving the Diversity Bottleneck in Sampled-Token On-Policy Distillation
- 2026STEPS: Selective On-Policy Self-Distillation for Reasoning
- 2026Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models?