mlsanityData to 19 Oct 2026

TS-OPD: Reconciling ASR and QA in Speech Language Models via Task-Specific On-Policy Distillation

arXiv 2609.29464 ↗First posted 24 Sep 2026cs.SDcited by 0

On the map: Speech and audio language models → Audio language models and spoken dialogue

Abstract (arXiv)

Speech Language Models (SLMs) inherit strong instruction-following capabilities from pretrained language models, yet ASR specialization can substantially degrade them. To address this ASR--QA trade-off, we propose Task-Specific On-Policy Distillation (TS-OPD), which leverages models before and after ASR specialization as complementary QA and ASR teachers. The student generates separate task-conditioned trajectories for ASR and QA, each supervised only by its corresponding teacher, thereby reducing direct competition between the two supervision signals. Experiments on basic ASR, contextual ASR, and QA demonstrate that TS-OPD improves recognition while preserving QA capability. Moreover, TS-OPD remains robust across different balancing coefficients and continues to benefit from increased distillation data.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Audio language models and spoken dialogue, one of 11 directions in Speech and audio language models. Builds on 22 of its 24 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

22 of 24 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Language-Specialized Multi-Teacher On-Policy Distillation for Multilingual LLM-Based ASR4 Aug
  2. 2026X3-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment23 Jul
  3. 2026StepAudio 2.5 Technical Report22 May
  4. 2026X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs6 Mar
  5. 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM1 Nov
  6. 2024DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data30 Sep

Other references · 16

  1. 2025Qwen3 Technical Reportcited by 9,258
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,661
  3. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 995
  4. 2025Qwen3-Omni Technical Reportcited by 578
  5. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 268
  6. 2026Qwen3-ASR Technical Reportcited by 162
  7. 2025ContextASR-Bench: A Massive Contextual Speech Recognition Benchmarkcited by 22
  8. 2024AlignFormer: Modality Matching Can Achieve Better Zero-shot Instruction-Following Speech-LLMcited by 21
  9. 2026Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillationcited by 21
  10. 2022Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognitioncited by 18
  11. 2024SSR: Alignment-Aware Modality Connector for Speech Language Modelscited by 17
  12. 2025Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Modelscited by 13
4 more
  1. 2024BLSP-KD: Bootstrapping Language-Speech Pre-training via Knowledge Distillationcited by 10
  2. 2025Cross-Modal Knowledge Distillation for Speech Large Language Modelscited by 9
  3. 2026On-Policy Self-Distillation for Multi-Dialect ASR: Mastering Dialects, Retaining Mandarincited by 2
  4. 2026SALMONN-2: Advancing General-Purpose Hearing Abilities with Self-Supervised Representationscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Data-Efficient On-Policy Distillation for Automatic Speech Recognition
  2. 2026Language-Aware Distillation for Multilingual Instruction-Following Speech LLMs with ASR-Only Supervision
  3. 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
  4. 2025Seewo's Submission to MLC-SLM: Lessons learned from Speech Reasoning Language Models
  5. 2024Leave No Knowledge Behind During Knowledge Distillation: Towards Practical and Effective Knowledge Distillation for Code-Switching ASR Using Realistic Data
  6. 2026Reward-Tilted On-Policy Distillation for Acoustic Grounding in Audio-Language Models
  7. 2023Multilingual DistilWhisper: Efficient Distillation of Multi-task Speech Models via Language-Specific Experts
  8. 2023Knowledge Distillation from Multiple Foundation Models for End-to-End Speech Recognition