# X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs

arXiv: https://arxiv.org/abs/2603.24596 · HTML: https://mlsanity.com/paper/2603.24596/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 30 of its 31 references that are in the map.
- First posted on arXiv 6 Mar 2026.

## Placed in

- Knowledge distillation / [On-policy and self-distillation for LLMs](https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/)

## Builds on (most cited first, 30 of 30)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning](https://mlsanity.com/paper/2408.05517/) (2024)
- [VoiceBench: Benchmarking LLM-Based Voice Assistants](https://mlsanity.com/paper/2410.17196/) (2024)
- [CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training](https://mlsanity.com/paper/2505.17589/) (2025)
- [FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs](https://mlsanity.com/paper/2407.04051/) (2024)
- [MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix](https://mlsanity.com/paper/2505.13032/) (2025)
- [Small Models Struggle to Learn from Strong Reasoners](https://mlsanity.com/paper/2502.12143/) (2025)
- [NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions](https://mlsanity.com/paper/2502.13124/) (2025)
- [DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment](https://mlsanity.com/paper/2507.02768/) (2025)
- [Voxtral](https://mlsanity.com/paper/2507.13264/) (2025)
- [SD-QA: Spoken Dialectal Question Answering for the Real World](https://mlsanity.com/paper/2109.12072/) (2021)
- [Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Models](https://mlsanity.com/paper/2505.19037/) (2025)
- [Audio MultiChallenge: A Multi-Turn Evaluation of Spoken Dialogue Systems on Natural Human Interaction](https://mlsanity.com/paper/2512.14865/) (2025)
- [Closing the Gap Between Text and Speech Understanding in LLMs](https://mlsanity.com/paper/2510.13632/) (2025)
- [Stronger Models are NOT Stronger Teachers for Instruction Tuning](https://mlsanity.com/paper/2411.07133/) (2024)
- [Cross-Modal Knowledge Distillation for Speech Large Language Models](https://mlsanity.com/paper/2509.14930/) (2025)
- [AHA: Aligning Large Audio-Language Models for Reasoning Hallucinations via Counterfactual Hard Negatives](https://mlsanity.com/paper/2512.24052/) (2025)
- [EchoX: Towards Mitigating Acoustic-Semantic Gap via Echo Training for Speech-to-Speech LLMs](https://mlsanity.com/paper/2509.09174/) (2025)
- [S2SBench: A Benchmark for Quantifying Intelligence Degradation in Speech-to-Speech Large Language Models](https://mlsanity.com/paper/2505.14438/) (2025)

## Built on by (most cited first, 16 of 16)

- [A Survey of On-Policy Distillation for Large Language Models](https://mlsanity.com/paper/2604.00626/) (2026)
- [Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe](https://mlsanity.com/paper/2605.03677/) (2026)
- [Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation](https://mlsanity.com/paper/2606.02684/) (2026)
- [Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillation](https://mlsanity.com/paper/2605.13643/) (2026)
- [Latent On-Policy Self-Distillation](https://mlsanity.com/paper/2608.13040/) (2026)
- [ParaBridge: Bridging Paralinguistic Perception and Dialogue Behavior in Speech Language Models](https://mlsanity.com/paper/2606.10581/) (2026)
- [A Formula-Driven Survey and Research Agenda for On-Policy Distillation](https://mlsanity.com/paper/2606.22793/) (2026)
- [X3-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment](https://mlsanity.com/paper/2607.21550/) (2026)
- [Multimodal Duplex Interaction Agent](https://mlsanity.com/paper/2609.08977/) (2026)
- [Minimizing Modality Gap from the Input Side: Your Speech LLM Can Be a Prosody-Aware Text LLM](https://mlsanity.com/paper/2605.05927/) (2026)
- [VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference](https://mlsanity.com/paper/2608.08569/) (2026)
- [Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation](https://mlsanity.com/paper/2609.15313/) (2026)
- [From Self-Distillation to Self-Practice: Privileged Information for Multi-Turn Agents](https://mlsanity.com/paper/2609.29051/) (2026)
- [Look Before You Select: Rethinking Vocabulary Sparsification in On-Policy Distillation](https://mlsanity.com/paper/2609.34386/) (2026)
- [Full-duplex Spoken Language Modeling with Discrete Diffusion](https://mlsanity.com/paper/2610.17780/) (2026)
- [Repairing Speech Access After Text Updates with Frozen Affine Fields](https://mlsanity.com/paper/2610.20253/) (2026)

## Abstract (arXiv)

While the shift from cascaded dialogue systems to end-to-end (E2E) speech Large Language Models (LLMs) improves latency and paralinguistic modeling, E2E models often exhibit a significant performance degradation compared to their text-based counterparts. The standard Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) training methods fail to close this gap. To address this, we propose X-OPD, a novel Cross-Modal On-Policy Distillation framework designed to systematically align the capabilities of Speech LLMs to their text-based counterparts. X-OPD enables the Speech LLM to explore its own distribution via on-policy rollouts, where a text-based teacher model evaluates these trajectories and provides token-level feedback, effectively distilling teacher's capabilities into student's multi-modal representations. Extensive experiments across multiple benchmarks demonstrate that X-OPD significantly narrows the gap in complex tasks while preserving the model's inherent capabilities.