mlsanityData to 7 Oct 2026

SPAR-K: Scheduled Periodic Alternating Early Exit for Spoken Language Models

arXiv 2603.09215 ↗First posted 10 Mar 2026cs.CL eess.AScited by 0

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Interleaved spoken language models (SLMs) alternately generate text and speech tokens, but decoding at full transformer depth for every step becomes costly, especially due to long speech sequences. We propose SPAR-K, a modality-aware early exit framework designed to accelerate interleaved SLM inference while preserving perceptual quality. SPAR-K introduces a speech alternating-depth schedule: most speech positions exit at a fixed intermediate layer, while periodic full-depth "refresh" steps mitigate distribution shift due to early exit. We evaluate our framework using Step-Audio-2-mini and GLM-4-Voice across four datasets spanning reasoning, factual QA, and dialogue tasks, measuring performance in terms of ASR transcription accuracy and perceptual quality. Experimental results demonstrate that SPAR-K largely preserves question-answering accuracy with a maximum accuracy drop of 0.82% while reducing average speech decoding depth by up to 11% on Step-Audio-2-mini and 5% on GLM-4-Voice, both with negligible changes in MOS and WER and no auxiliary computation overhead. We further demonstrate that confidence-based early exit strategies, widely used in text LLMs, are suboptimal for SLMs, highlighting that the unique statistical nature of speech tokens necessitates a specialized early exit design.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 41 of its 47 references that are in the map. First posted on arXiv 10 Mar 2026.

Builds on

41 of 47 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Splitformer: An improved early-exit architecture for automatic speech recognition on edge devices22 Jun
  2. 2025A Survey of Early Exit Deep Neural Networks in NLP13 Jan
  3. 2024An Efficient Inference Framework for Early-exit Large Language Models25 Jul
  4. 2024DAISY: Data Adaptive Self-Supervised Early Exit for Speech Representation Models8 Jun
  5. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding25 Apr
  6. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism8 Dec
  7. 2023Training dynamic models using early exits for automatic speech recognition on resource-constrained devices18 Sep
  8. 2022Confident Adaptive Language Modeling14 Jul
  9. 2022HuBERT-EE: Early Exiting HuBERT for Efficient Speech Recognition13 Apr

Other references · 32

  1. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  2. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  3. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  4. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  5. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  6. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  7. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  8. 2022Efficiently Scaling Transformer Inferencecited by 529
  9. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
  10. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 323
  11. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
  12. 2019Depth-Adaptive Transformercited by 227
20 more
  1. 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 218
  2. 2025Kimi-Audio Technical Reportcited by 210
  3. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 204
  4. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 116
  5. 2025Step-Audio 2 Technical Reportcited by 101
  6. 2023Textually Pretrained Speech Language Modelscited by 99
  7. 2023Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLMcited by 92
  8. 2023SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Modelscited by 85
  9. 2024Recent Advances in Speech Language Models: A Surveycited by 85
  10. 2023Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingcited by 80
  11. 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 78
  12. 2025Baichuan-Audio: A Unified Framework for End-to-End Speech Interactioncited by 68
  13. 2024SLAM-Omni: Timbre-Controllable Voice Interaction System with Single-Stage Trainingcited by 58
  14. 2024Scaling Speech-Text Pre-training with Synthetic Interleaved Datacited by 45
  15. 2023Toward Joint Language Modeling for Speech Units and Textcited by 27
  16. 2025STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Modelscited by 25
  17. 2025Characterizing and Optimizing LLM Inference Workloads on CPU-GPU Coupled Architecturescited by 8
  18. 2024Interleaved Speech-Text Language Models for Simple Streaming Text-to-Speech Synthesiscited by 6
  19. 2025Accelerating Autoregressive Speech Synthesis Inference With Speech Speculative Decodingcited by 6
  20. 2025Speech Token Prediction via Compressed-to-fine Language Modeling for Speech Generationcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
  2. 2024Internalizing ASR with Implicit Chain of Thought for Efficient Speech-to-Speech Conversational LLM
  3. 2026Stride-k Subsampling: Train-Free Audio Token Reduction for Whisper
  4. 2026Decoder-only Conformer with Modality-aware Sparse Mixtures of Experts for ASR
  5. 2025VoxEval: Benchmarking the Knowledge Understanding Capabilities of End-to-End Spoken Language Models
  6. 2025Seewo's Submission to MLC-SLM: Lessons learned from Speech Reasoning Language Models
  7. 2025Training and Inference Efficiency of Encoder-Decoder Speech Models
  8. 2025Whisfusion: Parallel ASR Decoding with Masked Diffusion