mlsanityData to 14 Oct 2026

SpeechPrune: Context-aware Token Pruning for Speech Information Retrieval

arXiv 2412.12009 ↗First posted 16 Dec 2024eess.AS cs.AI cs.CLcited by 21

On the map: Efficient LLM inference and serving → Audio and omni-modal token compression

Abstract (arXiv)

We introduce Speech Information Retrieval (SIR), a new long-context task for Speech Large Language Models (Speech LLMs), and present SPIRAL, a 1,012-sample benchmark testing models' ability to extract critical details from approximately 90-second spoken inputs. While current Speech LLMs excel at short-form tasks, they struggle with the computational and representational demands of longer audio sequences. To address this limitation, we propose SpeechPrune, a training-free token pruning strategy that uses speech-text similarity and approximated attention scores to efficiently discard irrelevant tokens. In SPIRAL, SpeechPrune achieves accuracy improvements of 29% and up to 47% over the original model and the random pruning model at a pruning rate of 20%, respectively. SpeechPrune can maintain network performance even at a pruning level of 80%. This approach highlights the potential of token-level pruning for efficient and scalable long-form speech understanding.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Audio and omni-modal token compression, one of 12 directions in Efficient LLM inference and serving. Builds on 22 of its 25 references that are in the map. First posted on arXiv 16 Dec 2024.

Builds on

22 of 25 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  4. 2024Qwen2-Audio Technical Reportcited by 560
  5. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  6. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  7. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Modelscited by 402
  8. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  9. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
  10. 2024AnyGPT: Unified Multimodal LLM with Discrete Sequence Modelingcited by 217
  11. 2023Listen, Think, and Understandcited by 209
  12. 2022Transformer Quality in Linear Timecited by 191
10 more
  1. 2019DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehensioncited by 156
  2. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
  3. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  4. 2021Learned Token Pruning for Transformerscited by 139
  5. 2022On The Computational Complexity of Self-Attentioncited by 139
  6. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 128
  7. 2024WavLLM: Towards Robust and Adaptive Speech Large Language Modelcited by 109
  8. 2024A Survey on Speech Large Language Models for Understandingcited by 71
  9. 2024Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Taskscited by 71
  10. 2024SpeechVerse: A Large-scale Generalizable Audio Language Modelcited by 63

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 14.

21 papers.

Sort:

By area:

  1. 2025KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systemscited by 40
  2. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 25
  3. 2025OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Modelscited by 22
  4. 2025Keyframe-oriented Vision Token Pruning: Enhancing Efficiency of Large Vision Language Models on Long-Form Video Processingcited by 18
  5. 2025Token Pruning in Audio Transformers: Optimizing Performance and Decoding Patch Importancecited by 12
  6. 2026DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compressioncited by 6
  7. 2026Do We Need Distinct Representations for Every Speech Token? Unveiling and Exploiting Redundancy in Large Speech Language Modelscited by 4
  8. 2026OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Modelscited by 3
  9. 2026OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Modelscited by 3
  10. 2025FastLongSpeech: Enhancing Large Speech-Language Models for Efficient Long-Speech Processingcited by 2
  11. 2025AsyncVoice Agent: Real-Time Explanation for LLM Planning and Reasoningcited by 2
  12. 2026EVA: Accelerating LLM Decoding via an Efficient Vector Quantization Architecturecited by 2
9 more
  1. 2026OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Modelscited by 2
  2. 2026Bridging the Perception Gap: A Lightweight Coarse-to-Fine Architecture for Edge Audio Systemscited by 1
  3. 2026OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Modelscited by 1
  4. 2026OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMscited by 1
  5. 2026FastSLM: Hierarchical Temporal Abstraction for Efficient Long-Form Speech Adaptationcited by 0
  6. 2026VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inferencecited by 0
  7. 2026Vortex: Bridging Extreme Compression and Efficient LLM Inferencecited by 0
  8. 2026Audio Token Attention Is Predictable Before the Language Model Runscited by 0
  9. 2026AudioTETRIS: Token Elimination via Transitional Importance Signals for Efficient Audio LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Segmentwise Pruning in Audio-Language Models
  2. 2026Omni-Prune: Query-Aware Unified Token Pruning for Efficient Omnimodal Large Language Models
  3. 2026FastAV: Efficient Token Pruning for Audio-Visual Large Language Model Inference
  4. 2026Keep What Audio Cannot Say: Context-Preserving Token Pruning for Omni-LLMs
  5. 2025Leveraging KV Similarity for Online Structured Pruning in LLMs
  6. 2026OmniSelect: Dynamic Modality-Aware Token Compression for Efficient Omni-modal Large Language Models
  7. 2023Accurate and Structured Pruning for Efficient Automatic Speech Recognition
  8. 2026OmniDrop: Layer-wise Token Pruning for Omni-modal LLMs via Query-Guidance