# Audio and omni-modal token compression

Area: [Efficient LLM inference and serving](https://mlsanity.com/inference/index.md). HTML: https://mlsanity.com/inference/audio-and-omni-modal-token-compression/

## Key facts

- Audio and omni-modal token compression: 32 papers on arXiv since 2023 (data through 14 Oct 2026).
- 11 of its 32 mapped papers (34%) cite “OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Models” (2026, arXiv 2602.04804).
- 9 of its 32 mapped papers (28%) cite “OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models” (2025, arXiv 2511.14582).

## Most cited here

- [OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Models](https://mlsanity.com/paper/2602.04804/) (2026), cited by 11 papers here
- [OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models](https://mlsanity.com/paper/2511.14582/) (2025), cited by 9 papers here
- [SpeechPrune: Context-aware Token Pruning for Speech Information Retrieval](https://mlsanity.com/paper/2412.12009/) (2024), cited by 7 papers here
- [Accelerating Transducers through Adjacent Token Merging](https://mlsanity.com/paper/2306.16009/) (2023), cited by 6 papers here
- [EchoingPixels: Aliasing-Resistant Joint Token Reduction for Audio-Visual LLMs](https://mlsanity.com/paper/2512.10324/) (2025), cited by 3 papers here
- [AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs](https://mlsanity.com/paper/2510.07293/) (2025), cited by 2 papers here
- [Towards Audio Token Compression in Large Audio Language Models](https://mlsanity.com/paper/2511.20973/) (2025), cited by 2 papers here
- [DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression](https://mlsanity.com/paper/2603.15685/) (2026), cited by 2 papers here
- [Keep What Audio Cannot Say: Context-Preserving Token Pruning for Omni-LLMs](https://mlsanity.com/paper/2605.11605/) (2026), cited by 2 papers here
- [OmniDrop: Layer-wise Token Pruning for Omni-modal LLMs via Query-Guidance](https://mlsanity.com/paper/2605.14458/) (2026), cited by 2 papers here