# Stage-adaptive Token Selection for Efficient Omni-modal LLMs

arXiv: https://arxiv.org/abs/2605.20035 · HTML: https://mlsanity.com/paper/2605.20035/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 19 May 2026.

## Builds on (most cited first, 14 of 14)

- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models](https://mlsanity.com/paper/2407.07895/) (2024)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe](https://mlsanity.com/paper/2509.18154/) (2025)
- [Qwen3.5-Omni Technical Report](https://mlsanity.com/paper/2604.15804/) (2026)
- [Baichuan-Omni-1.5 Technical Report](https://mlsanity.com/paper/2501.15368/) (2025)
- [Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities](https://mlsanity.com/paper/2505.17862/) (2025)
- [video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Models](https://mlsanity.com/paper/2506.15220/) (2025)
- [ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shorts](https://mlsanity.com/paper/2507.20939/) (2025)
- [OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Models](https://mlsanity.com/paper/2602.04804/) (2026)
- [LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs](https://mlsanity.com/paper/2603.19217/) (2026)

## Built on by (most cited first, 3 of 3)

- [OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models](https://mlsanity.com/paper/2608.03812/) (2026)
- [LEAP: Learned Block-wise Evidence Retrieval for Long Audio-Video Perception](https://mlsanity.com/paper/2609.39938/) (2026)
- [Learning to Reason with Compressed Context: Ground-Truth-Free Adaptation of OmniLLMs via Self-Distillation](https://mlsanity.com/paper/2609.39953/) (2026)

## Abstract (arXiv)

The unified audio-visual understanding capability of omni-modal large language models (om-LLMs) comes at the high cost of processing tens of thousands of visual and audio tokens throughout the LLM. To reduce the cost, training-free methods for om-LLM token selection are being actively developed, operating either pre-LLM or inner-LLM. Recognizing that the non-textual token redundancy varies across stages and LLM layers, we propose in this paper a novel stage-adaptive token selection (SEATS) method that operates in both pre-LLM and inner-LLM stages. Given an overall token budget measured by the token retention rate (TRR), SEATS performs top-down token budget allocation in an input-dependent manner. Specifically, a larger TRR is used for pre-LLM token selection, whilst within the LLM, all tokens are retained in shallow layers, increasingly decreased TRRs are applied in middle layers, and the non-textual tokens are fully removed in late layers. Extensive experiments on three representative om-LLMs and six benchmarks verify the viability of the proposed method. By retaining only 10% of visual and audio tokens, SEATS preserves 94.9% of the full-token performance on Qwen2.5-Omni while achieving a 9.3x FLOPs reduction and a 4.9x prefill speedup.