# OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs

arXiv: https://arxiv.org/abs/2607.25669 · HTML: https://mlsanity.com/paper/2607.25669/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Audio and omni-modal token compression, one of 12 directions in Efficient LLM inference and serving.
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 28 Jul 2026.

## Placed in

- Efficient LLM inference and serving / [Audio and omni-modal token compression](https://mlsanity.com/inference/audio-and-omni-modal-token-compression/)

## Builds on (most cited first, 29 of 29)

- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models](https://mlsanity.com/paper/2403.06764/) (2024)
- [LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models](https://mlsanity.com/paper/2403.15388/) (2024)
- [SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference](https://mlsanity.com/paper/2410.04417/) (2024)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs](https://mlsanity.com/paper/2502.04326/) (2025)
- [SoK: Agentic Skills -- Beyond Tool Use in LLM Agents](https://mlsanity.com/paper/2602.20867/) (2026)
- [Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models](https://mlsanity.com/paper/2409.10197/) (2024)
- [Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities](https://mlsanity.com/paper/2505.17862/) (2025)
- [video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models](https://mlsanity.com/paper/2406.15704/) (2024)
- [DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models](https://mlsanity.com/paper/2411.15024/) (2024)
- [PruneVid: Visual Token Pruning for Efficient Video Large Language Models](https://mlsanity.com/paper/2412.16117/) (2024)
- [FrameFusion: Combining Similarity and Importance for Video Token Reduction on Large Vision Language Models](https://mlsanity.com/paper/2501.01986/) (2024)
- [video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Models](https://mlsanity.com/paper/2506.15220/) (2025)
- [Audio-centric Video Understanding Benchmark without Text Shortcut](https://mlsanity.com/paper/2503.19951/) (2025)
- [InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue](https://mlsanity.com/paper/2510.13747/) (2025)
- [OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Models](https://mlsanity.com/paper/2602.04804/) (2026)
- [OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models](https://mlsanity.com/paper/2511.14582/) (2025)
- [SpeechPrune: Context-aware Token Pruning for Speech Information Retrieval](https://mlsanity.com/paper/2412.12009/) (2024)
- [Accelerating Transducers through Adjacent Token Merging](https://mlsanity.com/paper/2306.16009/) (2023)
- [Token Pruning in Audio Transformers: Optimizing Performance and Decoding Patch Importance](https://mlsanity.com/paper/2504.01690/) (2025)
- [Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects](https://mlsanity.com/paper/2604.05546/) (2026)
- [FastAV: Efficient Token Pruning for Audio-Visual Large Language Model Inference](https://mlsanity.com/paper/2601.13143/) (2026)
- [AccKV: Towards Efficient Audio-Video LLMs Inference via Adaptive-Focusing and Cross-Calibration KV Cache Optimization](https://mlsanity.com/paper/2511.11106/) (2025)
- [EchoingPixels: Aliasing-Resistant Joint Token Reduction for Audio-Visual LLMs](https://mlsanity.com/paper/2512.10324/) (2025)
- [OmniSelect: Dynamic Modality-Aware Token Compression for Efficient Omni-modal Large Language Models](https://mlsanity.com/paper/2605.18041/) (2026)
- [OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models](https://mlsanity.com/paper/2605.12056/) (2026)
- [HieraVid: Hierarchical Token Pruning for Fast Video Large Language Models](https://mlsanity.com/paper/2604.01881/) (2026)

## Built on by (most cited first, 1 of 1)

- [Learning to Reason with Compressed Context: Ground-Truth-Free Adaptation of OmniLLMs via Self-Distillation](https://mlsanity.com/paper/2609.39953/) (2026)

## Abstract (arXiv)

Emerging Omni-modal Large Language Models (OmniLLMs) enable unified understanding of text, audio, and video, but their long audio-video token sequences introduce substantial memory and inference costs. Existing compression methods mainly focus on selecting important tokens under fixed budgets, leaving the preceding budget-allocation problem underexplored. We show that direct query-to-audio/video similarity is unreliable for inter-modal budget allocation, and that uniform intra-modal budgets can miss key evidence while retaining redundant content. To address these limitations, we propose OmniDelta, a training-free, skill-driven framework that couples intent-aware inter-modal allocation with content-aware intra-modal allocation. OmniDelta first constructs audio and video skill pools to shift the fixed retained-token budget according to query demand, then reallocates modality budgets over audio segments and video frames using local complexity and temporal redundancy. The resulting local budgets can be combined with existing pruning strategies, preserving the total retained-token ratio while changing where the budget is spent. Experiments on four audio-video benchmarks with two Qwen2.5-Omni models show that OmniDelta establishes a new accuracy-efficiency Pareto frontier across pruning ratios. At 25% token retention on Qwen2.5-Omni-7B, OmniDelta reduces GPU memory by 22.0% and achieves a 1.64x end-to-end speedup over full-token inference.