# MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs

arXiv: https://arxiv.org/abs/2508.18264 · HTML: https://mlsanity.com/paper/2508.18264/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 30 of its 37 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Visual token pruning and efficiency](https://mlsanity.com/mllm/visual-token-pruning-and-efficiency/)

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension](https://mlsanity.com/paper/2307.16125/) (2023)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [Are We on the Right Way for Evaluating Large Vision-Language Models?](https://mlsanity.com/paper/2403.20330/) (2024)
- [An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models](https://mlsanity.com/paper/2403.06764/) (2024)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models](https://mlsanity.com/paper/2403.15388/) (2024)
- [SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference](https://mlsanity.com/paper/2410.04417/) (2024)
- [Seed1.5-VL Technical Report](https://mlsanity.com/paper/2505.07062/) (2025)
- [Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models](https://mlsanity.com/paper/2403.18814/) (2024)
- [VisionZip: Longer is Better but Not Necessary in Vision Language Models](https://mlsanity.com/paper/2412.04467/) (2024)
- [SmolVLM: Redefining small and efficient multimodal models](https://mlsanity.com/paper/2504.05299/) (2025)
- [DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models](https://mlsanity.com/paper/2503.02175/) (2025)
- [VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning](https://mlsanity.com/paper/2507.13348/) (2025)
- [Intra-Modal Proxy Learning for Zero-Shot Visual Categorization with CLIP](https://mlsanity.com/paper/2310.19752/) (2023)

## Built on by (most cited first, 30 of 33)

- [VideoNSA: Native Sparse Attention Scales Video Understanding](https://mlsanity.com/paper/2510.02295/) (2025)
- [VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs](https://mlsanity.com/paper/2510.16598/) (2025)
- [Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncation](https://mlsanity.com/paper/2607.14557/) (2026)
- [Eye Gaze Tells You Where to Compute: Gaze-Driven Efficient VLMs](https://mlsanity.com/paper/2509.16476/) (2025)
- [SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models](https://mlsanity.com/paper/2609.01004/) (2026)
- [MI-Pruner: Crossmodal Mutual Information-guided Token Pruner for Efficient MLLMs](https://mlsanity.com/paper/2604.03072/) (2026)
- [Rethinking Model Efficiency: Multi-Agent Inference with Large Models](https://mlsanity.com/paper/2604.04929/) (2026)
- [PathSelect: Sequential Token Selection for Whole Slide Pathology](https://mlsanity.com/paper/2607.23631/) (2026)
- [Who Speaks for the Pruned? Visual Token Pruning as Coverage Optimization](https://mlsanity.com/paper/2609.03158/) (2026)
- [StorageXTuner: An LLM Agent-Driven Automatic Tuning Framework for Heterogeneous Storage Systems](https://mlsanity.com/paper/2510.25017/) (2025)
- [Evading Visual Aphasia: Contrastive Adaptive Semantic Token Pruning for Vision-Language Models](https://mlsanity.com/paper/2605.09429/) (2026)
- [BreastGPT: A Multimodal Large Language Model for the Full Spectrum of Breast Cancer Clinical Routine](https://mlsanity.com/paper/2606.04911/) (2026)
- [Think in Sets for Streaming Video Token Compression](https://mlsanity.com/paper/2608.01169/) (2026)
- [DiffPrune: differentiable information throttling for token pruning in vision-language models](https://mlsanity.com/paper/2608.01985/) (2026)
- [DIVE: Dynamic Iterative Visual Evidence Construction for Efficient Vision-Language Models](https://mlsanity.com/paper/2608.04496/) (2026)
- [Learning to Predict Middle-Layer Attention in MLLMs for Visual Token Pruning](https://mlsanity.com/paper/2608.06411/) (2026)
- [Centering before Pruning: Lightweight Geometry Correction for Diversity-Based Visual Token Pruning in LVLMs](https://mlsanity.com/paper/2608.30263/) (2026)
- [QCPruner: Query-Conditioned Population Coverage for Visual Token Pruning](https://mlsanity.com/paper/2609.19990/) (2026)
- [Rethinking Long-Video Efficiency: A Joint Allocation Perspective on Frames, Pixels, and Front-End Latency](https://mlsanity.com/paper/2610.04318/) (2026)
- [Fast-Slow Efficient Training for Multimodal Large Language Models via Visual Token Pruning](https://mlsanity.com/paper/2602.03815/) (2026)
- [GOTS: Greedy Orthogonal Token Selection for High-Resolution Vision-Language Models](https://mlsanity.com/paper/2607.23913/) (2026)
- [Messages, Not Tokens: Grounded Coresets for Faithful VLM Compression](https://mlsanity.com/paper/2608.02134/) (2026)
- [CARVE: Cross-Slice Anisotropic Reallocation of Visual Evidence for Efficient 3D Medical Volume Understanding](https://mlsanity.com/paper/2608.04515/) (2026)
- [SeVeR: Selective Visual Exposure and Retrieval for 3D Medical Image Question Answering](https://mlsanity.com/paper/2608.25630/) (2026)
- [PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference](https://mlsanity.com/paper/2608.27206/) (2026)
- [StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection](https://mlsanity.com/paper/2609.16841/) (2026)
- [Pay More Attention To Text In High-Resolution MLLMs](https://mlsanity.com/paper/2609.23495/) (2026)
- [From Token Importance to Conditional Removability: Rethinking Visual Token Pruning in Multimodal Large Language Models](https://mlsanity.com/paper/2609.26484/) (2026)
- [CoViST: Visual Token Compression via Composable States](https://mlsanity.com/paper/2609.33397/) (2026)
- [MiCo: Mutual Information Coverage Optimization through Semantic Erasure Modeling for Efficient MLLM Inference](https://mlsanity.com/paper/2609.34330/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) demonstrate impressive performance in understanding visual content with language instruction by converting visual inputs to vision tokens. However, redundancy in vision tokens results in the degraded inference efficiency of VLMs. While many algorithms have been proposed to reduce the number of vision tokens, most of them apply only unimodal information (i.e., vision/text) for pruning and ignore the inherent multimodal property of vision-language tasks. Moreover, it lacks a generic criterion that can be applied to different modalities. To mitigate this limitation, in this work, we propose to leverage both vision and text tokens to select informative vision tokens by the coverage criterion. We first formulate the subset selection problem as a maximum coverage problem. Afterwards, a subset of vision tokens is optimized to cover the text tokens and the original set of vision tokens, simultaneously. The proposed method MMTok is extensively evaluated on benchmark datasets with different VLMs. The comparison illustrates that vision and text information are complementary, and combining multimodal information can surpass the unimodal baseline with a clear margin. Moreover, under the maximum coverage criterion on the POPE dataset, our method achieves a 1.87x speedup while maintaining 98.7% of the original performance on LLaVA-NeXT-13B. Finally, with only four vision tokens, 87.7% of the original performance is still preserved on LLaVA-1.5-7B. These results highlight the effectiveness of coverage in token selection. The code is available at https://github.com/Ironieser/mmtok