# Visual Tokens on Demand: Visual Budgeting for Scalable and Efficient Multi-Image Reasoning in Multimodal LLMs

arXiv: https://arxiv.org/abs/2610.18858 · HTML: https://mlsanity.com/paper/2610.18858/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 28 of its 52 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Visual token pruning and efficiency](https://mlsanity.com/mllm/visual-token-pruning-and-efficiency/)
- Efficient LLM inference and serving / [Visual token pruning in multimodal models](https://mlsanity.com/inference/visual-token-pruning-in-multimodal-models/)

## Builds on (most cited first, 28 of 28)

- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [A Corpus for Reasoning About Natural Language Grounded in Photographs](https://mlsanity.com/paper/1811.00491/) (2018)
- [LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding](https://mlsanity.com/paper/2410.17434/) (2024)
- [MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text](https://mlsanity.com/paper/2210.02928/) (2022)
- [MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding](https://mlsanity.com/paper/2406.09411/) (2024)
- [Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers](https://mlsanity.com/paper/2304.10716/) (2023)
- [MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models](https://mlsanity.com/paper/2408.02718/) (2024)
- [Learning to Merge Tokens in Vision Transformers](https://mlsanity.com/paper/2202.12015/) (2022)
- [Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmark](https://mlsanity.com/paper/2407.13766/) (2024)
- [Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning](https://mlsanity.com/paper/2406.12742/) (2024)
- [Don't Look Twice: Faster Video Transformers with Run-Length Tokenization](https://mlsanity.com/paper/2411.05222/) (2024)
- [MIBench: Evaluating Multimodal Large Language Models over Multiple Images](https://mlsanity.com/paper/2407.15272/) (2024)
- [Filter, Correlate, Compress: Training-Free Token Reduction for MLLM Acceleration](https://mlsanity.com/paper/2411.17686/) (2024)
- [ReMI: A Dataset for Reasoning with Multiple Images](https://mlsanity.com/paper/2406.09175/) (2024)
- [Prune Spatio-temporal Tokens by Semantic-aware Temporal Accumulation](https://mlsanity.com/paper/2308.04549/) (2023)
- [Video Token Merging for Long-form Video Understanding](https://mlsanity.com/paper/2410.23782/) (2024)
- [Efficient Video Action Detection with Token Dropout and Context Refinement](https://mlsanity.com/paper/2304.08451/) (2023)
- [VisionTrim: Unified Vision Token Compression for Training-Free MLLM Acceleration](https://mlsanity.com/paper/2601.22674/) (2026)
- [Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods](https://mlsanity.com/paper/2510.07143/) (2025)
- [Pruning One More Token is Enough: Leveraging Latency-Workload Non-Linearities for Vision Transformers on the Edge](https://mlsanity.com/paper/2407.05941/) (2024)
- [InfoTok: Adaptive Discrete Video Tokenizer via Information-Theoretic Compression](https://mlsanity.com/paper/2512.16975/) (2025)
- [CARES: Context-Aware Resolution Selector for VLMs](https://mlsanity.com/paper/2510.19496/) (2025)
- [ToFu: Visual Tokens Reduction via Fusion for Multi-modal, Multi-patch, Multi-image Task](https://mlsanity.com/paper/2503.04444/) (2025)
- [Frequency-Aware Token Reduction for Efficient Vision Transformer](https://mlsanity.com/paper/2511.21477/) (2025)

## Abstract (arXiv)

Multimodal large language models have advanced rapidly, but most remain optimized for single-image inputs. Multi-image reasoning is costly because each image adds many visual tokens, increasing encoding and attention demands. Existing techniques that address the above issue, including visual token pruning, token merging, visual token compression, learned resampling, hierarchical compression, retrieval-augmented visual reasoning, and memory-based architectures, have largely been designed for single-image scenarios, where redundancy is primarily spatial and local. Multi-image reasoning introduces additional challenges: redundancy may occur across images, task-relevant evidence may be sparse or distributed, and aggressive per-image compression can remove details needed for comparison, counting, temporal reasoning, retrieval, or evidence grounding. Moreover, existing token-reduction methods are often either (i) training-free but inference-inefficient, typically relying on accessing attention matrices to get token-importance that may be impractical to access with memory-efficient attention implementations such as FlashAttention, or (ii) training-dependent, requiring the base MLLM to be retrained or adapted to a new compressed-token paradigm. In this work, we study scalable multi-image reasoning and understanding without retraining the base model and without accessing attention matrices at inference time. To this end, we propose mmVB, a lightweight plug-and-play module that assigns query-conditioned per-image token budgets via cross-image context, adding only 1.6M trainable parameters while reducing visual input tokens by 9x. This reduction yields 5x faster time-to-first-token and 7x fewer prefill FLOPs, while maintaining 94.4% of full-resolution accuracy across eight multi-image reasoning benchmarks.