Resolution as a First-Class Decision: Task-Conditioned Routing for Efficient Multimodal Large Language Models
On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency
Abstract (arXiv)
The inference efficiency of Multimodal Large Language Models (MLLMs) is severely constrained by massive visual token sequences induced by high-resolution inputs, with computational cost scaling quadratically. Existing approaches primarily focus on downstream token compression, while overlooking a fundamental upstream inefficiency: input resolution is treated as a static, task-agnostic hyperparameter. We propose Task-Conditioned Resolution Routing (TCRR), which formulates visual compression as a task-conditioned decision and employs a lightweight cross-modal router that conditions backbone visual representations on textual semantics via feature-wise modulation and cross-attention to predict the minimal sufficient compression level per query. To support this, we curate a dataset of 500k samples across 12 task categories, labeled via a teacher-oracle pipeline to approximate Pareto-optimal compression scales. Extensive experiments across diverse architectures show that TCRR achieves a superior efficiency frontier, specifically reducing visual FLOPs by 40.9% and latency by 53.7% on Qwen3-VL-8B while preserving competitive performance. Further analysis of scaling behavior confirms that dynamically routing visual compression enables optimal resource allocation without modifying the MLLM backbone.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 36 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
34 of 36 references.
Closest ancestors · 3
Same direction, newest first
- 2024FastVLM: Efficient Vision Encoding for Vision Language Models17 Dec
- 2024VisionZip: Longer is Better but Not Necessary in Vision Language Models5 Dec
- 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
Other references · 31
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2023Visual Instruction Tuningcited by 7,869
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
19 more
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2019Towards VQA Models That Can Readcited by 1,718
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
- 2022Token Merging: Your ViT But Fastercited by 1,021
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
- 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
- 2021InfographicVQAcited by 437
- 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 332
- 2022Confident Adaptive Language Modelingcited by 310
- 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
- 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 244
- 2024MobileNetV4 -- Universal Models for the Mobile Ecosystemcited by 99
- 2024Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Datacited by 67
- 2025FineVision: Open Data Is All You Needcited by 53
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs
- 2026Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines
- 2025TokenCarve: Information-Preserving Visual Token Compression in Multimodal Large Language Models
- 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
- 2025A Survey of Token Compression for Efficient Multimodal Large Language Models
- 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
- 2026Stepwise Token Selection for Efficient Multimodal Large Language Models
- 2026FocusVTC: Efficient and High-Performance Visual Text Compression with Adaptive Resolution