mlsanityData to 14 Oct 2026

Resolution as a First-Class Decision: Task-Conditioned Routing for Efficient Multimodal Large Language Models

arXiv 2609.34942 ↗First posted 28 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

The inference efficiency of Multimodal Large Language Models (MLLMs) is severely constrained by massive visual token sequences induced by high-resolution inputs, with computational cost scaling quadratically. Existing approaches primarily focus on downstream token compression, while overlooking a fundamental upstream inefficiency: input resolution is treated as a static, task-agnostic hyperparameter. We propose Task-Conditioned Resolution Routing (TCRR), which formulates visual compression as a task-conditioned decision and employs a lightweight cross-modal router that conditions backbone visual representations on textual semantics via feature-wise modulation and cross-attention to predict the minimal sufficient compression level per query. To support this, we curate a dataset of 500k samples across 12 task categories, labeled via a teacher-oracle pipeline to approximate Pareto-optimal compression scales. Extensive experiments across diverse architectures show that TCRR achieves a superior efficiency frontier, specifically reducing visual FLOPs by 40.9% and latency by 53.7% on Qwen3-VL-8B while preserving competitive performance. Further analysis of scaling behavior confirms that dynamically routing visual compression enables optimal resource allocation without modifying the MLLM backbone.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 36 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

34 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024FastVLM: Efficient Vision Encoding for Vision Language Models17 Dec
  2. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Models5 Dec
  3. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct

Other references · 31

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  8. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  9. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  11. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  12. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
19 more
  1. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  2. 2019Towards VQA Models That Can Readcited by 1,718
  3. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  4. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  5. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  6. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  7. 2022Token Merging: Your ViT But Fastercited by 1,021
  8. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  9. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  10. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  11. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  12. 2021InfographicVQAcited by 437
  13. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 332
  14. 2022Confident Adaptive Language Modelingcited by 310
  15. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
  16. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 244
  17. 2024MobileNetV4 -- Universal Models for the Mobile Ecosystemcited by 99
  18. 2024Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Datacited by 67
  19. 2025FineVision: Open Data Is All You Needcited by 53

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs
  2. 2026Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines
  3. 2025TokenCarve: Information-Preserving Visual Token Compression in Multimodal Large Language Models
  4. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
  5. 2025A Survey of Token Compression for Efficient Multimodal Large Language Models
  6. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
  7. 2026Stepwise Token Selection for Efficient Multimodal Large Language Models
  8. 2026FocusVTC: Efficient and High-Performance Visual Text Compression with Adaptive Resolution