mlsanityData to 14 Oct 2026

MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning

arXiv 2607.24424 ↗First posted 27 Jul 2026cs.CVcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features can discard text, local attributes, and spatial relationships, while high-resolution inputs substantially increase context length and inference latency. We introduce method, a Multi-scale Adaptive Vision Encoder. method uses position-dependent gates to fuse shallow, intermediate, and deep features from a vision Transformer, preserving global semantics while enhancing edges, text, and local structure. It then performs question-conditioned token routing according to question relevance, local information content, global semantics, and spatial coverage, with a token budget that adapts to image complexity. To mitigate compression loss, we further introduce full-to-compressed representation distillation and a spatial diversity regularizer. In an illustrative simulation under a unified 7B language-model framework, method reduces the average number of SigLIP-SO400M visual tokens from 729 to 146 (approximately 80.0%) and improves the mean score on VQAv2, GQA, TextVQA, ScienceQA-IMG, and MMBench by 2.2 percentage points, while reducing single-image time to first token from 228,ms to 129,ms. We provide the full model design and evaluation protocol. All reported numbers currently serve only as placeholders for paper organization and experimental design; formal claims require real training runs, independent replications, and official benchmark evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 52 of its 74 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

52 of 74 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
  2. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
  3. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep

Other references · 49

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  11. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
37 more
  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  2. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  3. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  4. 2019Towards VQA Models That Can Readcited by 1,718
  5. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  6. 2022Token Merging: Your ViT But Fastercited by 1,021
  7. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  8. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 391
  9. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 362
  10. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 229
  11. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  12. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 197
  13. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 195
  14. 2024TokenPacker: Efficient Visual Projector for Multimodal LLMcited by 172
  15. 2024FastVLM: Efficient Vision Encoding for Vision Language Modelscited by 77
  16. 2023CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformerscited by 44
  17. 2025Cognitive Memory in Large Language Modelscited by 29
  18. 2023Continual Learning for Image Segmentation with Dynamic Querycited by 12
  19. 2025DynRsl-VLM: Enhancing Autonomous Driving Perception with Dynamic Resolution Vision-Language Modelscited by 12
  20. 2025GeoLocSFT: Efficient Visual Geolocation via Supervised Fine-Tuning of Multimodal Foundation Modelscited by 9
  21. 2025LLM-CoT Enhanced Graph Neural Recommendation with Harmonized Group Policy Optimizationcited by 6
  22. 2025RecLLM-R1: A Two-Stage Training Paradigm with Reinforcement Learning and Chain-of-Thought v1cited by 6
  23. 2024PAR: Prompt-Aware Token Reduction Method for Efficient Large Multimodal Modelscited by 5
  24. 2023End-to-end Remote Sensing Change Detection of Unregistered Bi-temporal Images for Natural Disasterscited by 4
  25. 2024Boosting General Trimap-free Matting in the Real-World Imagecited by 4
  26. 2024Edge-guided and Class-balanced Active Learning for Semantic Segmentation of Aerial Imagescited by 4
  27. 2025KV-Efficient VLA: A Method to Speed up Vision Language Models with RNN-Gated Chunked KV Cachecited by 4
  28. 2024The Binary Quantized Neural Network for Dense Prediction via Specially Designed Upsampling and Attentioncited by 3
  29. 2024Lifelong Learning and Selective Forgetting via Contrastive Strategycited by 3
  30. 2024Organizing Background to Explore Latent Classes for Incremental Few-shot Semantic Segmentationcited by 3
  31. 2025Synthetic Lung X-ray Generation through Cross-Attention and Affinity Transformationcited by 3
  32. 2025GMM-Based Comprehensive Feature Extraction and Relative Distance Preservation For Few-Shot Cross-Modal Retrievalcited by 3
  33. 2025GDGS: 3D Gaussian Splatting Via Geometry-Guided Initialization And Dynamic Density Controlcited by 2
  34. 2025A Global-Local Cross-Attention Network for Ultra-high Resolution Remote Sensing Image Semantic Segmentationcited by 1
  35. 2025Building Lightweight Semantic Segmentation Models for Aerial Images Using Dual Relation Distillationcited by 1
  36. 2025NeuroVoxel-LM: Language-Aligned 3D Perception via Dynamic Voxelization and Meta-Embeddingcited by 1
  37. 2025RAH-VLA: Resolution-Adaptive Hierarchical Vision-Language Alignment for Multimodal Remote Sensing Understandingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition
  2. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
  3. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
  4. 2024Recoverable Compression: A Multimodal Vision Token Recovery Mechanism Guided by Text Information
  5. 2026Rethinking Token Reduction for Large Vision-Language Models
  6. 2025Token Sequence Compression for Efficient Multimodal Computing
  7. 2026Mema: Memory-Augmented Adapter for Enhanced Vision-Language Understanding
  8. 2025Efficient Vision-Language Reasoning via Adaptive Token Pruning