mlsanityData to 7 Oct 2026

MS-Resampler: Multi-Scope Visual Resampling for Efficient Multimodal LLMs

arXiv 2606.31383 ↗First posted 30 Jun 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal large language models (MLLMs) typically employ resampling-based projectors to transform dense visual features into a compact token sequence for language modeling. Most existing resamplers adopt a single, fixed aggregation scope via global cross-attention, which can blur fine-grained local evidence and limit the ability to capture both local details and global context within a fixed token budget. In this work, we propose MS-Resampler, a multi-scope visual resampling framework for MLLMs. MS-Resampler instantiates multiple scope-specific resamplers by injecting explicit spatial scope priors into the resampling attention, enabling each branch to aggregate visual information at a particular granularity from local to global. The outputs of these scope-specific resamplers are then adaptively fused to produce the final visual representations for language modeling. Extensive experiments on ten public multimodal benchmarks show that MS-Resampler consistently improves visual understanding and multimodal reasoning over conventional single-scope resamplers, while introducing only minimal computational overhead.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 40 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

32 of 40 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites25 Apr
  2. 2023Honeybee: Locality-enhanced Projector for Multimodal LLM11 Dec
  3. 2023Improved Baselines with Visual Instruction Tuning5 Oct

Other references · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  9. 2023Qwen Technical Reportcited by 3,273
  10. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  11. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  12. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 629
17 more
  1. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  2. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Modelscited by 394
  3. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 378
  4. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 283
  5. 2025DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Modelscited by 177
  6. 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Modelcited by 172
  7. 2024TokenPacker: Efficient Visual Projector for Multimodal LLMcited by 170
  8. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 164
  9. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters Morecited by 149
  10. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Modelscited by 125
  11. 2021MPViT: Multi-Path Vision Transformer for Dense Predictioncited by 122
  12. 2024Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Modelcited by 67
  13. 2024Matryoshka Query Transformer for Large Vision-Language Modelscited by 49
  14. 2024Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reductioncited by 28
  15. 2025VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMscited by 11
  16. 2025Delta-LLaVA: Base-then-Specialize Alignment for Token-Efficient Vision-Language Modelscited by 7
  17. 2024Spatial-Aware Efficient Projector for MLLMs via Multi-Layer Feature Aggregationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving Visual Token Reduction via Rectifying Distortions for Efficient Multimodal LLM Inference
  2. 2024The Revolution of Multimodal Large Language Models: A Survey
  3. 2025Visual Representation Alignment for Multimodal Large Language Models
  4. 2024LLaVA-KD: A Framework of Distilling Multimodal Large Language Models
  5. 2025How Visual Representations Map to Language Feature Space in Multimodal LLMs
  6. 2025Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Models
  7. 2023ChEF: A Comprehensive Evaluation Framework for Standardized Assessment of Multimodal Large Language Models
  8. 2025Zoom-Refine: Boosting High-Resolution Multimodal Understanding via Localized Zoom and Self-Refinement