mlsanityData to 13 Oct 2026

MOVE: A Mixture-of-Vision-Encoders Approach for Domain-Focused Vision-Language Processing

arXiv 2502.15381 ↗First posted 21 Feb 2025cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal language models (MLMs) integrate visual and textual information by coupling a vision encoder with a large language model through the specific adapter. While existing approaches commonly rely on a single pre-trained vision encoder, there is a great variability of specialized encoders that can boost model's performance in distinct domains. In this work, we propose MOVE (Mixture of Vision Encoders) a simple yet effective approach to leverage multiple pre-trained encoders for specialized multimodal tasks. MOVE automatically routes inputs to the most appropriate encoder among candidates such as Unichat, InternViT, and Texify, thereby enhancing performance across a diverse set of benchmarks, including ChartQA, MMBench, and MMMU. Experimental results demonstrate that MOVE achieves competitive accuracy without incurring the complexities of image slicing for high-resolution images.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 38 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

28 of 38 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Rethinking the Mixture of Vision Encoders Paradigm for Enhanced Visual Understanding in Multimodal LLMs13 Jan
  2. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders28 Aug
  3. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models10 Jul
  4. 2024MoVA: Adapting Mixture of Vision Experts to Multimodal Context19 Apr
  5. 2024Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models5 Mar
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  7. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul

Other references · 21

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2023Visual Instruction Tuningcited by 7,827
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,363
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,463
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,068
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,044
  7. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,896
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,461
  9. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,419
  10. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,392
  11. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,296
  12. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,295
9 more
  1. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,202
  2. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 899
  3. 2016A Diagram Is Worth A Dozen Imagescited by 796
  4. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  5. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 273
  6. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 158
  7. 2023MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuningcited by 149
  8. 2023Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyondcited by 51
  9. 2023Delving Deeper into Data Scaling in Masked Image Modelingcited by 5

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Investigating Redundancy in Multimodal Large Language Models with Multiple Vision Encoderscited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2026A Strong Baseline for Evaluating Vision Encoders in Multimodal Large Language Models
  2. 2025MoVE-KD: Knowledge Distillation for VLMs with Mixture of Visual Encoders
  3. 2024Optimizing Vision-Language Interactions Through Decoder-Only Models
  4. 2024Harnessing Frozen Unimodal Encoders for Flexible Multimodal Alignment
  5. 2025Mixpert: Mitigating Multimodal Learning Conflicts with Efficient Mixture-of-Vision-Experts
  6. 2025Unified Multimodal Understanding via Byte-Pair Visual Encoding
  7. 2023A Survey of Vision-Language Pre-training from the Lens of Multimodal Machine Translation
  8. 2025Towards General Continuous Memory for Vision-Language Models