mlsanityData to 7 Oct 2026

Scope: Selective Cross-modal Orchestration of Visual Perception Experts

arXiv 2510.12974 ↗First posted 14 Oct 2025cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models (VLMs) benefit from multiple vision encoders, but naively stacking them yields diminishing returns while multiplying inference costs. We propose SCOPE, a Mixture-of-Encoders (MoEnc) framework that dynamically selects one specialized encoder per image-text pair via instance-level routing, unlike token-level routing in traditional MoE. SCOPE maintains a shared encoder and a pool of routed encoders. A lightweight router uses cross-attention between text prompts and shared visual features to select the optimal encoder from the routed encoders. To train this router, we introduce dual entropy regularization with auxiliary losses to balance dataset-level load distribution with instance-level routing confidence. Remarkably, SCOPE with one shared plus one routed encoder outperforms models using all four extra encoders simultaneously, while reducing compute by 24-49%. This demonstrates that intelligent encoder selection beats brute-force aggregation, challenging the prevailing paradigm in multi-encoder VLMs.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 43 references that are in the map. First posted on arXiv 14 Oct 2025.

Builds on

36 of 43 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024VCR: A Task for Pixel-Level Complex Reasoning in Vision Language Models via Restoring Occluded Text10 Jun
  2. 2024BRAVE: Broadening the visual encoding of vision-language models10 Apr
  3. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models27 Mar
  4. 2023Prismer: A Vision-Language Model with Multi-Task Experts4 Mar

Other references · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Segment Anythingcited by 8,158
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2025Qwen2.5-VL Technical Reportcited by 5,418
  7. 2024GPT-4o System Cardcited by 4,467
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  9. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  10. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  11. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  12. 2024Mixtral of Expertscited by 1,785
20 more
  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  2. 2025DINOv3cited by 1,297
  3. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  4. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  5. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  6. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  7. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  8. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  9. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 283
  10. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  11. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoderscited by 137
  12. 2025Ovis2.5 Technical Reportcited by 103
  13. 2024Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Modelscited by 97
  14. 2024MoVA: Adapting Mixture of Vision Experts to Multimodal Contextcited by 82
  15. 2024LocCa: Visual Pretraining with Location-aware Captionerscited by 35
  16. 2024ConvLLaVA: Hierarchical Backbones as Visual Encoder for Large Multimodal Modelscited by 26
  17. 2024BigDocs: An Open Dataset for Training Multimodal Models on Document and Code Taskscited by 10
  18. 2025Rethinking the Mixture of Vision Encoders Paradigm for Enhanced Visual Understanding in Multimodal LLMscited by 6
  19. 2025Investigating Redundancy in Multimodal Large Language Models with Multiple Vision Encoderscited by 5
  20. 2025METEOR: Multi-Encoder Collaborative Token Pruning for Efficient Vision Language Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMscited by 0
  2. 2026When Vision Meets Graphs: A Survey on Graph Reasoning and Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Mixture of Layers: Dynamic Layer Routing for Visual Reasoning
  2. 2026CoME-VL: Scaling Complementary Multi-Encoder Vision-Language Learning
  3. 2026Rethinking Model Selection in VLM Through the Lens of Gromov-Wasserstein Distance
  4. 2025Mixpert: Mitigating Multimodal Learning Conflicts with Efficient Mixture-of-Vision-Experts
  5. 2026PETR: Prompt Ensembling with Training-free Routing for Vision-Language Models
  6. 2026ConsensusDrop: Fusing Visual and Cross-Modal Saliency for Efficient Vision Language Models
  7. 2024Unveiling Encoder-Free Vision-Language Models
  8. 2026An Effective Router for Vision-Language Model Selection