# Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts

arXiv: https://arxiv.org/abs/2604.08541 · HTML: https://mlsanity.com/paper/2604.08541/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts.
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Placed in

- Mixture-of-experts / [Multimodal MoE models](https://mlsanity.com/moe/multimodal-moe-models/)

## Builds on (most cited first, 30 of 30)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset](https://mlsanity.com/paper/2402.14804/) (2024)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Do Llamas Work in English? On the Latent Language of Multilingual Transformers](https://mlsanity.com/paper/2402.10588/) (2024)
- [OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models](https://mlsanity.com/paper/2402.01739/) (2024)
- [UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning](https://mlsanity.com/paper/2503.21620/) (2025)
- [Linearly Mapping from Image to Text Space](https://mlsanity.com/paper/2209.15162/) (2022)
- [SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization](https://mlsanity.com/paper/2604.02268/) (2026)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [Harder Tasks Need More Experts: Dynamic Routing in MoE Models](https://mlsanity.com/paper/2403.07652/) (2024)
- [The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities](https://mlsanity.com/paper/2411.04986/) (2024)
- [xVerify: Efficient Answer Verifier for Reasoning Model Evaluations](https://mlsanity.com/paper/2504.10481/) (2025)
- [Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models](https://mlsanity.com/paper/2501.11873/) (2025)
- [A Closer Look into Mixture-of-Experts in Large Language Models](https://mlsanity.com/paper/2406.18219/) (2024)
- [UI-S1: Advancing GUI Automation via Semi-online Reinforcement Learning](https://mlsanity.com/paper/2509.11543/) (2025)
- [Multilingual Routing in Mixture-of-Experts](https://mlsanity.com/paper/2510.04694/) (2025)
- [Steering MoE LLMs via Expert (De)Activation](https://mlsanity.com/paper/2509.09660/) (2025)
- [Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs](https://mlsanity.com/paper/2405.16700/) (2024)
- [Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast](https://mlsanity.com/paper/2405.14507/) (2024)
- [SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification](https://mlsanity.com/paper/2506.17368/) (2025)
- [Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training](https://mlsanity.com/paper/2505.14681/) (2025)
- [Vision-Language Models Create Cross-Modal Task Representations](https://mlsanity.com/paper/2410.22330/) (2024)
- [EasySteer: A Unified Framework for High-Performance and Extensible LLM Steering](https://mlsanity.com/paper/2509.25175/) (2025)
- [R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts](https://mlsanity.com/paper/2502.20395/) (2025)
- [GSM8K-V: Can Vision Language Models Solve Grade School Math Word Problems in Visual Contexts](https://mlsanity.com/paper/2509.25160/) (2025)
- [Same Content, Different Answers: Cross-Modal Inconsistency in MLLMs](https://mlsanity.com/paper/2512.08923/) (2025)
- [D2HScore: Reasoning-Aware Hallucination Detection via Semantic Breadth and Depth Analysis in LLMs](https://mlsanity.com/paper/2509.11569/) (2025)

## Built on by (most cited first, 9 of 9)

- [H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation](https://mlsanity.com/paper/2607.02592/) (2026)
- [MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understanding](https://mlsanity.com/paper/2608.04587/) (2026)
- [Yuvion VL: A Multimodal Foundation Model for Adversarial Content and AI Safety](https://mlsanity.com/paper/2606.25034/) (2026)
- [Counting the Cost of War Under Satellite Embargo: Zero-Shot Estimation of Impacted Infrastructure](https://mlsanity.com/paper/2608.00119/) (2026)
- [EvoHarmBench: Breaking Content Moderation with Iterative Human-Like Evasion](https://mlsanity.com/paper/2608.27844/) (2026)
- [A Persistent State for Auditable Mixture-of-Experts Routing](https://mlsanity.com/paper/2609.34634/) (2026)
- [DecoMoE: Decoupling Visual Propagation and Expert Computation for Efficient Multimodal MoE Inference](https://mlsanity.com/paper/2609.38823/) (2026)
- [Cross-Lingual Alignment for Decoder-Only Models using MoE Routers](https://mlsanity.com/paper/2610.01921/) (2026)
- [Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation](https://mlsanity.com/paper/2610.02123/) (2026)

## Abstract (arXiv)

Multimodal Mixture-of-Experts (MoE) models have achieved remarkable performance on vision-language tasks. However, we identify a puzzling phenomenon termed Seeing but Not Thinking: models accurately perceive image content yet fail in subsequent reasoning, while correctly solving identical problems presented as pure text. Through systematic analysis, we first verify that cross-modal semantic sharing exists in MoE architectures, ruling out semantic alignment failure as the sole explanation. We then reveal that visual experts and domain experts exhibit layer-wise separation, with image inputs inducing significant routing divergence from text inputs in middle layers where domain experts concentrate. Based on these findings, we propose the Routing Distraction hypothesis: when processing visual inputs, the routing mechanism fails to adequately activate task-relevant reasoning experts. To validate this hypothesis, we design a routing-guided intervention method that enhances domain expert activation. Experiments on three multimodal MoE models across six benchmarks demonstrate consistent improvements, with gains of up to 3.17% on complex visual reasoning tasks. Our analysis further reveals that domain expert identification locates cognitive functions rather than sample-specific solutions, enabling effective transfer across tasks with different information structures.