mlsanityData to 7 Oct 2026

Why Vision Fails as a Universal Bridge: Rectifying Modality Asynchrony in Multilingual MLLMs

arXiv 2608.15085 ↗First posted 15 Aug 2026cs.CLcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multimodal large language models (MLLMs) exhibit substantial performance degradation in non-English visual reasoning, despite the strong multilingual competence of their text-only backbones. While mechanistic evidence from text-only models suggests that non-English inputs are routed through an English-centric latent space, the multimodal implications of this phenomenon remain unexplored. Through rigorous mechanistic analysis, we identify the Ghost Anchor phenomenon: a temporal modality asynchrony where linguistic translation to the English semantic manifold completes in early layers, while visual semanticization remains immature. Consequently, visual signals are physically present yet functionally invisible during the early alignment window. To rectify this, we propose ANCHOR, a training framework employing Proactive Visual Anchoring (PVA) to accelerate early visual semantic emergence, ensuring visual representations proactively guide linguistic translation. Mechanistic interventions confirm that ANCHOR successfully restores the causal influence of visual signals during early translation. Furthermore, extensive experiments on XMMMU, MaXM, and CVQA demonstrate that ANCHOR consistently outperforms standard baselines, achieving robust visual reasoning across both fine-tuned and zero-shot languages.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 27 of its 34 references that are in the map. First posted on arXiv 15 Aug 2026.

Builds on

27 of 34 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set24 Oct
  2. 2025How Visual Representations Map to Language Feature Space in Multimodal LLMs13 Jun

Other references · 25

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  5. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  6. 2025OpenAI GPT-5 System Cardcited by 910
  7. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  8. 2024How do Large Language Models Handle Multilingualism?cited by 148
  9. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmarkcited by 118
  10. 2025Do Multilingual LLMs Think In English?cited by 75
  11. 2021xGQA: Cross-Lingual Visual Question Answeringcited by 72
  12. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
13 more
  1. 2021UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-trainingcited by 64
  2. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languagescited by 63
  3. 2022IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languagescited by 54
  4. 2021Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Modelscited by 41
  5. 2023mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMscited by 41
  6. 2024Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in?cited by 40
  7. 2026Beyond Language Modeling: An Exploration of Multimodal Pretrainingcited by 32
  8. 2025Aya Vision: Advancing the Frontier of Multilingual Multimodalitycited by 31
  9. 2025Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainabilitycited by 28
  10. 2024Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Modelscited by 22
  11. 2025Steering into New Embedding Spaces: Analyzing Cross-Lingual Alignment Induced by Model Interventions in Multilingual Language Modelscited by 10
  12. 2025MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluationcited by 4
  13. 2025When Alignment Hurts: Decoupling Representational Spaces in Multilingual Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Visual Instruction Tuning Aligns Modalities through Abstraction
  2. 2025Perceiving Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models
  3. 2025Towards Long-window Anchoring in Vision-Language Model Distillation
  4. 2026Causal Probing for Internal Visual Representations in Multimodal Large Language Models
  5. 2025Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMs
  6. 2026Mitigating Visual Degradation in MLLMs via Spatial-Spectral Visual Anchor Learning
  7. 2026Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention
  8. 2026Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Models