mlsanityData to 7 Oct 2026

Do Multilingual VLMs Reason Equally? A Cross-Lingual Visual Reasoning Audit for Indian Languages

arXiv 2603.26742 ↗First posted 23 Mar 2026cs.CL cs.LGcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Vision-language models score well on mathematical, scientific, and spatial reasoning benchmarks, yet these evaluations are overwhelmingly English. I present the first cross-lingual visual reasoning audit for Indian languages. 980 questions from MathVista, ScienceQA, and MMMU are translated into Hindi, Tamil, Telugu, Bengali, Kannada, and Marathi using IndicTrans2, with Gemini 2.0 Flash cross-verification on 50 samples per language (inter-translator agreement 0.79-0.84). Eight VLMs, from 7B open-source models to GPT-4o, are evaluated across all seven languages, yielding 68,600 inference records that include text-only and chain-of-thought ablations. I find accuracy drops of 9.8-25 percentage points when switching from English to an Indian language, with Dravidian languages suffering up to 13.2 pp more than Indo-Aryan. Chain-of-thought prompting degrades Bengali (-14.4 pp) and Kannada (-11.4 pp) rather than helping, exposing English-centric reasoning chains. Aya-Vision-8B, built for 23 languages, still drops 28.5 pp on Dravidian scripts; multilingual pretraining alone does not transfer visual reasoning. I release the translated benchmark and all model outputs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 23 of its 27 references that are in the map. First posted on arXiv 23 Mar 2026.

Builds on

23 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026VIRAASAT: Traversing Novel Paths for Indian Cultural Reasoning20 Feb
  2. 2025The Reasoning Lingua Franca: A Double-Edged Sword for Multilingual AI23 Oct

Other references · 21

  1. 2025Qwen2.5-VL Technical Reportcited by 5,418
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  6. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  7. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  8. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  9. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  10. 2021Visually Grounded Reasoning across Languages and Culturescited by 188
  11. 2023IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languagescited by 149
  12. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmarkcited by 118
9 more
  1. 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 99
  2. 2021xGQA: Cross-Lingual Visual Question Answeringcited by 72
  3. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languagescited by 63
  4. 2024All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languagescited by 55
  5. 2025Aya Vision: Advancing the Frontier of Multilingual Multimodalitycited by 31
  6. 2025DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culturecited by 13
  7. 2025Chitrarth: Bridging Vision and Language for a Billion Peoplecited by 10
  8. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMscited by 7
  9. 2025VLURes: Benchmarking VLM Visual and Linguistic Understanding in Low-Resource Languagescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025mmJEE-Eval: A Bilingual Multimodal Benchmark for Evaluating Scientific Reasoning in Vision-Language Models
  2. 2025GanitBench: A bi-lingual benchmark for evaluating mathematical reasoning in Vision Language Models
  3. 2026Beyond 'One Language, One Script': Quantifying Orthographic Bias in Multilingual VLMs with PuMVR
  4. 2026Not Truly Multilingual: Script Consistency as a Missing Dimension in VLM Evaluation
  5. 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
  6. 2024Constructing Multilingual Visual-Text Datasets Revealing Visual Multilingual Ability of Vision Language Models
  7. 2024What Is Missing in Multilingual Visual Reasoning and How to Fix It
  8. 2025HinTel-AlignBench: A Framework and Benchmark for Hindi-Telugu with English-Aligned Samples