mlsanityData to 14 Oct 2026

Mind the (Language) Gap: Towards Probing Numerical and Cross-Lingual Limits of LVLMs

arXiv 2508.17334 ↗First posted 24 Aug 2025cs.CV cs.AI cs.CLcited by 2

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

We introduce MMCRICBENCH-3K, a benchmark for Visual Question Answering (VQA) on cricket scorecards, designed to evaluate large vision-language models (LVLMs) on complex numerical and cross-lingual reasoning over semi-structured tabular images. MMCRICBENCH-3K comprises 1,463 synthetically generated scorecard images from ODI, T20, and Test formats, accompanied by 1,500 English QA pairs. It includes two subsets: MMCRICBENCH-E-1.5K, featuring English scorecards, and MMCRICBENCH-H-1.5K, containing visually similar Hindi scorecards, with all questions and answers kept in English to enable controlled cross-script evaluation. The task demands reasoning over structured numerical data, multi-image context, and implicit domain knowledge. Empirical results show that even state-of-the-art LVLMs, such as GPT-4o and Qwen2.5VL, struggle on the English subset despite it being their primary training language and exhibit a further drop in performance on the Hindi subset. This reveals key limitations in structure-aware visual text understanding, numerical reasoning, and cross-lingual generalization. The dataset is publicly available via Hugging Face at https://huggingface.co/datasets/DIALab/MMCricBench, to promote LVLM research in this direction.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 25 of its 41 references that are in the map. First posted on arXiv 24 Aug 2025.

Builds on

25 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Visual Text Matters: Improving Text-KVQA with Visual Text Entity Knowledge-aware Large Multimodal Assistant24 Oct

Other references · 24

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  5. 2019Towards VQA Models That Can Readcited by 1,718
  6. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  7. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  8. 2019Scene Text Visual Question Answeringcited by 331
  9. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 274
  10. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 237
  11. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 179
  12. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 174
12 more
  1. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 151
  2. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 114
  3. 2023DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understandingcited by 106
  4. 2020On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answeringcited by 78
  5. 2023UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understandingcited by 74
  6. 2024TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergycited by 71
  7. 2024TableVQA-Bench: A Visual Question Answering Benchmark on Multiple Table Domainscited by 69
  8. 2024Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmarkcited by 55
  9. 2024Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMscited by 40
  10. 2020Visual Question Answering on Image Setscited by 26
  11. 2024NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalizationcited by 16
  12. 2024TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoningcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026INDOTABVQA: A Benchmark for Cross-Lingual Table Understanding in Bahasa Indonesia Documentscited by 0
  2. 2026Evaluating Cross-lingual Knowledge Consistency in Code-Mixed vis-a-vis Indian Languages using IndicKLARcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025GanitBench: A bi-lingual benchmark for evaluating mathematical reasoning in Vision Language Models
  2. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs
  3. 2025HinTel-AlignBench: A Framework and Benchmark for Hindi-Telugu with English-Aligned Samples
  4. 2024Cross-Lingual Text-Rich Visual Comprehension: An Information Theory Perspective
  5. 2025MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space
  6. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
  7. 2024ChitroJera: A Regionally Relevant Visual Question Answering Dataset for Bangla
  8. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark