mlsanityData to 7 Oct 2026

Scene Text Visual Question Answering

arXiv 1905.13648 ↗First posted 31 May 2019cs.CVcited by 327

Not on the map. Why?

Cited by 143 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Current visual question answering datasets do not consider the rich semantic information conveyed by text within an image. In this work, we present a new dataset, ST-VQA, that aims to highlight the importance of exploiting high-level semantic information present in images as textual cues in the VQA process. We use this dataset to define a series of tasks of increasing difficulty for which reading the scene text in the context provided by the visual information is necessary to reason and generate an appropriate answer. We propose a new evaluation metric for these tasks to account both for reasoning errors as well as shortcomings of the text recognition module. In addition we put forward a series of baseline methods, which provide further insight to the newly released dataset, and set the scene for further research.

Key facts. Cited by 327 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 29; 2023: 50; 2024: 87; 2025: 80; 2026: 59.

327 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  3. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  4. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  5. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  6. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  7. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  8. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  9. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  10. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  11. 2021InfographicVQAcited by 430
  12. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 356
38 more
  1. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
  2. 2024What matters when building vision-language models?cited by 321
  3. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  4. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
  5. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
  6. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  7. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 238
  8. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 226
  9. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 214
  10. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  11. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  12. 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 190
  13. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 173
  14. 2023MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learningcited by 171
  15. 2021TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene textcited by 170
  16. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  17. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 165
  18. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDcited by 156
  19. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
  20. 2023UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Modelcited by 143
  21. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
  22. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 136
  23. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoningcited by 134
  24. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  25. 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuningcited by 126
  26. 2023PaLI-3 Vision Language Models: Smaller, Faster, Strongercited by 125
  27. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 125
  28. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 124
  29. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  30. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 110
  31. 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answeringcited by 105
  32. 2022Hierarchical multimodal transformers for Multi-Page DocVQAcited by 102
  33. 2022Towards End-to-End Unified Scene Text Detection and Layout Analysiscited by 96
  34. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  35. 2024MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answeringcited by 93
  36. 2024The All-Seeing Project V2: Towards General Relation Comprehension of the Open Worldcited by 90
  37. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  38. 2024MoVA: Adapting Mixture of Vision Experts to Multimodal Contextcited by 82

50 most cited of 327 All 327 (JSON)