mlsanityData to 7 Oct 2026

Towards VQA Models That Can Read

arXiv 1904.08920 ↗First posted 18 Apr 2019cs.CL cs.CV cs.LGcited by 1,696

Not on the map. Why?

Cited by 1,053 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Studies have shown that a dominant class of questions asked by visually impaired users on images of their surroundings involves reading text in the image. But today's VQA models can not read! Our paper takes a first step towards addressing this problem. First, we introduce a new "TextVQA" dataset to facilitate progress on this important problem. Existing datasets either have a small proportion of questions about text (e.g., the VQA dataset) or are too small (e.g., the VizWiz dataset). TextVQA contains 45,336 questions on 28,408 images that require reasoning about text to answer. Second, we introduce a novel model architecture that reads text in the image, reasons about it in the context of the image and the question, and predicts an answer which might be a deduction based on the text and the image or composed of the strings found in the image. Consequently, we call our approach Look, Read, Reason & Answer (LoRRA). We show that LoRRA outperforms existing state-of-the-art VQA models on our TextVQA dataset. We find that the gap between human performance and machine performance is significantly larger on TextVQA than on VQA 2.0, suggesting that TextVQA is well-suited to benchmark progress along directions complementary to VQA 2.0.

Key facts. Cited by 1,696 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 37; 2022: 54; 2023: 146; 2024: 448; 2025: 553; 2026: 458.

1,696 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  7. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  9. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  10. 2025Gemma 3 Technical Reportcited by 1,686
  11. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  12. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
38 more
  1. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  2. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  3. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  4. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  5. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  6. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  7. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  8. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  9. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  10. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  11. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  12. 2025Qwen2.5-Omni Technical Reportcited by 704
  13. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  14. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  15. 2023VILA: On Pre-training for Visual Language Modelscited by 686
  16. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  17. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  18. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
  19. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  20. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  21. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
  22. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  23. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  24. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  25. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  26. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  27. 2021InfographicVQAcited by 430
  28. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
  29. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
  30. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Modelscited by 394
  31. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 378
  32. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 361
  33. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
  34. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 356
  35. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
  36. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  37. 2024What matters when building vision-language models?cited by 321
  38. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321

50 most cited of 1,696 All 1,696 (JSON)