mlsanityData to 7 Oct 2026

Visual7W: Grounded Question Answering in Images

arXiv 1511.03416 ↗First posted 11 Nov 2015cs.CV cs.LG cs.NEcited by 315

Not on the map. Why?

Cited by 108 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We have seen great progress in basic perceptual tasks such as object recognition and detection. However, AI models still fail to match humans in high-level vision tasks due to the lack of capacities for deeper reasoning. Recently the new task of visual question answering (QA) has been proposed to evaluate a model's capacity for deep image understanding. Previous works have established a loose, global association between QA sentences and images. However, many questions and answers, in practice, relate to local regions in the images. We establish a semantic link between textual descriptions and image regions by object-level grounding. It enables a new type of QA with visual answers, in addition to textual answers used in previous work. We study the visual QA tasks in a grounded setting with a large collection of 7W multiple-choice QA pairs. Furthermore, we evaluate human performance and several baseline models on the QA tasks. Finally, we propose a novel LSTM model with spatial attention to tackle the 7W QA tasks.

Key facts. Cited by 315 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 44; 2022: 58; 2023: 54; 2024: 64; 2025: 62; 2026: 33.

315 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  3. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  4. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  5. 2024Yi: Open Foundation Models by 01.AIcited by 706
  6. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  7. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  8. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  9. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  10. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  11. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  12. 2024What matters when building vision-language models?cited by 321
38 more
  1. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  2. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 300
  3. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 277
  4. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
  5. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 264
  6. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 262
  7. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  8. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  9. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  10. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 203
  11. 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 175
  12. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  13. 2023ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Promptscited by 160
  14. 2021Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphscited by 144
  15. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoderscited by 137
  16. 2021VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomenacited by 134
  17. 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 133
  18. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 125
  19. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 124
  20. 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
  21. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmarkcited by 118
  22. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  23. 2023The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open Worldcited by 114
  24. 2021Towards artificial general intelligence via a multimodal foundation modelcited by 112
  25. 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 112
  26. 2022MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuningcited by 109
  27. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  28. 2024Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Wantcited by 98
  29. 2024MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answeringcited by 93
  30. 2022Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoningcited by 90
  31. 2024The All-Seeing Project V2: Towards General Relation Comprehension of the Open Worldcited by 90
  32. 2022Clotho-AQA: A Crowdsourced Dataset for Audio Question Answeringcited by 84
  33. 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 81
  34. 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Modelscited by 80
  35. 2021Multi-Modal Answer Validation for Knowledge-Based VQAcited by 78
  36. 2022Multi-Modal Knowledge Graph Construction and Application: A Surveycited by 74
  37. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  38. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 73

50 most cited of 315 All 315 (JSON)