mlsanityData to 7 Oct 2026

On the Cognition of Visual Question Answering Models and Human Intelligence: A Comparative Study

arXiv 2310.02528 ↗First posted 4 Oct 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Visual Question Answering (VQA) is a challenging task that requires cross-modal understanding and reasoning of visual image and natural language question. To inspect the association of VQA models to human cognition, we designed a survey to record human thinking process and analyzed VQA models by comparing the outputs and attention maps with those of humans. We found that although the VQA models resemble human cognition in architecture and performs similarly with human on the recognition-level, they still struggle with cognitive inferences. The analysis of human thinking procedure serves to direct future research and introduce more cognitive capacity into modeling features and architectures.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 4 Oct 2023.

Builds on

29 of 42 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  8. 2015VQA: Visual Question Answeringcited by 2,489
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  10. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  11. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  12. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
17 more
  1. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  2. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 424
  3. 2015Visual7W: Grounded Question Answering in Imagescited by 315
  4. 2017Learning to Reason: End-to-End Module Networks for Visual Question Answeringcited by 125
  5. 2016Analyzing the Behavior of Visual Question Answering Modelscited by 119
  6. 2021A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Modelscited by 101
  7. 2021You Only Learn One Representation: Unified Network for Multiple Taskscited by 86
  8. 2019MUREL: Multimodal Relational Reasoning for Visual Question Answeringcited by 68
  9. 2015Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answeringcited by 66
  10. 2015Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sourcescited by 42
  11. 2017Exploring Human-like Attention Supervision in Visual Question Answeringcited by 36
  12. 2017Show, Ask, Attend, and Answer: A Strong Baseline For Visual Question Answeringcited by 31
  13. 2015Image Question Answering using Convolutional Neural Network with Dynamic Parameter Predictioncited by 26
  14. 2015Multimodal Convolutional Neural Networks for Matching Image and Sentencecited by 25
  15. 2016Zero-Shot Visual Question Answeringcited by 20
  16. 2018Question Type Guided Attention in Visual Question Answeringcited by 8
  17. 2020Generating Rationales in Visual Question Answeringcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The Quest for Visual Understanding: A Journey Through the Evolution of Visual Question Answering
  2. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithms
  3. 2025Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison
  4. 2025Visual question answering: from early developments to recent advances -- a survey
  5. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
  6. 2026Evaluating VQA in Vision Language Models using Cooperative Principles
  7. 2024Guiding Vision-Language Model Selection for Visual Question-Answering Across Tasks, Domains, and Knowledge Types
  8. 2025How Well Can Vison-Language Models Understand Humans' Intention? An Open-ended Theory of Mind Question Evaluation Benchmark