mlsanityData to 7 Oct 2026

Ask Your Neurons: A Neural-based Approach to Answering Questions about Images

arXiv 1505.01121 ↗First posted 5 May 2015cs.CV cs.AI cs.CLcited by 51

Not on the map. Why?

Abstract (arXiv)

We address a question answering task on real-world images that is set up as a Visual Turing Test. By combining latest advances in image representation and natural language processing, we propose Neural-Image-QA, an end-to-end formulation to this problem for which all parts are trained jointly. In contrast to previous efforts, we are facing a multi-modal problem where the language output (answer) is conditioned on visual and natural language input (image and question). Our approach Neural-Image-QA doubles the performance of the previous best approach on this problem. We provide additional insights into the problem by analyzing how much information is contained only in the language part for which we provide a new human baseline. To study human consensus, which is related to the ambiguities inherent in this challenging task, we propose two novel metrics and collect additional answers which extends the original DAQUAR dataset to DAQUAR-Consensus.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 May 2015.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 13; 2022: 11; 2023: 11; 2024: 9; 2025: 7.

51 papers.

Sort:

By area:

  1. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmarkcited by 118
  2. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  3. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  4. 2021Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answeringcited by 62
  5. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Modelscited by 28
  6. 2022Video-Text Representation Learning via Differentiable Weak Temporal Alignmentcited by 24
  7. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  8. 2023Cross-modal Active Complementary Learning with Self-refining Correspondencecited by 22
  9. 2024LOVA3: Learning to Visual Question Answering, Asking and Assessmentcited by 21
  10. 2021Multimodal Integration of Human-Like Attention in Visual Question Answeringcited by 20
  11. 2021AdaVQA: Overcoming Language Priors with Adapted Margin Cosine Losscited by 18
  12. 2021Contextualized Keyword Representations for Multi-modal Retinal Image Captioningcited by 16
38 more
  1. 2021Longer Version for "Deep Context-Encoding Network for Retinal Image Captioning"cited by 16
  2. 2021VQA-MHUG: A Gaze Dataset to Study Multimodal Neural Attention in Visual Question Answeringcited by 16
  3. 2022Multimodal Conversational AI: A Survey of Datasets and Approachescited by 16
  4. 2021Dealing with Missing Modalities in the Visual Question Answer-Difference Prediction Task through Knowledge Distillationcited by 14
  5. 2021Cognitive Visual Commonsense Reasoning Using Dynamic Working Memorycited by 14
  6. 2022VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challengescited by 14
  7. 2021Interpretable Visual Understanding with Cognitive Attention Networkcited by 12
  8. 2021A survey on VQA_Datasets and Approachescited by 11
  9. 2022Joint Answering and Explanation for Visual Commonsense Reasoningcited by 11
  10. 2023Answer Mining from a Pool of Images: Towards Retrieval-Based Visual Question Answeringcited by 10
  11. 2025ExDDV: A New Dataset for Explainable Deepfake Detection in Videocited by 10
  12. 2023Improving Visual Question Answering Models through Robustness Analysis and In-Context Learning with a Chain of Basic Questionscited by 9
  13. 2022From Pixels to Objects: Cubic Visual Attention for Visual Question Answeringcited by 7
  14. 2021A Neural Question Answering System for Basic Questions about Subroutinescited by 5
  15. 2022AlignVE: Visual Entailment Recognition Based on Alignment Relationscited by 5
  16. 2023Towards AGI in Computer Vision: Lessons Learned from GPT and Large Language Modelscited by 5
  17. 2024Text-Guided Coarse-to-Fine Fusion Network for Robust Remote Sensing Visual Question Answeringcited by 5
  18. 2025IMoRe: Implicit Program-Guided Reasoning for Human Motion Q&Acited by 5
  19. 2025Using Large Language Models for education managements in Vietnamese with low resourcescited by 4
  20. 2023LOIS: Looking Out of Instance Semantics for Visual Question Answeringcited by 3
  21. 2023Emergent Communication in Interactive Sketch Question Answeringcited by 3
  22. 2024Survey of Natural Language Processing for Education: Taxonomy, Systematic Review, and Future Trendscited by 3
  23. 2025Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQAcited by 3
  24. 2025RoadSceneVQA: Benchmarking Visual Question Answering in Roadside Perception Systems for Intelligent Transportation Systemcited by 3
  25. 2021Towards Solving Multimodal Comprehensioncited by 2
  26. 2023Multimodal Prompt Retrieval for Generative Visual Question Answeringcited by 2
  27. 2023Multi-Clue Reasoning with Memory Augmentation for Knowledge-based Visual Question Answeringcited by 2
  28. 2025A Curriculum Learning Approach to Reinforcement Learning: Leveraging RAG for Multimodal Question Answeringcited by 2
  29. 2022Learning to embed semantic similarity for joint image-text retrievalcited by 1
  30. 2024AdaCoder: Adaptive Prompt Compression for Programmatic Visual Question Answeringcited by 1
  31. 2024Pyramid Coder: Hierarchical Code Generator for Compositional Visual Question Answeringcited by 1
  32. 2025Light as Deception: GPT-driven Natural Relighting Against Vision-Language Pre-training Modelscited by 1
  33. 2021Contrastive Learning of Visual-Semantic Embeddingscited by 0
  34. 2022Attention Mechanism based Cognition-level Scene Understandingcited by 0
  35. 2022EBMs vs. CL: Exploring Self-Supervised Visual Pretraining for Visual Question Answeringcited by 0
  36. 2023Analysis of Visual Question Answering Algorithms with attention modelcited by 0
  37. 2023Knowledge Detection by Relevant Question and Image Attributes in Visual Question Answeringcited by 0
  38. 2024Enhanced Textual Feature Extraction for Visual Question Answering: A Simple Convolutional Approachcited by 0

50 most cited of 51 All 51 (JSON)