mlsanityData to 7 Oct 2026

A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input

arXiv 1410.0210 ↗First posted 1 Oct 2014cs.AI cs.CL cs.CVcited by 112

Not on the map. Why?

Abstract (arXiv)

We propose a method for automatically answering questions about images by bringing together recent advances from natural language processing and computer vision. We combine discrete reasoning with uncertain predictions by a multi-world approach that represents uncertainty about the perceived world in a bayesian framework. Our approach can handle human questions of high complexity about realistic scenes and replies with range of answer like counts, object classes, instances and lists of them. The system is directly trained from question-answer pairs. We establish a first benchmark for this task that can be seen as a modern attempt at a visual turing test.

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Oct 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 25; 2023: 23; 2024: 24; 2025: 10; 2026: 7.

112 papers.

Sort:

By area:

  1. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  2. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  3. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  4. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  5. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoningcited by 119
  6. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmarkcited by 118
  7. 2023Encyclopedic VQA: Visual questions about detailed properties of fine-grained categoriescited by 105
  8. 2022Clotho-AQA: A Crowdsourced Dataset for Audio Question Answeringcited by 84
  9. 2022MapQA: A Dataset for Question Answering on Choropleth Mapscited by 82
  10. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  11. 20213D Question Answeringcited by 41
  12. 2022Gender and Racial Bias in Visual Question Answering Datasetscited by 39
38 more
  1. 2022SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answeringcited by 38
  2. 2022From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Datacited by 38
  3. 2022Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matricescited by 37
  4. 2023FunQA: Towards Surprising Video Comprehensioncited by 35
  5. 2021Select, Substitute, Search: A New Benchmark for Knowledge-Augmented Visual Question Answeringcited by 33
  6. 2022A Review on Methods and Applications in Multimodal Deep Learningcited by 29
  7. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Modelscited by 28
  8. 2021Recent Advances and Trends in Multimodal Deep Learning: A Reviewcited by 26
  9. 2022GRIT: General Robust Image Task Benchmarkcited by 26
  10. 2022Towards Explainable 3D Grounded Visual Question Answering: A New Benchmark and Strong Baselinecited by 25
  11. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  12. 2021Measuring and Improving BERT's Mathematical Abilities by Predicting the Order of Reasoningcited by 23
  13. 2023Robust Visual Question Answering: Datasets, Methods, and Future Challengescited by 23
  14. 2023UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Modelscited by 23
  15. 2024Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Modelcited by 22
  16. 2021Multimodal Integration of Human-Like Attention in Visual Question Answeringcited by 20
  17. 2022Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalizationcited by 20
  18. 2021MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Groundingcited by 19
  19. 2022Locate before Answering: Answer Guided Question Localization for Video Question Answeringcited by 19
  20. 2023WebVLN: Vision-and-Language Navigation on Websitescited by 19
  21. 2021Contextualized Keyword Representations for Multi-modal Retinal Image Captioningcited by 16
  22. 2021VQA-MHUG: A Gaze Dataset to Study Multimodal Neural Attention in Visual Question Answeringcited by 16
  23. 2023Foundation Model is Efficient Multimodal Multitask Model Selectorcited by 16
  24. 2024Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Modelscited by 16
  25. 2022A Neuro-Symbolic ASP Pipeline for Visual Question Answeringcited by 14
  26. 2022VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challengescited by 14
  27. 2021DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialoguecited by 13
  28. 2024CausalChaos! Dataset for Comprehensive Causal Action Question Answering Over Longer Causal Chains Grounded in Dynamic Visual Scenescited by 13
  29. 2026MMR-Bench: A Comprehensive Benchmark for Multimodal LLM Routingcited by 13
  30. 2021Selective Replay Enhances Learning in Online Continual Analogical Reasoningcited by 12
  31. 2023HaVQA: A Dataset for Visual Question Answering and Multimodal Research in Hausa Languagecited by 12
  32. 2024Grounding Language Models for Visual Entity Recognitioncited by 12
  33. 2021A survey on VQA_Datasets and Approachescited by 11
  34. 2021Weighted Intersection over Union (wIoU) for Evaluating Image Segmentationcited by 11
  35. 2021Classification-Regression for Chart Comprehensioncited by 11
  36. 2021'Just because you are right, doesn't mean I am wrong': Overcoming a Bottleneck in the Development and Evaluation of Open-Ended Visual Question Answering (VQA) Taskscited by 10
  37. 2023Multimodal Graph Transformer for Multimodal Question Answeringcited by 10
  38. 2023S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learningcited by 10

50 most cited of 112 All 112 (JSON)