mlsanityData to 7 Oct 2026

Visual Question Answering on Image Sets

arXiv 2008.11976 ↗First posted 27 Aug 2020cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

We introduce the task of Image-Set Visual Question Answering (ISVQA), which generalizes the commonly studied single-image VQA problem to multi-image settings. Taking a natural language question and a set of images as input, it aims to answer the question based on the content of the images. The questions can be about objects and relationships in one or more images or about the entire scene depicted by the image set. To enable research in this new topic, we introduce two ISVQA datasets - indoor and outdoor scenes. They simulate the real-world scenarios of indoor image collections and multiple car-mounted cameras, respectively. The indoor-scene dataset contains 91,479 human annotated questions for 48,138 image sets, and the outdoor-scene dataset has 49,617 questions for 12,746 image sets. We analyze the properties of the two datasets, including question-and-answer distributions, types of questions, biases in dataset, and question-image dependencies. We also build new baseline models to investigate new research challenges in ISVQA.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 2; 2023: 5; 2024: 9; 2025: 5; 2026: 2.

26 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  3. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 178
  4. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 175
  5. 2023X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoningcited by 78
  6. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 76
  7. 2024Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmarkcited by 53
  8. 20213D Question Answeringcited by 41
  9. 2022Image Retrieval from Contextual Descriptionscited by 26
  10. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  11. 2021Iterative Shrinking for Referring Expression Grounding Using Deep Reinforcement Learningcited by 20
  12. 2024VipAct: Visual-Perception Enhancement via Specialized VLM Agent Collaboration and Tool-usecited by 17
14 more
  1. 2024Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documentscited by 17
  2. 2022Fully and Weakly Supervised Referring Expression Segmentation with End-to-End Learningcited by 14
  3. 2024Step Differences in Instructional Videocited by 11
  4. 2023Answer Mining from a Pool of Images: Towards Retrieval-Based Visual Question Answeringcited by 10
  5. 2024JDocQA: Japanese Document Question Answering Dataset for Generative Language Modelscited by 9
  6. 2025ToFu: Visual Tokens Reduction via Fusion for Multi-modal, Multi-patch, Multi-image Taskcited by 2
  7. 2025Co-VisiON: Co-Visibility ReasONing on Sparse Image Sets of Indoor Scenescited by 2
  8. 2025Mind the (Language) Gap: Towards Probing Numerical and Cross-Lingual Limits of LVLMscited by 2
  9. 2021Analysis on Image Set Visual Question Answeringcited by 1
  10. 2024Audiopedia: Audio QA with Knowledgecited by 1
  11. 2025Natural Language Generation from Visual Events: State-of-the-Art and Key Open Questionscited by 1
  12. 2025Contra4: Evaluating Contrastive Cross-Modal Reasoning in Audio, Video, Image, and 3Dcited by 0
  13. 2026Where is the multimodal goal post? On the Ability of Foundation Models to Recognize Contextually Important Momentscited by 0
  14. 2026Instruct-ICL: Instruction-Guided In-Context Learning for Post-Disaster Damage Assessmentcited by 0