mlsanityData to 7 Oct 2026

Multi-Agent VQA: Exploring Multi-Agent Foundation Models in Zero-Shot Visual Question Answering

arXiv 2403.14783 ↗First posted 21 Mar 2024cs.CV cs.AI cs.CLcited by 15

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

This work explores the zero-shot capabilities of foundation models in Visual Question Answering (VQA) tasks. We propose an adaptive multi-agent system, named Multi-Agent VQA, to overcome the limitations of foundation models in object detection and counting by using specialized agents as tools. Unlike existing approaches, our study focuses on the system's performance without fine-tuning it on specific VQA datasets, making it more practical and robust in the open world. We present preliminary experimental results under zero-shot scenarios and highlight some failure cases, offering new directions for future research.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 10 of its 15 references that are in the map. First posted on arXiv 21 Mar 2024.

Builds on

10 of 15 references.

References · 10

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  6. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  7. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  8. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 424
  9. 2023ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalitiescited by 117
  10. 2023CLIP-Count: Towards Text-Guided Zero-Shot Object Countingcited by 82

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 7.

15 papers.

Sort:

By area:

  1. 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?cited by 105
  2. 2024LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Modelscited by 31
  3. 2026Dual Latent Memory for Visual Multi-agent Systemcited by 9
  4. 2025Describe Anything Model for Visual Question Answering on Text-rich Imagescited by 5
  5. 2025Training-Free Time Series Classification via In-Context Reasoning with LLM Agentscited by 4
  6. 2025Refine and Align: Confidence Calibration through Multi-Agent Interaction in VQAcited by 4
  7. 2026Efficient Multimodal Planning Agent for Visual Question-Answeringcited by 2
  8. 2025GeoGrid-Bench: Can Foundation Models Understand Multimodal Gridded Geo-Spatial Data?cited by 0
  9. 2025Structured Prompting and Multi-Agent Knowledge Distillation for Traffic Video Interpretation and Risk Inferencecited by 0
  10. 2025Towards Signboard-Oriented Visual Question Answering: ViSignVQA Dataset, Method and Benchmarkcited by 0
  11. 2026Understanding Multi-Agent Reasoning with Large Language Models for Cartoon VQAcited by 0
  12. 2026Diagnosing Failure Modes of Shared-State Collaboration in Resource-Constrained Visual Agentscited by 0
3 more
  1. 2026Multi-Modal Generative Fuzzy System: Fuzzy Inference Guided Large Model Interactive Question Answering Frameworkcited by 0
  2. 2026Question-Guided Evidence Acquisition for Multimodal Visual Question Answeringcited by 0
  3. 2026A2Safe: Counterfactual Evidence-Aligned Adaptive Agent Collaboration for Safe and Effective Visual Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering
  2. 2025ViQAgent: Zero-Shot Video Question Answering via Agent with Open-Vocabulary Grounding Validation
  3. 2023Modularized Zero-shot VQA with Pre-trained Models
  4. 2024Multi-Agents Based on Large Language Models for Knowledge-based Visual Question Answering
  5. 2023Exploring Question Decomposition for Zero-Shot VQA
  6. 2025The Quest for Visual Understanding: A Journey Through the Evolution of Visual Question Answering
  7. 2026Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidence
  8. 2025Visual question answering: from early developments to recent advances -- a survey