mlsanityData to 14 Oct 2026

OWLViz: An Open-World Benchmark for Visual Question Answering

arXiv 2503.07631 ↗First posted 4 Mar 2025cs.LG cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

We present a challenging benchmark for the Open WorLd VISual question answering (OWLViz) task. OWLViz presents concise, unambiguous queries that require integrating multiple capabilities, including visual understanding, web exploration, and specialized tool usage. While humans achieve 69.2% accuracy on these intuitive tasks, even state-of-the-art VLMs struggle, with the best model, Gemini 2.0, achieving only 26.6% accuracy. Current agentic VLMs, which rely on limited vision and vision-language models as tools, perform even worse. This performance gap reveals significant limitations in multimodal systems' ability to select appropriate tools and execute complex reasoning sequences, establishing new directions for advancing practical AI research.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 36 of its 51 references that are in the map. First posted on arXiv 4 Mar 2025.

Builds on

36 of 51 references.

References · 36

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  9. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  10. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  11. 2015VQA: Visual Question Answeringcited by 2,506
  12. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
24 more
  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  3. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  4. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  5. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  6. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  7. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  8. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
  9. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
  10. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  11. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
  12. 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 243
  13. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 232
  14. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 193
  15. 2015Exploring Models and Data for Image Question Answeringcited by 139
  16. 2014A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Inputcited by 112
  17. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 102
  18. 2023Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Modelscited by 66
  19. 2024DynaSaur: Large Language Agents Beyond Predefined Actionscited by 32
  20. 2019CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsensecited by 27
  21. 2024VisionGPT: Vision-Language Understanding Agent Using Generalized Multimodal Frameworkcited by 20
  22. 2024VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Valuescited by 12
  23. 2024SADL: An Effective In-Context Learning Method for Compositional Visual QAcited by 5
  24. 2024InsightSee: Advancing Multi-agent Vision-Language Models for Enhanced Visual Understandingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Seek-and-Solve: Benchmarking MLLMs for Visual Clue-Driven Reasoning in Daily Scenarioscited by 2
  2. 2026Chain-of-Procedure: Hierarchical Visual-Language Reasoning for Procedural QAcited by 2
  3. 2026DUALVISION: RGB-Infrared Multimodal Large Language Models for Robust Visual Reasoningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025VQ-VA World: Towards High-Quality Visual Question-Visual Answering
  2. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge
  3. 2026NoteVQA: Benchmarking VLMs on Real-Life Questions from Human Communities
  4. 2024Zero-Shot Visual Reasoning by Vision-Language Models: Benchmarking and Analysis
  5. 2025Fast or Slow? Integrating Fast Intuition and Deliberate Thinking for Enhancing Visual Question Answering
  6. 2026VisualWorldBench: A Fine-Grained Multi-Task Benchmark for Evaluating Visual World Knowledge in MLLMs
  7. 2026WikiVQABench: A Knowledge-Grounded Visual Question Answering Benchmark from Wikipedia and Wikidata
  8. 2024Guiding Vision-Language Model Selection for Visual Question-Answering Across Tasks, Domains, and Knowledge Types