# Pix2Fact: When Vision Is Not Enough -- Benchmarking Fine-Grained VQA with Web Verification on High-Resolution Real-World Scenes

arXiv: https://arxiv.org/abs/2602.00593 · HTML: https://mlsanity.com/paper/2602.00593/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 31 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 29 of 29)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs](https://mlsanity.com/paper/2312.14135/) (2023)
- [MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries](https://mlsanity.com/paper/2401.15391/) (2024)
- [MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding](https://mlsanity.com/paper/2406.09411/) (2024)
- [SlideVQA: A Dataset for Document Visual Question Answering on Multiple Images](https://mlsanity.com/paper/2301.04883/) (2023)
- [VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding](https://mlsanity.com/paper/2406.12384/) (2024)
- [DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks](https://mlsanity.com/paper/2309.17167/) (2023)
- [CRAG -- Comprehensive RAG Benchmark](https://mlsanity.com/paper/2406.04744/) (2024)
- [CriticBench: Benchmarking LLMs for Critique-Correct Reasoning](https://mlsanity.com/paper/2402.14809/) (2024)
- [Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge](https://mlsanity.com/paper/2501.18099/) (2025)
- [EchoSight: Advancing Visual-Language Models with Wiki Knowledge](https://mlsanity.com/paper/2407.12735/) (2024)
- [Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning](https://mlsanity.com/paper/2406.12742/) (2024)
- [Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphs](https://mlsanity.com/paper/2402.11199/) (2024)
- [Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification](https://mlsanity.com/paper/2506.07235/) (2025)
- [WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point](https://mlsanity.com/paper/2502.08047/) (2025)
- [AerialVG: A Challenging Benchmark for Aerial Visual Grounding by Exploring Positional Relations](https://mlsanity.com/paper/2504.07836/) (2025)
- [MedSG-Bench: A Benchmark for Medical Image Sequences Grounding](https://mlsanity.com/paper/2505.11852/) (2025)
- [ReasonVQA: A Multi-hop Reasoning Benchmark with Structural Knowledge for Visual Question Answering](https://mlsanity.com/paper/2507.16403/) (2025)
- [Do Vision-Language Models Measure Up? Benchmarking Visual Measurement Reading with MeasureBench](https://mlsanity.com/paper/2510.26865/) (2025)
- [Jack and Masters of all Trades: One-Pass Learning Sets of Model Sets From Large Pre-Trained Models](https://mlsanity.com/paper/2205.00671/) (2022)
- [BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining](https://mlsanity.com/paper/2401.06443/) (2024)
- [OmniGround: A Comprehensive Spatio-Temporal Grounding Benchmark for Real-World Complex Scenarios](https://mlsanity.com/paper/2511.16937/) (2025)
- [From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes](https://mlsanity.com/paper/2506.04897/) (2025)
- [VLR-Bench: Multilingual Benchmark Dataset for Vision-Language Retrieval Augmented Generation](https://mlsanity.com/paper/2412.10151/) (2024)
- [RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarios](https://mlsanity.com/paper/2512.24561/) (2025)
- [PathVG: A New Benchmark and Dataset for Pathology Visual Grounding](https://mlsanity.com/paper/2502.20869/) (2025)
- [Reasoning Court: Combining Reasoning, Action, and Judgment for Multi-Hop Reasoning](https://mlsanity.com/paper/2504.09781/) (2025)

## Built on by (most cited first, 3 of 3)

- [MultiHaystack: Benchmarking Multimodal Retrieval and Reasoning over 40K Images, Videos, and Documents](https://mlsanity.com/paper/2603.05697/) (2026)
- [VistaHop: Benchmarking Long-Horizon Visual DeepSearch](https://mlsanity.com/paper/2606.03273/) (2026)
- [UltraVR: A Diagnostic Ultra-Resolution Image-VQA Benchmark for Evidence-Grounded Reasoning](https://mlsanity.com/paper/2606.05576/) (2026)

## Abstract (arXiv)

Despite progress on general tasks, vision-language models (VLMs) still struggle with challenges that demand both fine-grained visual grounding and external knowledge, a synergy overlooked by existing benchmarks that evaluate these abilities in isolation. To fill this void, we introduce Pix2Fact, a visual question-answering benchmark designed to assess expert-level visual perception and knowledge search. Pix2Fact comprises 1,000 high-resolution (4K+) images spanning eight scenarios. Its questions and answers are meticulously crafted by PhD-holding annotators from top global universities across diverse disciplines. Each question requires detailed visual grounding and the integration of external knowledge. Evaluating ten state-of-the-art VLMs, including proprietary models such as Gemini-3.1-Pro and GPT-5.4, we find that Pix2Fact poses a formidable challenge: the most advanced model (Gemini-3.1-Pro) achieves only 51.7% average accuracy, even with access to visual ground truth and search tools. Our analysis attributes this low accuracy to three factors, frequent visual grounding errors even with visual ground truth, shallow search harnessing, and VLM's inability to retrieve long-tail, unstructured local information. This striking gap exposes the limitations of current models in assisting humans with real-world scenarios that demand overwhelming visual comprehension. We believe Pix2Fact will serve as a critical benchmark to drive the next generation of language-vision agents that seamlessly integrate fine-grained perception with robust knowledge search.