PixelEyes: Decoupling Perception and Reasoning for Pinpoint Visual Evidence Seeking
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
This paper explores multi-turn visual reasoning and observes that MLLMs repeatedly fail to localize the target, leading to long, redundant trajectories. We attribute this failure to the entanglement of reasoning and perception within a single model, the MLLM reasons and localizes simultaneously, and inaccurate localization triggers additional reasoning turns that bloat the trajectory. To solve this problem, we propose PixelEyes, a multi-turn visual reasoning agent that explicitly decouples reasoning from perception, i.e., the reasoner decides what to look for, while a specialized perception tool answers where it is. Specifically, PixelEyes introduces 1) Mask-guided Visual Search. A referring segmentation model is invoked to provide mask-precise localization, freeing the reasoner from the need to compensate for imprecise grounding. 2) Semantic-region Breadth-first Search (BFS). To eliminate redundant loops caused by repeatedly cropping incorrect sub-regions, we organize exploration as a breadth-first search over semantic regions. To internalize these capabilities, we construct the PixelEyes-6K dataset by resynthesizing expert trajectories from existing data. This explicitly embeds our mask-guided search and BFS logic into the model. We further introduce Pinpoint-Bench, a zero-hint visual search benchmark, i.e., no location cues are provided in the question, with instance-level masks and bounding boxes that separate localization failures from reasoning failures, enabling fine-grained analysis of failure modes such as inattentional blindness. Recent state-of-the-art MLLMs and visual reasoning agents leave large headroom on Pinpoint-Bench, demonstrating its quality and difficulty. Code and models are open-sourced.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 49 of its 51 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
49 of 51 references.
Closest ancestors · 17
Same direction, newest first
- 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence23 Oct
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug
- 2025Thyme: Think Beyond Images15 Aug
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers30 Jun
- 2025FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering26 Jun
- 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use25 May
- 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning21 May
- 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs21 May
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
- 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
5 more
- 2024HyperSeg: Towards Universal Visual Segmentation with Large Language Model26 Nov
- 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding27 Jun
- 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun
- 2023LISA: Reasoning Segmentation via Large Language Model1 Aug
- 2022Visual Programming: Compositional visual reasoning without training18 Nov
Other references · 32
- 2023Segment Anythingcited by 8,158
- 2023Visual Instruction Tuningcited by 7,792
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
20 more
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
- 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 447
- 2025Seed1.5-VL Technical Reportcited by 367
- 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 284
- 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 214
- 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 154
- 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 151
- 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 132
- 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 116
- 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Explorationcited by 99
- 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 67
- 2025Detect Anything via Next Point Predictioncited by 66
- 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 47
- 2025X-SAM: From Segment Anything to Any Segmentationcited by 30
- 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearchcited by 25
- 2025HiMTok: Learning Hierarchical Mask Tokens for Image Segmentation with Large Multimodal Modelcited by 19
- 2025Grasp Any Region: Towards Precise, Contextual Pixel Understanding for Multimodal LLMscited by 18
- 2024Instruction-Guided Visual Maskingcited by 17
- 2026SAMTok: Representing Any Mask with Two Wordscited by 10
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026From Web to Pixels: Bringing Agentic Search into Visual Perception
- 2026The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Design
- 2026Improving Visual Reasoning with Iterative Evidence Refinement
- 2026DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models
- 2025BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception
- 2026LOCI: A Locator-Critic with Refinement Loop
- 2026Pixelis: Reasoning in Pixels, from Seeing to Acting
- 2026CAVE: A Structured Credit Assignment Approach for Fragmented Visual Evidence Reasoning