# InSight-o3: Empowering Multimodal Foundation Models with Generalized Visual Search

arXiv: https://arxiv.org/abs/2512.18745 · HTML: https://mlsanity.com/paper/2512.18745/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2025.

## Built on by (most cited first, 9 of 9)

- [AdaptMMBench: Benchmarking Adaptive Multimodal Reasoning for Mode Selection and Reasoning Process](https://mlsanity.com/paper/2602.02676/) (2026)
- [E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes](https://mlsanity.com/paper/2604.17969/) (2026)
- [VistaHop: Benchmarking Long-Horizon Visual DeepSearch](https://mlsanity.com/paper/2606.03273/) (2026)
- [Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models](https://mlsanity.com/paper/2603.18118/) (2026)
- [CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition](https://mlsanity.com/paper/2607.25294/) (2026)
- [VC-Tooler: Learning Compositional and Adaptive Visual Tool Use](https://mlsanity.com/paper/2608.02217/) (2026)
- [InSight-doc: Agentic Visual Perception for Long-Document Understanding](https://mlsanity.com/paper/2608.10628/) (2026)
- [Matched Outcomes, Divergent Gaze: How Foveated MLLMs Search Compared to Humans](https://mlsanity.com/paper/2608.16514/) (2026)
- [Agentic Tool-Augmented Reasoning for Explainable Image Forgery Detection](https://mlsanity.com/paper/2609.39066/) (2026)

## Abstract (arXiv)

The ability for AI agents to "think with images" requires a sophisticated blend of reasoning and perception. However, current open multimodal agents still largely fall short on the reasoning aspect crucial for real-world tasks like analyzing documents with dense charts/diagrams and navigating maps. To address this gap, we introduce O3-Bench, a new benchmark designed to evaluate multimodal reasoning with interleaved attention to visual details. O3-Bench features challenging problems that require agents to piece together subtle visual information from distinct image areas through multi-step reasoning. The problems are highly challenging even for frontier systems like OpenAI o3, which only obtains 40.8% accuracy on O3-Bench. To make progress, we propose InSight-o3, a multi-agent framework consisting of a visual reasoning agent (vReasoner) and a visual search agent (vSearcher) for which we introduce the task of generalized visual search -- locating relational, fuzzy, or conceptual regions described in free-form language, beyond just simple objects or figures in natural images. We then present a multimodal LLM purpose-trained for this task via reinforcement learning. As a plug-and-play agent, our vSearcher empowers frontier multimodal models (as vReasoners), significantly improving their performance on a wide range of benchmarks. This marks a concrete step towards powerful o3-like open systems. Our code and dataset can be found at https://github.com/m-Just/InSight-o3 .