# VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning

arXiv: https://arxiv.org/abs/2505.12081 · HTML: https://mlsanity.com/paper/2505.12081/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 40 of its 48 references that are in the map.
- First posted on arXiv 17 May 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 30 of 40)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic](https://mlsanity.com/paper/2306.15195/) (2023)
- [MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding](https://mlsanity.com/paper/2104.12763/) (2021)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [YOLO-World: Real-Time Open-Vocabulary Object Detection](https://mlsanity.com/paper/2401.17270/) (2024)
- [OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models](https://mlsanity.com/paper/2305.07895/) (2023)
- [GLaMM: Pixel Grounding Large Multimodal Model](https://mlsanity.com/paper/2311.03356/) (2023)
- [Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models](https://mlsanity.com/paper/2403.18814/) (2024)
- [LAVT: Language-Aware Vision Transformer for Referring Image Segmentation](https://mlsanity.com/paper/2112.02244/) (2021)

## Built on by (most cited first, 30 of 51)

- [Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers](https://mlsanity.com/paper/2506.23918/) (2025)
- [Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding](https://mlsanity.com/paper/2601.10611/) (2026)
- [Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulation](https://mlsanity.com/paper/2508.13998/) (2025)
- [Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoning](https://mlsanity.com/paper/2510.12712/) (2025)
- [Sketch-in-Latents: Eliciting Unified Reasoning in MLLMs](https://mlsanity.com/paper/2512.16584/) (2025)
- [Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs](https://mlsanity.com/paper/2511.19773/) (2025)
- [Reinforcing Video Reasoning Segmentation to Think Before It Segments](https://mlsanity.com/paper/2508.11538/) (2025)
- [DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoning](https://mlsanity.com/paper/2509.20912/) (2025)
- [IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentation](https://mlsanity.com/paper/2601.03054/) (2026)
- [Reasoning-Aligned Perception Decoupling for Scalable Multi-modal Reasoning](https://mlsanity.com/paper/2506.04559/) (2025)
- [Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Images](https://mlsanity.com/paper/2512.17306/) (2025)
- [Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes](https://mlsanity.com/paper/2506.09557/) (2025)
- [Counterfactual Segmentation Reasoning: Diagnosing and Mitigating Pixel-Grounding Hallucination](https://mlsanity.com/paper/2506.21546/) (2025)
- [VGent: Visual Grounding via Modular Design for Disentangling Reasoning and Prediction](https://mlsanity.com/paper/2512.11099/) (2025)
- [Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forward](https://mlsanity.com/paper/2510.20696/) (2025)
- [Bridging Semantics and Geometry: A Decoupled LVLM-SAM Framework for Reasoning Segmentation in Optical Remote Sensing](https://mlsanity.com/paper/2512.19302/) (2025)
- [RSAgent: Learning to Reason and Act for Text-Guided Segmentation via Multi-Turn Tool Invocations](https://mlsanity.com/paper/2512.24023/) (2025)
- [Seg-ReSearch: Segmentation with Interleaved Reasoning and External Search](https://mlsanity.com/paper/2602.04454/) (2026)
- [LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation](https://mlsanity.com/paper/2604.11789/) (2026)
- [A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoning](https://mlsanity.com/paper/2512.14442/) (2025)
- [ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learning](https://mlsanity.com/paper/2512.02835/) (2025)
- [Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought](https://mlsanity.com/paper/2602.23959/) (2026)
- [Affordance Agent Harness: Verification-Gated Skill Orchestration](https://mlsanity.com/paper/2605.00663/) (2026)
- [SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation](https://mlsanity.com/paper/2605.22658/) (2026)
- [CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correction](https://mlsanity.com/paper/2601.17420/) (2026)
- [Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design](https://mlsanity.com/paper/2603.00152/) (2026)
- [Discriminative Perception via Anchored Description for Reasoning Segmentation](https://mlsanity.com/paper/2603.04002/) (2026)
- [Semantic-Enriched Latent Visual Reasoning](https://mlsanity.com/paper/2605.19342/) (2026)
- [Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Views](https://mlsanity.com/paper/2606.23557/) (2026)
- [StAR: Segment Anything Reasoner](https://mlsanity.com/paper/2603.14382/) (2026)

## Abstract (arXiv)

Large vision-language models exhibit inherent capabilities to handle diverse visual perception tasks. In this paper, we introduce VisionReasoner, a unified framework capable of reasoning and solving multiple visual perception tasks within a shared model. Specifically, by designing a unified reward mechanism and multi-object cognitive learning strategies, VisionReasoner enhances its reasoning capabilities to analyze visual inputs, and addresses diverse perception tasks within a unified model. VisionReasoner generates a structured reasoning process before delivering the desired outputs responding to user queries. Human evaluation reveals the reasoning process of VisionReasoner is faithful and reliable even without annotated reasoning train data. To rigorously assess unified visual perception capabilities, we evaluate VisionReasoner on ten diverse tasks spanning three critical domains: detection, segmentation, and counting. Experimental results show that VisionReasoner achieves superior performance as a unified model, outperforming the baseline Qwen2.5VL by relative margins of 29.1% on COCO (detection), 22.1% on ReasonSeg (segmentation), and 13.2% on CountBench (counting).