VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Large vision-language models exhibit inherent capabilities to handle diverse visual perception tasks. In this paper, we introduce VisionReasoner, a unified framework capable of reasoning and solving multiple visual perception tasks within a shared model. Specifically, by designing a unified reward mechanism and multi-object cognitive learning strategies, VisionReasoner enhances its reasoning capabilities to analyze visual inputs, and addresses diverse perception tasks within a unified model. VisionReasoner generates a structured reasoning process before delivering the desired outputs responding to user queries. Human evaluation reveals the reasoning process of VisionReasoner is faithful and reliable even without annotated reasoning train data. To rigorously assess unified visual perception capabilities, we evaluate VisionReasoner on ten diverse tasks spanning three critical domains: detection, segmentation, and counting. Experimental results show that VisionReasoner achieves superior performance as a unified model, outperforming the baseline Qwen2.5VL by relative margins of 29.1% on COCO (detection), 22.1% on ReasonSeg (segmentation), and 13.2% on CountBench (counting).
Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 40 of its 48 references that are in the map. First posted on arXiv 17 May 2025.
Builds on
40 of 48 references.
Closest ancestors · 5
Same direction, newest first
- 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
- 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
- 2023PixelLM: Pixel Reasoning with Large Multimodal Model4 Dec
- 2023LISA: Reasoning Segmentation via Large Language Model1 Aug
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar
Other references · 35
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
23 more
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
- 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
- 2021Grounded Language-Image Pre-trainingcited by 1,156
- 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
- 2016Modeling Context in Referring Expressionscited by 1,063
- 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
- 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
- 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
- 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
- 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 365
- 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
- 2021TransVG: End-to-End Visual Grounding with Transformerscited by 271
- 2023GRES: Generalized Referring Expression Segmentationcited by 254
- 2023Teaching CLIP to Count to Tencited by 198
- 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
- 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 89
- 2023Vision Grid Transformer for Document Layout Analysiscited by 47
- 2022DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Groundingcited by 24
- 2024Lyra: An Efficient and Speech-Centric Framework for Omni-Cognitioncited by 20
Built on by
Citing papers per year: 2025: 18; 2026: 33.
51 papers.
Sort:
By area:
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
- 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 139
- 2025Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulationcited by 38
- 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 29
- 2025Sketch-in-Latents: Eliciting Unified Reasoning in MLLMscited by 25
- 2025Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMscited by 17
- 2025Reinforcing Video Reasoning Segmentation to Think Before It Segmentscited by 15
- 2025DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoningcited by 10
- 2026IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentationcited by 10
- 2025Reasoning-Aligned Perception Decoupling for Scalable Multi-modal Reasoningcited by 9
- 2025Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Imagescited by 8
- 2025Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenescited by 7
38 more
- 2025Counterfactual Segmentation Reasoning: Diagnosing and Mitigating Pixel-Grounding Hallucinationcited by 7
- 2025VGent: Visual Grounding via Modular Design for Disentangling Reasoning and Predictioncited by 7
- 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forwardcited by 6
- 2025Bridging Semantics and Geometry: A Decoupled LVLM-SAM Framework for Reasoning Segmentation in Optical Remote Sensingcited by 6
- 2025RSAgent: Learning to Reason and Act for Text-Guided Segmentation via Multi-Turn Tool Invocationscited by 6
- 2026Seg-ReSearch: Segmentation with Interleaved Reasoning and External Searchcited by 6
- 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generationcited by 6
- 2025A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoningcited by 5
- 2025ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learningcited by 4
- 2026Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thoughtcited by 4
- 2026Affordance Agent Harness: Verification-Gated Skill Orchestrationcited by 4
- 2026SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentationcited by 4
- 2026CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correctioncited by 3
- 2026Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Designcited by 3
- 2026Discriminative Perception via Anchored Description for Reasoning Segmentationcited by 3
- 2026Semantic-Enriched Latent Visual Reasoningcited by 3
- 2026Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Viewscited by 3
- 2026StAR: Segment Anything Reasonercited by 2
- 2026Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Rewardcited by 2
- 2026SetCon: Towards Open-Ended Referring Segmentation via Set-Level Concept Predictioncited by 2
- 2026Reason Twice: Segmentation via Candidate Discovery and Comparative Reasoningcited by 2
- 2025Syn-GRPO: Self-Evolving Data Synthesis for MLLM Perception Reasoningcited by 1
- 2026Topo-R1: Detecting Topological Anomalies via Vision-Language Modelscited by 1
- 2026EgoFun3D: Modeling Interactive Objects from Egocentric Videos using Function Templatescited by 1
- 2026Vision Harnessing Agent for Open Ad-hoc Segmentationcited by 1
- 2026GroundedSurg: A Multi-Procedure Benchmark for Language-Conditioned Surgical Tool Segmentationcited by 0
- 2026QTrack: Query-Driven Reasoning for Multi-modal MOTcited by 0
- 2026MHPO: Modulated Hazard-aware Policy Optimization for Stable Reinforcement Learningcited by 0
- 2026Pest-Thinker: Learning to Think and Reason like Entomologists via Reinforcement Learningcited by 0
- 2026RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentationcited by 0
- 2026Does Seeing More Mean Knowing More? Mono-Anchored Advantage Normalization for Multi-Source Visual Reasoningcited by 0
- 2026CR-Seg: Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentationcited by 0
- 2026CFCamo: A Counterfactual Detect-or-Abstain Framework for Camouflaged Object Detectioncited by 0
- 2026Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoningcited by 0
- 2026MMBench-Live: A Continuously Evolving Benchmark for Multimodal Modelscited by 0
- 2026DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentationcited by 0
- 2026Position Is All You Need: A Free Lunch Token Compression Strategy for MLLM-based Referring Expression Segmentationcited by 0
- 2026LogiScope-VQA: Benchmarking Vision-Language Models for Logistics Hazard Identification in Industrial Scenarioscited by 0
50 most cited of 51 All 51 (JSON)
- 2026ContourVLA: A Closed-Loop Perception-Action Contour Policy for Generalized Referring Expression Segmentation8 Oct 2026
- 2026LogiScope-VQA: Benchmarking Vision-Language Models for Logistics Hazard Identification in Industrial Scenarios9 Sep 2026
- 2026DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation6 Jul 2026
- 2026MMBench-Live: A Continuously Evolving Benchmark for Multimodal Models2 Jul 2026
- 2026Position Is All You Need: A Free Lunch Token Compression Strategy for MLLM-based Referring Expression Segmentation26 Jun 2026
- 2026Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Views22 Jun 2026
- 2026Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoning11 Jun 2026
- 2026Reason Twice: Segmentation via Candidate Discovery and Comparative Reasoning8 Jun 2026
- 2026CR-Seg: Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentation2 Jun 2026
- 2026CFCamo: A Counterfactual Detect-or-Abstain Framework for Camouflaged Object Detection28 May 2026
- 2026Does Seeing More Mean Knowing More? Mono-Anchored Advantage Normalization for Multi-Source Visual Reasoning25 May 2026
- 2026SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation21 May 2026
38 more
- 2026SetCon: Towards Open-Ended Referring Segmentation via Set-Level Concept Prediction19 May 2026
- 2026Vision Harnessing Agent for Open Ad-hoc Segmentation19 May 2026
- 2026Semantic-Enriched Latent Visual Reasoning19 May 2026
- 2026RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentation8 May 2026
- 2026Pest-Thinker: Learning to Think and Reason like Entomologists via Reinforcement Learning7 May 2026
- 2026Affordance Agent Harness: Verification-Gated Skill Orchestration1 May 2026
- 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation13 Apr 2026
- 2026EgoFun3D: Modeling Interactive Objects from Egocentric Videos using Function Templates13 Apr 2026
- 2026Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Reward6 Apr 2026
- 2026StAR: Segment Anything Reasoner15 Mar 2026
- 2026MHPO: Modulated Hazard-aware Policy Optimization for Stable Reinforcement Learning14 Mar 2026
- 2026QTrack: Query-Driven Reasoning for Multi-modal MOT14 Mar 2026
- 2026Topo-R1: Detecting Topological Anomalies via Vision-Language Models13 Mar 2026
- 2026Discriminative Perception via Anchored Description for Reasoning Segmentation4 Mar 2026
- 2026GroundedSurg: A Multi-Procedure Benchmark for Language-Conditioned Surgical Tool Segmentation1 Mar 2026
- 2026Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought27 Feb 2026
- 2026Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design25 Feb 2026
- 2026Seg-ReSearch: Segmentation with Interleaved Reasoning and External Search4 Feb 2026
- 2026CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correction24 Jan 2026
- 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding15 Jan 2026
- 2026IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentation6 Jan 2026
- 2025RSAgent: Learning to Reason and Act for Text-Guided Segmentation via Multi-Turn Tool Invocations30 Dec 2025
- 2025Bridging Semantics and Geometry: A Decoupled LVLM-SAM Framework for Reasoning Segmentation in Optical Remote Sensing22 Dec 2025
- 2025Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Images19 Dec 2025
- 2025Sketch-in-Latents: Eliciting Unified Reasoning in MLLMs18 Dec 2025
- 2025A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoning16 Dec 2025
- 2025VGent: Visual Grounding via Modular Design for Disentangling Reasoning and Prediction11 Dec 2025
- 2025ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learning2 Dec 2025
- 2025Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs24 Nov 2025
- 2025Syn-GRPO: Self-Evolving Data Synthesis for MLLM Perception Reasoning24 Nov 2025
- 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forward23 Oct 2025
- 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoning14 Oct 2025
- 2025DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoning25 Sep 2025
- 2025Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulation19 Aug 2025
- 2025Reinforcing Video Reasoning Segmentation to Think Before It Segments15 Aug 2025
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers30 Jun 2025
- 2025Counterfactual Segmentation Reasoning: Diagnosing and Mitigating Pixel-Grounding Hallucination26 Jun 2025
- 2025Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes11 Jun 2025
50 newest of 51 All 51 (JSON)
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
- 2025Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulationcited by 38
- 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 29
- 2025Sketch-in-Latents: Eliciting Unified Reasoning in MLLMscited by 25
- 2025Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMscited by 17
- 2025Reinforcing Video Reasoning Segmentation to Think Before It Segmentscited by 15
- 2025DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoningcited by 10
- 2026IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentationcited by 10
- 2025Reasoning-Aligned Perception Decoupling for Scalable Multi-modal Reasoningcited by 9
- 2025Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Imagescited by 8
- 2025Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenescited by 7
- 2025Counterfactual Segmentation Reasoning: Diagnosing and Mitigating Pixel-Grounding Hallucinationcited by 7
12 most cited of 38 All 38 (JSON)
- 2026LogiScope-VQA: Benchmarking Vision-Language Models for Logistics Hazard Identification in Industrial Scenarios9 Sep 2026
- 2026DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation6 Jul 2026
- 2026Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Views22 Jun 2026
- 2026Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoning11 Jun 2026
- 2026Reason Twice: Segmentation via Candidate Discovery and Comparative Reasoning8 Jun 2026
- 2026CR-Seg: Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentation2 Jun 2026
- 2026Does Seeing More Mean Knowing More? Mono-Anchored Advantage Normalization for Multi-Source Visual Reasoning25 May 2026
- 2026SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation21 May 2026
- 2026Semantic-Enriched Latent Visual Reasoning19 May 2026
- 2026RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentation8 May 2026
- 2026Pest-Thinker: Learning to Think and Reason like Entomologists via Reinforcement Learning7 May 2026
- 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation13 Apr 2026
12 newest of 38 All 38 (JSON)
- 2026Vision Harnessing Agent for Open Ad-hoc Segmentationcited by 1
- 2026Vision Harnessing Agent for Open Ad-hoc Segmentation19 May 2026
- 2026Affordance Agent Harness: Verification-Gated Skill Orchestrationcited by 4
- 2026StAR: Segment Anything Reasonercited by 2
- 2026SetCon: Towards Open-Ended Referring Segmentation via Set-Level Concept Predictioncited by 2
- 2026EgoFun3D: Modeling Interactive Objects from Egocentric Videos using Function Templatescited by 1
- 2026GroundedSurg: A Multi-Procedure Benchmark for Language-Conditioned Surgical Tool Segmentationcited by 0
- 2026CFCamo: A Counterfactual Detect-or-Abstain Framework for Camouflaged Object Detectioncited by 0
- 2026ContourVLA: A Closed-Loop Perception-Action Contour Policy for Generalized Referring Expression Segmentationcited by 0
- 2026ContourVLA: A Closed-Loop Perception-Action Contour Policy for Generalized Referring Expression Segmentation8 Oct 2026
- 2026CFCamo: A Counterfactual Detect-or-Abstain Framework for Camouflaged Object Detection28 May 2026
- 2026SetCon: Towards Open-Ended Referring Segmentation via Set-Level Concept Prediction19 May 2026
- 2026Affordance Agent Harness: Verification-Gated Skill Orchestration1 May 2026
- 2026EgoFun3D: Modeling Interactive Objects from Egocentric Videos using Function Templates13 Apr 2026
- 2026StAR: Segment Anything Reasoner15 Mar 2026
- 2026GroundedSurg: A Multi-Procedure Benchmark for Language-Conditioned Surgical Tool Segmentation1 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2025Artemis: Structured Visual Reasoning for Perception Policy Learning
- 2026Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning
- 2024ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom
- 2026RegionReasoner: Region-Grounded Multi-Round Visual Reasoning
- 2025Unbiased Visual Reasoning with Controlled Visual Inputs
- 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
- 2026What does RL improve for Visual Reasoning? A Frankenstein-Style Analysis
- 2026UniCAR-RL: Seeing Better before Thinking Deeper in Visual Mathematics