mlsanityData to 14 Oct 2026

VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning

arXiv 2505.12081 ↗First posted 17 May 2025cs.CVcited by 51

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Large vision-language models exhibit inherent capabilities to handle diverse visual perception tasks. In this paper, we introduce VisionReasoner, a unified framework capable of reasoning and solving multiple visual perception tasks within a shared model. Specifically, by designing a unified reward mechanism and multi-object cognitive learning strategies, VisionReasoner enhances its reasoning capabilities to analyze visual inputs, and addresses diverse perception tasks within a unified model. VisionReasoner generates a structured reasoning process before delivering the desired outputs responding to user queries. Human evaluation reveals the reasoning process of VisionReasoner is faithful and reliable even without annotated reasoning train data. To rigorously assess unified visual perception capabilities, we evaluate VisionReasoner on ten diverse tasks spanning three critical domains: detection, segmentation, and counting. Experimental results show that VisionReasoner achieves superior performance as a unified model, outperforming the baseline Qwen2.5VL by relative margins of 29.1% on COCO (detection), 22.1% on ReasonSeg (segmentation), and 13.2% on CountBench (counting).

Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 40 of its 48 references that are in the map. First posted on arXiv 17 May 2025.

Builds on

40 of 48 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
  2. 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
  3. 2023PixelLM: Pixel Reasoning with Large Multimodal Model4 Dec
  4. 2023LISA: Reasoning Segmentation via Large Language Model1 Aug
  5. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 35

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2024Qwen2.5 Technical Reportcited by 4,942
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  11. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  12. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
23 more
  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  4. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  5. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  6. 2021Grounded Language-Image Pre-trainingcited by 1,156
  7. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  8. 2016Modeling Context in Referring Expressionscited by 1,063
  9. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  10. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  11. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  12. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  13. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
  14. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 365
  15. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
  16. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 271
  17. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  18. 2023Teaching CLIP to Count to Tencited by 198
  19. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  20. 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 89
  21. 2023Vision Grid Transformer for Document Layout Analysiscited by 47
  22. 2022DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Groundingcited by 24
  23. 2024Lyra: An Efficient and Speech-Centric Framework for Omni-Cognitioncited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 18; 2026: 33.

51 papers.

Sort:

By area:

  1. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  2. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 139
  3. 2025Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulationcited by 38
  4. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 29
  5. 2025Sketch-in-Latents: Eliciting Unified Reasoning in MLLMscited by 25
  6. 2025Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMscited by 17
  7. 2025Reinforcing Video Reasoning Segmentation to Think Before It Segmentscited by 15
  8. 2025DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoningcited by 10
  9. 2026IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentationcited by 10
  10. 2025Reasoning-Aligned Perception Decoupling for Scalable Multi-modal Reasoningcited by 9
  11. 2025Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Imagescited by 8
  12. 2025Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenescited by 7
38 more
  1. 2025Counterfactual Segmentation Reasoning: Diagnosing and Mitigating Pixel-Grounding Hallucinationcited by 7
  2. 2025VGent: Visual Grounding via Modular Design for Disentangling Reasoning and Predictioncited by 7
  3. 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forwardcited by 6
  4. 2025Bridging Semantics and Geometry: A Decoupled LVLM-SAM Framework for Reasoning Segmentation in Optical Remote Sensingcited by 6
  5. 2025RSAgent: Learning to Reason and Act for Text-Guided Segmentation via Multi-Turn Tool Invocationscited by 6
  6. 2026Seg-ReSearch: Segmentation with Interleaved Reasoning and External Searchcited by 6
  7. 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generationcited by 6
  8. 2025A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoningcited by 5
  9. 2025ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learningcited by 4
  10. 2026Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thoughtcited by 4
  11. 2026Affordance Agent Harness: Verification-Gated Skill Orchestrationcited by 4
  12. 2026SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentationcited by 4
  13. 2026CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correctioncited by 3
  14. 2026Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Designcited by 3
  15. 2026Discriminative Perception via Anchored Description for Reasoning Segmentationcited by 3
  16. 2026Semantic-Enriched Latent Visual Reasoningcited by 3
  17. 2026Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Viewscited by 3
  18. 2026StAR: Segment Anything Reasonercited by 2
  19. 2026Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Rewardcited by 2
  20. 2026SetCon: Towards Open-Ended Referring Segmentation via Set-Level Concept Predictioncited by 2
  21. 2026Reason Twice: Segmentation via Candidate Discovery and Comparative Reasoningcited by 2
  22. 2025Syn-GRPO: Self-Evolving Data Synthesis for MLLM Perception Reasoningcited by 1
  23. 2026Topo-R1: Detecting Topological Anomalies via Vision-Language Modelscited by 1
  24. 2026EgoFun3D: Modeling Interactive Objects from Egocentric Videos using Function Templatescited by 1
  25. 2026Vision Harnessing Agent for Open Ad-hoc Segmentationcited by 1
  26. 2026GroundedSurg: A Multi-Procedure Benchmark for Language-Conditioned Surgical Tool Segmentationcited by 0
  27. 2026QTrack: Query-Driven Reasoning for Multi-modal MOTcited by 0
  28. 2026MHPO: Modulated Hazard-aware Policy Optimization for Stable Reinforcement Learningcited by 0
  29. 2026Pest-Thinker: Learning to Think and Reason like Entomologists via Reinforcement Learningcited by 0
  30. 2026RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentationcited by 0
  31. 2026Does Seeing More Mean Knowing More? Mono-Anchored Advantage Normalization for Multi-Source Visual Reasoningcited by 0
  32. 2026CR-Seg: Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentationcited by 0
  33. 2026CFCamo: A Counterfactual Detect-or-Abstain Framework for Camouflaged Object Detectioncited by 0
  34. 2026Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoningcited by 0
  35. 2026MMBench-Live: A Continuously Evolving Benchmark for Multimodal Modelscited by 0
  36. 2026DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentationcited by 0
  37. 2026Position Is All You Need: A Free Lunch Token Compression Strategy for MLLM-based Referring Expression Segmentationcited by 0
  38. 2026LogiScope-VQA: Benchmarking Vision-Language Models for Logistics Hazard Identification in Industrial Scenarioscited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Artemis: Structured Visual Reasoning for Perception Policy Learning
  2. 2026Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning
  3. 2024ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom
  4. 2026RegionReasoner: Region-Grounded Multi-Round Visual Reasoning
  5. 2025Unbiased Visual Reasoning with Controlled Visual Inputs
  6. 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
  7. 2026What does RL improve for Visual Reasoning? A Frankenstein-Style Analysis
  8. 2026UniCAR-RL: Seeing Better before Thinking Deeper in Visual Mathematics