# Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models arXiv: https://arxiv.org/abs/2509.13031 ยท HTML: https://mlsanity.com/paper/2509.13031/ ## Key facts - Cited by 17 papers in the map (data through 14 Oct 2026). - Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. - Builds on 33 of its 49 references that are in the map. - First posted on arXiv 16 Sep 2025. ## Placed in - LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/) - Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/) ## Builds on (most cited first, 30 of 33) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017) - [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024) - [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025) - [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025) - [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024) - [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022) - [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022) - [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025) - [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023) - [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024) - [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022) - [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025) - [VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model](https://mlsanity.com/paper/2504.07615/) (2025) - [Seed1.5-VL Technical Report](https://mlsanity.com/paper/2505.07062/) (2025) - [Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning](https://mlsanity.com/paper/2209.14610/) (2022) - [VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning](https://mlsanity.com/paper/2504.08837/) (2025) - [R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization](https://mlsanity.com/paper/2503.12937/) (2025) - [Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search](https://mlsanity.com/paper/2412.18319/) (2024) - [DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models](https://mlsanity.com/paper/2411.00836/) (2024) - [SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement](https://mlsanity.com/paper/2504.07934/) (2025) - [OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles](https://mlsanity.com/paper/2503.17352/) (2025) - [CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoning](https://mlsanity.com/paper/2208.05358/) (2022) - [Perception-R1: Pioneering Perception Policy with Reinforcement Learning](https://mlsanity.com/paper/2504.07954/) (2025) - [Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning](https://mlsanity.com/paper/2505.14677/) (2025) - [FG-CLIP: Fine-Grained Visual and Textual Alignment](https://mlsanity.com/paper/2505.05071/) (2025) - [R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO](https://mlsanity.com/paper/2505.16673/) (2025) - [Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging](https://mlsanity.com/paper/2505.05464/) (2025) - [Qwen Look Again: Guiding Vision-Language Reasoning Models to Re-attention Visual Information](https://mlsanity.com/paper/2505.23558/) (2025) - [ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs](https://mlsanity.com/paper/2506.10128/) (2025) ## Built on by (most cited first, 17 of 17) - [SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models](https://mlsanity.com/paper/2510.08531/) (2025) - [On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs](https://mlsanity.com/paper/2602.12506/) (2026) - [PosterCopilot: Toward Layout Reasoning and Controllable Editing for Professional Graphic Design](https://mlsanity.com/paper/2512.04082/) (2025) - [Learning Modal-Mixed Chain-of-Thought Reasoning with Latent Embeddings](https://mlsanity.com/paper/2602.00574/) (2026) - [CapGeo-Bench: Decoupling Visual Perception from Reasoning and Evaluating Geometric Understanding](https://mlsanity.com/paper/2510.09302/) (2025) - [Learning When to Look: A Disentangled Curriculum for Strategic Perception in Multimodal Reasoning](https://mlsanity.com/paper/2512.17227/) (2025) - [MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Images](https://mlsanity.com/paper/2511.19119/) (2025) - [When Thinking Hurts: Mitigating Visual Forgetting in Video Reasoning via Frame Repetition](https://mlsanity.com/paper/2603.16256/) (2026) - [Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning](https://mlsanity.com/paper/2607.01191/) (2026) - [Thinking With Tools, Not With Pixels: Tool Calls as Text Scaffolds for Visual Reasoning](https://mlsanity.com/paper/2608.09682/) (2026) - [CogniEdit: Dense Gradient Flow Optimization for Fine-Grained Image Editing](https://mlsanity.com/paper/2512.13276/) (2025) - [Structured Role-Aware Policy Optimization for Multimodal Reasoning](https://mlsanity.com/paper/2605.07274/) (2026) - [More Than the Final Answer: Improving Visual Extraction and Logical Consistency in Vision-Language Models](https://mlsanity.com/paper/2512.12487/) (2025) - [On Asymmetric Optimization of Reasoning and Perception in Vision-Language Model Post-Training](https://mlsanity.com/paper/2605.29496/) (2026) - [Video-FLAIR: Not Whether to Reason, But How](https://mlsanity.com/paper/2608.26495/) (2026) - [WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learning](https://mlsanity.com/paper/2608.27508/) (2026) - [VC-OPD: Visual Counterfactual On-Policy Distillation for Grounded Vision-Language Reasoning](https://mlsanity.com/paper/2610.14939/) (2026) ## Abstract (arXiv) Reinforcement learning (RL) has proven highly effective in eliciting the reasoning capabilities of large language models (LLMs). Inspired by this success, recent studies have explored applying similar techniques to vision-language models (VLMs), aiming to enhance their reasoning performance. However, directly transplanting RL methods from LLMs to VLMs is suboptimal, as the tasks faced by VLMs are inherently more complex. Specifically, VLMs must first accurately perceive and understand visual inputs before reasoning can be effectively performed. To address this challenge, we propose a two-stage reinforcement learning framework designed to jointly enhance both the perceptual and reasoning capabilities of VLMs. To mitigate the vanishing advantage issue commonly observed in RL training, we first perform dataset-level sampling to selectively strengthen specific capabilities using distinct data sources. During training, the first stage focuses on improving the model's visual perception through coarse- and fine-grained visual understanding, while the second stage targets the enhancement of reasoning abilities. After the proposed two-stage reinforcement learning process, we obtain PeBR-R1, a vision-language model with significantly enhanced perceptual and reasoning capabilities. Experimental results on seven benchmark datasets demonstrate the effectiveness of our approach and validate the superior performance of PeBR-R1 across diverse visual reasoning tasks.