# Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models

arXiv: https://arxiv.org/abs/2503.20752 · HTML: https://mlsanity.com/paper/2503.20752/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 46 of its 106 references that are in the map.
- First posted on arXiv 26 Mar 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 30 of 46)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Aligning Large Multimodal Models with Factually Augmented RLHF](https://mlsanity.com/paper/2309.14525/) (2023)
- [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training](https://mlsanity.com/paper/2501.17161/) (2025)
- [WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct](https://mlsanity.com/paper/2308.09583/) (2023)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [LLaVA-CoT: Let Vision Language Models Reason Step-by-Step](https://mlsanity.com/paper/2411.10440/) (2024)
- [Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning](https://mlsanity.com/paper/2105.04165/) (2021)
- [LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models](https://mlsanity.com/paper/2407.12772/) (2024)
- [IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning](https://mlsanity.com/paper/2110.13214/) (2021)
- [GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning](https://mlsanity.com/paper/2105.14517/) (2021)
- [G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model](https://mlsanity.com/paper/2312.11370/) (2023)
- [Pixtral 12B](https://mlsanity.com/paper/2410.07073/) (2024)
- [MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning](https://mlsanity.com/paper/2502.19634/) (2025)
- [Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planning](https://mlsanity.com/paper/2311.17842/) (2023)
- [MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning](https://mlsanity.com/paper/2310.03731/) (2023)
- [Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning](https://mlsanity.com/paper/1905.06088/) (2019)
- [LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs](https://mlsanity.com/paper/2501.06186/) (2025)

## Built on by (most cited first, 30 of 99)

- [The Landscape of Agentic Reinforcement Learning for LLMs: A Survey](https://mlsanity.com/paper/2509.02547/) (2025)
- [Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models](https://mlsanity.com/paper/2505.04921/) (2025)
- [SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning](https://mlsanity.com/paper/2506.01713/) (2025)
- [Look-Back: Implicit Visual Re-focusing in MLLM Reasoning](https://mlsanity.com/paper/2507.03019/) (2025)
- [R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO](https://mlsanity.com/paper/2505.16673/) (2025)
- [MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources](https://mlsanity.com/paper/2509.21268/) (2025)
- [Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning](https://mlsanity.com/paper/2506.18234/) (2025)
- [BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens](https://mlsanity.com/paper/2508.17196/) (2025)
- [Explain Before You Answer: A Survey on Compositional Visual Reasoning](https://mlsanity.com/paper/2508.17298/) (2025)
- [VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning](https://mlsanity.com/paper/2512.22315/) (2025)
- [VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoning](https://mlsanity.com/paper/2507.22607/) (2025)
- [From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models](https://mlsanity.com/paper/2509.25373/) (2025)
- [WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2506.07905/) (2025)
- [AdaTooler-V: Adaptive Tool-Use for Images and Videos](https://mlsanity.com/paper/2512.16918/) (2025)
- [Perceptual-Evidence Anchored Reinforced Learning for Multimodal Reasoning](https://mlsanity.com/paper/2511.18437/) (2025)
- [SpaceVista: All-Scale Visual Spatial Reasoning from mm to km](https://mlsanity.com/paper/2510.09606/) (2025)
- [DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning](https://mlsanity.com/paper/2505.09655/) (2025)
- [Reasoning in Computer Vision: Taxonomy, Models, Tasks, and Methodologies](https://mlsanity.com/paper/2508.10523/) (2025)
- [SVQA-R1: Reinforcing Spatial Reasoning in MLLMs via View-Consistent Reward Optimization](https://mlsanity.com/paper/2506.01371/) (2025)
- [Play to Generalize: Learning to Reason Through Game Play](https://mlsanity.com/paper/2506.08011/) (2025)
- [Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Models](https://mlsanity.com/paper/2509.12132/) (2025)
- [AtomicVLA: Unlocking the Potential of Atomic Skill Learning in Robots](https://mlsanity.com/paper/2603.07648/) (2026)
- [GeoVLM-R1: Reinforcement Fine-Tuning for Improved Remote Sensing Reasoning](https://mlsanity.com/paper/2509.25026/) (2025)
- [Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models](https://mlsanity.com/paper/2505.18536/) (2025)
- [Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuning](https://mlsanity.com/paper/2509.23993/) (2025)
- [Towards Cross-View Point Correspondence in Vision-Language Models](https://mlsanity.com/paper/2512.04686/) (2025)
- [Sherlock: Self-Correcting Reasoning in Vision-Language Models](https://mlsanity.com/paper/2505.22651/) (2025)
- [VisualTrans: A Benchmark for Real-World Visual Transformation Reasoning](https://mlsanity.com/paper/2508.04043/) (2025)
- [Retrv-R1: A Reasoning-Driven MLLM Framework for Universal and Efficient Multimodal Retrieval](https://mlsanity.com/paper/2510.02745/) (2025)
- [Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigation](https://mlsanity.com/paper/2511.13269/) (2025)

## Abstract (arXiv)

Visual reasoning abilities play a crucial role in understanding complex multimodal data, advancing both domain-specific applications and artificial general intelligence (AGI). Existing methods enhance Vision-Language Models (VLMs) through Chain-of-Thought (CoT) supervised fine-tuning using meticulously annotated data. However, this approach may lead to overfitting and cognitive rigidity, limiting the model's generalization ability under domain shifts and reducing real-world applicability. To overcome these limitations, we propose Reason-RFT, a two-stage reinforcement fine-tuning framework for visual reasoning. First, Supervised Fine-Tuning (SFT) with curated CoT data activates the reasoning potential of VLMs. This is followed by reinforcement learning based on Group Relative Policy Optimization (GRPO), which generates multiple reasoning-response pairs to enhance adaptability to domain shifts. To evaluate Reason-RFT, we reconstructed a comprehensive dataset covering visual counting, structural perception, and spatial transformation, serving as a benchmark for systematic assessment across three key dimensions. Experimental results highlight three advantages: (1) performance enhancement, with Reason-RFT achieving state-of-the-art results and outperforming both open-source and proprietary models; (2) generalization superiority, maintaining robust performance under domain shifts across various tasks; and (3) data efficiency, excelling in few-shot learning scenarios and surpassing full-dataset SFT baselines. Reason-RFT introduces a novel training paradigm for visual reasoning and marks a significant step forward in multimodal research. Project website: https://tanhuajie.github.io/ReasonRFT