# Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuning

arXiv: https://arxiv.org/abs/2503.16188 · HTML: https://mlsanity.com/paper/2503.16188/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 41 of its 84 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 30 of 41)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://mlsanity.com/paper/2312.08935/) (2023)
- [DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://mlsanity.com/paper/2403.05525/) (2024)
- [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training](https://mlsanity.com/paper/2501.17161/) (2025)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs](https://mlsanity.com/paper/2412.21187/) (2024)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking](https://mlsanity.com/paper/2501.04519/) (2025)
- [MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning](https://mlsanity.com/paper/2503.07365/) (2025)
- [To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning](https://mlsanity.com/paper/2409.12183/) (2024)
- [Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning](https://mlsanity.com/paper/2502.14768/) (2025)
- [LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL](https://mlsanity.com/paper/2503.07536/) (2025)
- [REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization](https://mlsanity.com/paper/2501.03262/) (2025)
- [When More is Less: Understanding Chain-of-Thought Length in LLMs](https://mlsanity.com/paper/2502.07266/) (2025)
- [R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model](https://mlsanity.com/paper/2503.05132/) (2025)
- [The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks](https://mlsanity.com/paper/2502.08235/) (2025)
- [Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models](https://mlsanity.com/paper/2503.13939/) (2025)

## Built on by (most cited first, 30 of 44)

- [UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning](https://mlsanity.com/paper/2503.21620/) (2025)
- [Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens](https://mlsanity.com/paper/2506.17218/) (2025)
- [GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents](https://mlsanity.com/paper/2505.15810/) (2025)
- [VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents](https://mlsanity.com/paper/2510.16907/) (2025)
- [TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs](https://mlsanity.com/paper/2512.14698/) (2025)
- [Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles](https://mlsanity.com/paper/2505.23590/) (2025)
- [Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillation](https://mlsanity.com/paper/2602.02994/) (2026)
- [Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle](https://mlsanity.com/paper/2509.16679/) (2025)
- [PixelThink: Towards Efficient Chain-of-Pixel Reasoning](https://mlsanity.com/paper/2505.23727/) (2025)
- [More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models](https://mlsanity.com/paper/2509.25848/) (2025)
- [Audio-Thinker: Guiding Audio Language Model When and How to Think via Reinforcement Learning](https://mlsanity.com/paper/2508.08039/) (2025)
- [Mull-Tokens: Modality-Agnostic Latent Thinking](https://mlsanity.com/paper/2512.10941/) (2025)
- [PyVision-RL: Forging Open Agentic Vision Models via RL](https://mlsanity.com/paper/2602.20739/) (2026)
- [Multimodal Tabular Reasoning with Privileged Structured Information](https://mlsanity.com/paper/2506.04088/) (2025)
- [Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models](https://mlsanity.com/paper/2505.18536/) (2025)
- [Training LLMs for EHR-Based Reasoning Tasks via Reinforcement Learning](https://mlsanity.com/paper/2505.24105/) (2025)
- [SABER: Switchable and Balanced Training for Efficient LLM Reasoning](https://mlsanity.com/paper/2508.10026/) (2025)
- [Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by Chain-of-Thought Reasoning](https://mlsanity.com/paper/2602.07605/) (2026)
- [TaoSR1: The Thinking Model for E-commerce Relevance Search](https://mlsanity.com/paper/2508.12365/) (2025)
- [TTRV: Test-Time Reinforcement Learning for Vision Language Models](https://mlsanity.com/paper/2510.06783/) (2025)
- [Enhancing Spatial Reasoning in Vision-Language Models via Chain-of-Thought Prompting and Reinforcement Learning](https://mlsanity.com/paper/2507.13362/) (2025)
- [Ref-Adv: Exploring MLLM Visual Reasoning in Referring Expression Tasks](https://mlsanity.com/paper/2602.23898/) (2026)
- [Rethinking Token-Level Policy Optimization for Multimodal Chain-of-Thought](https://mlsanity.com/paper/2603.22847/) (2026)
- [How to Train Your Deep Research Agent? Prompt, Reward, and Policy Optimization in Search-R1](https://mlsanity.com/paper/2602.19526/) (2026)
- [Unified Generation and Self-Verification for Vision-Language Models via Advantage Decoupled Preference Optimization](https://mlsanity.com/paper/2601.01483/) (2026)
- [Reasoning-Driven Multimodal LLM for Domain Generalization](https://mlsanity.com/paper/2602.23777/) (2026)
- [Learning Adaptive Reasoning Paths for Efficient Visual Reasoning](https://mlsanity.com/paper/2604.14568/) (2026)
- [Enhancing Radiology Report Generation and Visual Grounding using Reinforcement Learning](https://mlsanity.com/paper/2512.10691/) (2025)
- [Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models](https://mlsanity.com/paper/2603.00431/) (2026)
- [Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Models](https://mlsanity.com/paper/2604.25591/) (2026)

## Abstract (arXiv)

This paper investigates the role of explicit thinking process in rule-based reinforcement fine-tuning (RFT) for MLLMs. We first propose CLS-RL for MLLM image classification, using verifiable rewards for fine-tuning. Experiments show CLS-RL significantly outperforms SFT and yields a cross-dataset generalization effect. We then rethink and question whether explicit thinking in RFT is always necessary. Challenging the convention that explicit thinking is crucial for the success of RFT, we introduce No-Thinking-RL, exploring RFT without thinking by introducing a simple equality accuracy reward. We evaluate No-Thinking-RL on 6 diverse tasks across different model sizes and types. Experimental results reveal three key findings: 1). Visual perception tasks do not require thinking during RFT, as No-Thinking-RL consistently outperforms or matches Thinking-based RFT across model sizes. 2).} Models with limited capabilities struggle to generate high-quality CoT for RFT, making Thinking-based RFT less effective than No-Thinking-RL. 3). There are inconsistencies between the answers in the thinking and answer tags for some responses of thinking-based RFT, which show lower accuracy than the overall accuracy. We hypothesize that explicit thinking before verifiable answers may hinder reward convergence and reduce performance. To test this hypothesis, we propose Think-After-Answer, which places thinking after the answer to mitigate this effect for experimental verification. Lastly, we conduct a pilot study to explore whether MLLMs can learn when to think during RFT, introducing an Adaptive-Thinking method. Experiments show that it converges to a specific prompt depending on model capability and task complexity, achieving comparable or better performance than both Thinking and No-Thinking-RL. This suggests MLLMs can adaptively decide to think or not based on their capabilities and task complexity.