# Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models

arXiv: https://arxiv.org/abs/2505.16854 · HTML: https://mlsanity.com/paper/2505.16854/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 21 of its 42 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [CogAgent: A Visual Language Model for GUI Agents](https://mlsanity.com/paper/2312.08914/) (2023)
- [SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents](https://mlsanity.com/paper/2401.10935/) (2024)
- [MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action](https://mlsanity.com/paper/2303.11381/) (2023)
- [A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis](https://mlsanity.com/paper/2307.12856/) (2023)
- [Android in the Wild: A Large-Scale Dataset for Android Device Control](https://mlsanity.com/paper/2307.10088/) (2023)
- [WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning](https://mlsanity.com/paper/2411.02337/) (2024)
- [ShowUI: One Vision-Language-Action Model for GUI Visual Agent](https://mlsanity.com/paper/2411.17465/) (2024)
- [GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning](https://mlsanity.com/paper/2105.14517/) (2021)
- [DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning](https://mlsanity.com/paper/2406.11896/) (2024)
- [Android in the Zoo: Chain-of-Action-Thought for GUI Agents](https://mlsanity.com/paper/2403.02713/) (2024)
- [VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Driving](https://mlsanity.com/paper/2412.15544/) (2024)
- [Automatic Curriculum Expert Iteration for Reliable LLM Reasoning](https://mlsanity.com/paper/2410.07627/) (2024)

## Built on by (most cited first, 30 of 30)

- [Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers](https://mlsanity.com/paper/2506.23918/) (2025)
- [Perception-Aware Policy Optimization for Multimodal Reasoning](https://mlsanity.com/paper/2507.06448/) (2025)
- [TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agents](https://mlsanity.com/paper/2604.24005/) (2026)
- [GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL](https://mlsanity.com/paper/2602.22190/) (2026)
- [SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Models](https://mlsanity.com/paper/2510.06871/) (2025)
- [Reinforcement Learning for Large Model: A Survey](https://mlsanity.com/paper/2508.08189/) (2025)
- [Reasoning Efficiently Through Adaptive Chain-of-Thought Compression: A Self-Optimizing Framework](https://mlsanity.com/paper/2509.14093/) (2025)
- [VaseVQA: Multimodal Agent and Benchmark for Ancient Greek Pottery](https://mlsanity.com/paper/2509.17191/) (2025)
- [Learning Adaptive Reasoning Paths for Efficient Visual Reasoning](https://mlsanity.com/paper/2604.14568/) (2026)
- [Learning to Think Fast and Slow for Visual Language Models](https://mlsanity.com/paper/2511.16670/) (2025)
- [AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraints](https://mlsanity.com/paper/2603.13348/) (2026)
- [Boosting Omni-Modal Language Models: Staged Post-Training with Visually Debiased Evaluation](https://mlsanity.com/paper/2605.12034/) (2026)
- [RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning](https://mlsanity.com/paper/2606.28266/) (2026)
- [CoEditor++: Instruction-based Visual Editing via Cognitive Reasoning](https://mlsanity.com/paper/2603.05518/) (2026)
- [EgoReasoner: Learning Egocentric 4D Reasoning via Task-Adaptive Structured Thinking](https://mlsanity.com/paper/2603.06561/) (2026)
- [Reasoning as Compression: Unifying Budget Forcing via the Conditional Information Bottleneck](https://mlsanity.com/paper/2603.08462/) (2026)
- [One Model, Two Minds: Task-Conditioned Reasoning for Unified Image Quality and Aesthetic Assessment](https://mlsanity.com/paper/2603.19779/) (2026)
- [Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models](https://mlsanity.com/paper/2604.03179/) (2026)
- [Efficient Agentic Reasoning Through Self-Regulated Simulative Planning](https://mlsanity.com/paper/2605.22138/) (2026)
- [Test-Time Scaling for Small VLMs on Multilingual Visual MCQ](https://mlsanity.com/paper/2607.09438/) (2026)
- [ClueTracer: Question-to-Vision Clue Tracing for Training-Free Hallucination Suppression in Multimodal Reasoning](https://mlsanity.com/paper/2602.02004/) (2026)
- [A Survey of Multimodal Mathematical Reasoning: From Perception, Alignment to Reasoning](https://mlsanity.com/paper/2603.08291/) (2026)
- [Efficient Reasoning on the Edge](https://mlsanity.com/paper/2603.16867/) (2026)
- [Batched Contextual Reinforcement: A Task-Scaling Law for Efficient Reasoning](https://mlsanity.com/paper/2604.02322/) (2026)
- [Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning](https://mlsanity.com/paper/2607.08572/) (2026)
- [Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs](https://mlsanity.com/paper/2608.12781/) (2026)
- [REChart: Reasoning-Efficient Chart Editing with Large Reasoning Models](https://mlsanity.com/paper/2608.17414/) (2026)
- [SAVER: Selective Auditing of Verbal Evidence for Error Recovery in VLM Change Reasoning](https://mlsanity.com/paper/2608.22857/) (2026)
- [Collapse, Not Complexity: Failure-Conditioned Decomposition Repair for End-to-End Document Parsing](https://mlsanity.com/paper/2609.23592/) (2026)
- [CounterRoute: Self-Routed Reasoning via Hierarchical Counterfactual Credit Assignment](https://mlsanity.com/paper/2609.29109/) (2026)

## Abstract (arXiv)

Reinforcement Learning (RL) has proven to be an effective post-training strategy for enhancing reasoning in vision-language models (VLMs). Group Relative Policy Optimization (GRPO) is a recent prominent method that encourages models to generate complete reasoning traces before answering, leading to increased token usage and computational cost. Inspired by the human-like thinking process-where people skip reasoning for easy questions but think carefully when needed-we explore how to enable VLMs to first decide when reasoning is necessary. To realize this, we propose TON, a two-stage training strategy: (i) a supervised fine-tuning (SFT) stage with a simple yet effective 'thought dropout' operation, where reasoning traces are randomly replaced with empty thoughts. This introduces a think-or-not format that serves as a cold start for selective reasoning; (ii) a GRPO stage that enables the model to freely explore when to think or not, while maximizing task-aware outcome rewards. Experimental results show that TON can reduce the completion length by up to 90% compared to vanilla GRPO, without sacrificing performance or even improving it. Further evaluations across LLM (GSM8K), VLM (CLEVR, Super-CLEVR, GeoQA), and Agentic (AITZ) tasks-covering a range of reasoning difficulties under both 3B and 7B models-consistently reveal that the model progressively learns to bypass unnecessary reasoning steps as training advances. These findings shed light on the path toward human-like reasoning patterns in RL approaches. Our code is available at https://github.com/kokolerk/TON.