Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Models
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Recent studies posit that Reinforcement Learning with Verifiable Rewards (RLVR) primarily amplifies behaviors inherent to the pre-training distribution rather than inducing new capabilities, but these insights are predominantly limited to language-only domains, leaving the dynamics of visual-centric spatial reasoning under-explored. To examine the impact of RLVR on the capability boundaries of Vision-Language Models (VLMs), we introduce Ariadne, a controlled framework based on synthetic maze navigation where the reasoning difficulty is precisely regulated by path length and the number of turns. We demonstrate that applying RLVR extends the spatial reasoning boundary, achieving success on problems where the base policy VLM consistently attains 0% accuracy despite increasing pass@k sampling budgets, indicating that the optimized policy successfully navigates search spaces that were effectively unreachable by the base distribution. Furthermore, despite being trained exclusively on synthetic mazes, we evaluate the model on two real-world navigation benchmarks (MapBench and ReasonMap) in a zero-shot setting. The observed improvements in these out-of-domain tasks suggest genuine spatial reasoning capability expansion rather than mere sampling efficiency.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 33 references that are in the map. First posted on arXiv 1 Nov 2025.
Builds on
21 of 33 references.
Closest ancestors · 4
Same direction, newest first
- 2025SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning2 Jun
- 2025ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Maps24 May
- 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
Other references · 17
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024OpenAI o1 System Cardcited by 1,244
- 2024Qwen2.5-Coder Technical Reportcited by 1,203
- 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
- 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 129
- 2024InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoningcited by 119
- 2025The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reasoncited by 49
- 2025Rethinking Reflection in Pre-Trainingcited by 43
- 2025Emergent Hierarchical Reasoning in LLMs through Reinforcement Learningcited by 42
5 more
- 2025Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RLcited by 36
- 2025AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPOcited by 33
- 2025Can Large Vision Language Models Read Maps Like a Human?cited by 30
- 2025On the Emergence of Thinking in LLMs I: Searching for the Right Intuitioncited by 22
- 2025Unlocking Reasoning Capabilities in LLMs via Reinforcement Learning Explorationcited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic View
- 2026What does RL improve for Visual Reasoning? A Frankenstein-Style Analysis
- 2025Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme
- 2025LaViPlan : Language-Guided Visual Path Planning with RLVR
- 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
- 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Models
- 2025MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?