mlsanityData to 7 Oct 2026

Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning

arXiv 2505.15966 ↗First posted 21 May 2025cs.CV cs.AI cs.CLcited by 267

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Chain-of-thought reasoning has significantly improved the performance of Large Language Models (LLMs) across various domains. However, this reasoning process has been confined exclusively to textual space, limiting its effectiveness in visually intensive tasks. To address this limitation, we introduce the concept of reasoning in the pixel-space. Within this novel framework, Vision-Language Models (VLMs) are equipped with a suite of visual reasoning operations, such as zoom-in and select-frame. These operations enable VLMs to directly inspect, interrogate, and infer from visual evidences, thereby enhancing reasoning fidelity for visual tasks. Cultivating such pixel-space reasoning capabilities in VLMs presents notable challenges, including the model's initially imbalanced competence and its reluctance to adopt the newly introduced pixel-space operations. We address these challenges through a two-phase training approach. The first phase employs instruction tuning on synthesized reasoning traces to familiarize the model with the novel visual operations. Following this, a reinforcement learning (RL) phase leverages a curiosity-driven reward scheme to balance exploration between pixel-space reasoning and textual reasoning. With these visual operations, VLMs can interact with complex visual inputs, such as information-rich images or videos to proactively gather necessary information. We demonstrate that this approach significantly improves VLM performance across diverse visual reasoning benchmarks. Our 7B model, model, achieves 84% on V* bench, 74% on TallyQA-Complex, and 84% on InfographicsVQA, marking the highest accuracy achieved by any open-source model to date. These results highlight the importance of pixel-space reasoning and the effectiveness of our framework.

Key facts. Cited by 267 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 43 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

33 of 43 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
  2. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  3. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  4. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale6 Dec
  5. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
  6. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun
  7. 2024CogCoM: A Visual Language Model with Chain-of-Manipulations Reasoning6 Feb
  8. 2023Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models5 Dec
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct

Other references · 24

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  9. 2025Gemma 3 Technical Reportcited by 1,686
  10. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  11. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  12. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
12 more
  1. 2017Constrained Policy Optimizationcited by 928
  2. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
  3. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  4. 2018TallyQA: Answering Complex Counting Questionscited by 152
  5. 2024LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecturecited by 100
  6. 2024Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Datacited by 67
  7. 2024MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Taskscited by 41
  8. 2024VISA: Retrieval Augmented Generation with Visual Source Attributioncited by 28
  9. 2024Instruction-Guided Visual Maskingcited by 17
  10. 2022ROI-Constrained Bidding via Curriculum-Guided Bayesian Reinforcement Learningcited by 12
  11. 2020Learning Context-aware Task Reasoning for Efficient Meta-reinforcement Learningcited by 11
  12. 2023Adversarial Constrained Bidding via Minimax Regret Optimization with Causality-Aware Reinforcement Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 97; 2026: 170.

267 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  2. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  4. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
  5. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 116
  6. 2025Latent Visual Reasoningcited by 99
  7. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 95
  8. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 92
  9. 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Modelscited by 83
  10. 2025Monet: Reasoning in Latent Visual Space Beyond Images and Languagecited by 75
  11. 2025PyVision: Agentic Vision with Dynamic Toolingcited by 71
  12. 2026Scaling Reasoning Efficiently via Relaxed On-Policy Distillationcited by 71
38 more
  1. 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 71
  2. 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 71
  3. 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 67
  4. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 58
  5. 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoningcited by 55
  6. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 47
  7. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learningcited by 46
  8. 2025Emergent Hierarchical Reasoning in LLMs through Reinforcement Learningcited by 42
  9. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
  10. 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMscited by 41
  11. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 39
  12. 2025MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agentscited by 35
  13. 2025TIR-Bench: A Comprehensive Benchmark for Agentic Thinking-with-Images Reasoningcited by 34
  14. 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
  15. 2025Reverse-Engineered Reasoning for Open-Ended Generationcited by 32
  16. 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedbackcited by 31
  17. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
  18. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 31
  19. 2025SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learningcited by 31
  20. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 28
  21. 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoningcited by 28
  22. 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoningcited by 27
  23. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  24. 2025Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Spacecited by 25
  25. 2025Simple o3: Towards Interleaved Vision-Language Reasoningcited by 24
  26. 2025CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimizationcited by 24
  27. 2025VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoningcited by 24
  28. 2025Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Toolscited by 23
  29. 2025Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Taskscited by 22
  30. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methodscited by 22
  31. 2026Thinking with Blueprints: Assisting Vision-Language Models in Spatial Reasoning via Structured Object Representationcited by 21
  32. 2025AdaTooler-V: Adaptive Tool-Use for Images and Videoscited by 20
  33. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
  34. 2025AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisitioncited by 19
  35. 2025DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Modelscited by 18
  36. 2025HSSBench: Benchmarking Humanities and Social Sciences Ability for Multimodal Large Language Modelscited by 17
  37. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsingcited by 17
  38. 2025On the Faithfulness of Visual Thinking: Measurement and Enhancementcited by 17

50 most cited of 267 All 267 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoning
  2. 2025PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning
  3. 2025RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learning
  4. 2025More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models
  5. 2026Pixelis: Reasoning in Pixels, from Seeing to Acting
  6. 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models
  7. 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
  8. 2026Vero: An Open RL Recipe for General Visual Reasoning