Thinking with Programming Vision: Towards a Unified View for Thinking with Images
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Multimodal large language models (MLLMs) that think with images can interactively use tools to reason about visual inputs, but current approaches often rely on a narrow set of tools with limited real-world necessity and scalability. In this work, we first reveal a critical and previously overlooked weakness: even state-of-the-art MLLMs are surprisingly brittle, showing significant performance degradation on images with simple orientation changes or natural corruptions, underscoring the need for more robust tool-based reasoning. To address this, we propose CodeVision, a flexible and scalable code-as-tool framework where the model generates code as a universal interface to invoke any image operation, moving beyond fixed tool registries. We train our model using a two-stage methodology, beginning with Supervised Fine-Tuning (SFT) on a high-quality dataset curated for complex, multi-turn tool composition and error recovery, followed by Reinforcement Learning (RL) with a novel and dense process reward function to encourage strategic and efficient tool use. To facilitate this research, we construct new SFT and RL datasets and introduce a challenging new benchmark suite designed to rigorously evaluate robustness to orientation changes and multi-tool reasoning. Experiments on Qwen2.5-VL and Qwen3-VL series show that our approach significantly improves model performance and fosters emergent capabilities such as flexible tool composition, efficient chained execution, and robust error recovery from runtime feedback. Code is available at https://github.com/ByteDance-BandAI/CodeVision.
Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 24 of its 38 references that are in the map. First posted on arXiv 3 Dec 2025.
Builds on
24 of 38 references.
Closest ancestors · 9
Same direction, newest first
- 2025Thyme: Think Beyond Images15 Aug
- 2025We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning14 Aug
- 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning21 May
- 2025GRIT: Teaching MLLMs to Think with Images21 May
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
- 2025Visual Agentic Reinforcement Fine-Tuning20 May
- 2025Observe-R1: Unlocking Reasoning Abilities of MLLMs with Dynamic Progressive Reinforcement Learning18 May
- 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
- 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement10 Apr
Other references · 15
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
- 2025Group Sequence Policy Optimizationcited by 835
- 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
- 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 120
- 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answeringcited by 108
- 2025Soft Adaptive Policy Optimizationcited by 96
3 more
- 2025Tongyi DeepResearch Technical Reportcited by 66
- 2025APO: Enhancing Reasoning Ability of MLLMs via Asymmetric Policy Optimizationcited by 9
- 2025LLM-I: LLMs are Naturally Interleaved Multimodal Creatorscited by 6
Built on by
Citing papers per year: 2026: 15.
15 papers.
Sort:
By area:
- 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 75
- 2026PyVision-RL: Forging Open Agentic Vision Models via RLcited by 16
- 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligencecited by 6
- 2026AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generationcited by 5
- 2026Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimizationcited by 5
- 2026Dissecting Failure Dynamics in Large Language Model Reasoningcited by 2
- 2026SPyCE: Skill-Policy Co-evolution for Multimodal Agentscited by 2
- 2026Self-Evolving Code-with-Image Reasoningcited by 2
- 2026SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planningcited by 1
- 2026Enhancing MLLM Spatial Understanding via Active 3D Scene Exploration for Multi-Perspective Reasoningcited by 1
- 2026OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectoriescited by 1
- 2026ToolVision: Learning When and How to Use Visual Tools with Capability-Aligned Supervisioncited by 1
- 2026SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows1 Oct 2026
- 2026EviRover: Reinforcing Agentic Perception Beyond a Glance30 Sep 2026
- 2026Self-Evolving Code-with-Image Reasoning11 Aug 2026
- 2026ToolVision: Learning When and How to Use Visual Tools with Capability-Aligned Supervision9 Aug 2026
- 2026OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories9 Aug 2026
- 2026SPyCE: Skill-Policy Co-evolution for Multimodal Agents15 Jul 2026
- 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence14 Jun 2026
- 2026InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward26 May 2026
- 2026Dissecting Failure Dynamics in Large Language Model Reasoning16 Apr 2026
- 2026Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimization8 Apr 2026
- 2026Enhancing MLLM Spatial Understanding via Active 3D Scene Exploration for Multi-Perspective Reasoning8 Apr 2026
- 2026SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning24 Mar 2026
- 2026Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimizationcited by 5
- 2026Dissecting Failure Dynamics in Large Language Model Reasoningcited by 2
- 2026SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planningcited by 1
- 2026Enhancing MLLM Spatial Understanding via Active 3D Scene Exploration for Multi-Perspective Reasoningcited by 1
- 2026InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Rewardcited by 0
- 2026SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flowscited by 0
- 2026SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows1 Oct 2026
- 2026InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward26 May 2026
- 2026Dissecting Failure Dynamics in Large Language Model Reasoning16 Apr 2026
- 2026Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimization8 Apr 2026
- 2026Enhancing MLLM Spatial Understanding via Active 3D Scene Exploration for Multi-Perspective Reasoning8 Apr 2026
- 2026SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning24 Mar 2026
- 2026PyVision-RL: Forging Open Agentic Vision Models via RLcited by 16
- 2026EviRover: Reinforcing Agentic Perception Beyond a Glancecited by 0
- 2026EviRover: Reinforcing Agentic Perception Beyond a Glance30 Sep 2026
- 2026PyVision-RL: Forging Open Agentic Vision Models via RL24 Feb 2026
- 2026SPyCE: Skill-Policy Co-evolution for Multimodal Agentscited by 2
- 2026SPyCE: Skill-Policy Co-evolution for Multimodal Agents15 Jul 2026
- 2026Self-Evolving Code-with-Image Reasoningcited by 2
- 2026Self-Evolving Code-with-Image Reasoning11 Aug 2026
- 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 75
- 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligencecited by 6
- 2026OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectoriescited by 1
- 2026ToolVision: Learning When and How to Use Visual Tools with Capability-Aligned Supervisioncited by 1
- 2026ToolVision: Learning When and How to Use Visual Tools with Capability-Aligned Supervision9 Aug 2026
- 2026OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories9 Aug 2026
- 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence14 Jun 2026
- 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agents12 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoning
- 2026Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought
- 2025DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoning
- 2026Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs
- 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback
- 2026MM-OPD: Towards One More Bottleneck Between Perception and Reasoning
- 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forward
- 2026LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models