mlsanityData to 14 Oct 2026

DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoning

arXiv 2509.25866 ↗First posted 30 Sep 2025cs.CVcited by 14

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

The "thinking with images" paradigm represents a pivotal shift in the reasoning of Vision Language Models (VLMs), moving from text-dominant chain-of-thought to image-interactive reasoning. By invoking visual tools or generating intermediate visual representations, VLMs can iteratively attend to fine-grained regions, enabling deeper image understanding and more faithful multimodal reasoning. As an emerging paradigm, however, it still leaves substantial room for exploration in data construction accuracy, structural design, and broader application scenarios, which offer rich opportunities for advancing multimodal reasoning. To further advance this line of work, we present DeepSketcher, a comprehensive suite comprising both an image-text interleaved dataset and a self-contained model. The dataset contains 31k chain-of-thought (CoT) reasoning trajectories with diverse tool calls and resulting edited images, covering a wide range of data types and manipulation instructions with high annotation accuracy. Building on this resource, we design a model that performs interleaved image-text reasoning and natively generates "visual thoughts" by operating directly in the visual embedding space, rather than invoking external tools and repeatedly re-encoding generated images. This design enables tool-free and more flexible "thinking with images". Extensive experiments on multimodal reasoning benchmarks demonstrate strong performance, validating both the utility of the dataset and the effectiveness of the model design.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 45 of its 61 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

45 of 61 references.

Closest ancestors · 24

Same direction, newest first

  1. 2025Self-Rewarding Vision-Language Model via Reasoning Decomposition27 Aug
  2. 2025Thyme: Think Beyond Images15 Aug
  3. 2025Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning22 Jul
  4. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens20 Jun
  5. 2025Metis-RISE: RL Incentivizes and SFT Enhances Multimodal Reasoning Model Learning16 Jun
  6. 2025ChartReasoner: Code-Driven Modality Bridging for Long-Chain Reasoning in Chart Question Answering11 Jun
  7. 2025MiMo-VL Technical Report4 Jun
  8. 2025Thinking with Generated Images28 May
  9. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
  10. 2025MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning15 May
  11. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
  12. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models10 Apr
12 more
  1. 2025Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoning17 Mar
  2. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization13 Mar
  3. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning10 Mar
  4. 2025ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding9 Jan
  5. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
  6. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?1 Jul
  7. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun
  8. 2024MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classification7 Apr
  9. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  11. 2023ViperGPT: Visual Inference via Python Execution for Reasoning14 Mar
  12. 2022Visual Programming: Compositional visual reasoning without training18 Nov

Other references · 21

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  5. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  6. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  7. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  8. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  9. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  10. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301
  11. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 228
  12. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
9 more
  1. 2023ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Promptscited by 161
  2. 2022UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expressioncited by 137
  3. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 136
  4. 2025OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoningcited by 69
  5. 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generationcited by 58
  6. 2024List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMscited by 32
  7. 2024GeoGPT4V: Towards Geometric Multi-modal Large Language Models with Geometric Image Generationcited by 32
  8. 2025Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMscited by 23
  9. 2025Effective Training Data Synthesis for Improving MLLM Chart Understandingcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 9.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoning
  2. 2025Thinking with Programming Vision: Towards a Unified View for Thinking with Images
  3. 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forward
  4. 2026Reasoning with Image Generation
  5. 2025VGR: Visual Grounded Reasoning
  6. 2025SIFThinker: Spatially-Aware Image Focus for Visual Reasoning
  7. 2025VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning
  8. 2026Distilling Visual Reasoning into Text Space