mlsanityData to 14 Oct 2026

v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning

arXiv 2505.18842 ↗First posted 24 May 2025cs.CL cs.CVcited by 25

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

When thinking with images, humans rarely rely on a single glance: they revisit visual evidence while reasoning. In contrast, most Multimodal Language Models encode an image once to key-value cache and then reason purely in text, making it hard to re-ground intermediate steps. We empirically confirm this: as reasoning chains lengthen, models progressively lose focus on relevant regions. We introduce v1, a lightweight extension for active visual referencing via point-and-copy: the model selects relevant image patches and copies their embeddings back into the reasoning stream. Crucially, our point-and-copy mechanism retrieves patches using their semantic representations as keys, ensuring perceptual evidence remains aligned with the reasoning space. To train this behavior, we build v1g, a dataset of 300K multimodal reasoning traces with interleaved grounding annotations. Across multimodal mathematical reasoning benchmarks, v1 consistently outperforms comparable baselines. We release our code, model, and data.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 32 of its 55 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

32 of 55 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning20 May
  2. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
  3. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement10 Apr
  4. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  5. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles21 Mar
  6. 2025Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoning17 Mar
  7. 2025Grounded Chain-of-Thought for Multimodal Large Language Models17 Mar
  8. 2025Visualizing Thought: Conceptual Diagrams Enable Robust Planning in LMMs14 Mar
  9. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning10 Mar
  10. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  11. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
  12. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark9 Jan
5 more
  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  2. 2024Interleaved-Modal Chain-of-Thought29 Nov
  3. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
  4. 2024Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models19 Mar
  5. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action20 Mar

Other references · 15

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2025Gemma 3 Technical Reportcited by 1,824
  9. 2025s1: Simple test-time scalingcited by 1,341
  10. 2024OpenAI o1 System Cardcited by 1,244
  11. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  12. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
3 more
  1. 2024PaliGemma 2: A Family of Versatile VLMs for Transfercited by 207
  2. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
  3. 2025Teaching Metric Distance to Discrete Autoregressive Language Modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 11.

25 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  2. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  4. 2025Monet: Reasoning in Latent Visual Space Beyond Images and Languagecited by 82
  5. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  6. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 29
  7. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  8. 2025Simple o3: Towards Interleaved Vision-Language Reasoningcited by 24
  9. 2026Embed-RL: Reinforcement Learning for Reasoning-Driven Multimodal Embeddingscited by 24
  10. 2025CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoningcited by 16
  11. 2025Video-CoM: Interactive Video Reasoning via Chain of Manipulationscited by 9
  12. 2025Position: Reasoning After Perception Means Reasoning Without Visioncited by 8
13 more
  1. 2025What MLLMs Learn about When they Learn about Multimodal Reasoningcited by 6
  2. 2025Learning When to Look: A Disentangled Curriculum for Strategic Perception in Multimodal Reasoningcited by 5
  3. 2026Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thoughtcited by 4
  4. 2026Multimodal Latent Reasoning via Predictive Embeddingscited by 4
  5. 2026Thinking With Tools, Not With Pixels: Tool Calls as Text Scaffolds for Visual Reasoningcited by 3
  6. 2025TRACE: Textual Reasoning for Affordance Coordinate Extractioncited by 1
  7. 2026VisRef: Visual Refocusing while Thinking Improves Test-Time Scaling in Multi-Modal Large Reasoning Modelscited by 1
  8. 2026ReflectCAP: Detailed Image Captioning with Reflective Memorycited by 1
  9. 2026LatentGeo: Learnable Auxiliary Constructions in Latent Space for Multimodal Geometric Reasoningcited by 0
  10. 2026Simulating Validity: Modal Decoupling in MLLM Generated Feedback on Science Drawingscited by 0
  11. 2026Reflection Anchors for Propagation-Aware Visual Retention in Long-Chain Multimodal Reasoningcited by 0
  12. 2026Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learningcited by 0
  13. 2026Grounding What Shapes the Plan: Rethinking Groundedness for Physical Intelligence in Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VGR: Visual Grounded Reasoning
  2. 2026Distilling Visual Reasoning into Text Space
  3. 2025LaRe: Latent Refocusing for Multimodal Reasoning
  4. 2025Revisiting Visual Understanding in Multimodal Reasoning through a Lens of Image Perturbation
  5. 2025ChainV: Atomic Visual Hints Make Multimodal Reasoning Shorter and Better
  6. 2025Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning
  7. 2026See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs
  8. 2026Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning