mlsanityData to 7 Oct 2026

Learning When to Look: A Disentangled Curriculum for Strategic Perception in Multimodal Reasoning

arXiv 2512.17227 ↗First posted 19 Dec 2025cs.CVcited by 5

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) demonstrate significant potential but remain brittle in complex, long-chain visual reasoning tasks. A critical failure mode is "visual forgetting", where models progressively lose visual grounding as reasoning extends, a phenomenon aptly described as "think longer, see less". We posit this failure stems from current training paradigms prematurely entangling two distinct cognitive skills: (1) abstract logical reasoning "how-to-think") and (2) strategic visual perception ("when-to-look"). This creates a foundational cold-start deficiency -- weakening abstract reasoning -- and a strategic perception deficit, as models lack a policy for when to perceive. In this paper, we propose a novel curriculum-based framework to disentangle these skills. First, we introduce a disentangled Supervised Fine-Tuning (SFT) curriculum that builds a robust abstract reasoning backbone on text-only data before anchoring it to vision with a novel Perception-Grounded Chain-of-Thought (PG-CoT) paradigm. Second, we resolve the strategic perception deficit by formulating timing as a reinforcement learning problem. We design a Pivotal Perception Reward that teaches the model when to look by coupling perceptual actions to linguistic markers of cognitive uncertainty (e.g., "wait", "verify"), thereby learning an autonomous grounding policy. Our contributions include the formalization of these two deficiencies and the development of a principled, two-stage framework to address them, transforming the model from a heuristic-driven observer to a strategic, grounded reasoner. Code: urlhttps://github.com/gaozilve-max/learning-when-to-look.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 54 references that are in the map. First posted on arXiv 19 Dec 2025.

Builds on

36 of 54 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models30 Sep
  2. 2025MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources25 Sep
  3. 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models16 Sep
  4. 2025Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation18 Aug
  5. 2025We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning14 Aug
  6. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoning2 Jul
  7. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward8 Jun
  8. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning4 Jun
  9. 2025Grounded Reinforcement Learning for Visual Reasoning29 May
  10. 2025Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning26 May
  11. 2025v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning24 May
  12. 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning20 May
11 more
  1. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models10 Apr
  2. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
  3. 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learning10 Apr
  4. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement10 Apr
  5. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles21 Mar
  6. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning10 Mar
  7. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model7 Mar
  8. 2024DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models29 Oct
  9. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  11. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoning10 Aug

Other references · 13

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  6. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  7. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  8. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  9. 2025Kimi K2: Open Agentic Intelligencecited by 523
  10. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  11. 2025Seed1.5-VL Technical Reportcited by 367
  12. 2025ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shapingcited by 16
1 more
  1. 2025GeoPQA: Bridging the Visual Perception Gap in MLLMs for Geometric Reasoningcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs
  2. 2026Visually-Guided Policy Optimization for Multimodal Reasoning
  3. 2025Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space
  4. 2026From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models
  5. 2026Distilling Visual Reasoning into Text Space
  6. 2026Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning
  7. 2025Thinking with Visual Abstract: Enhancing Multimodal Reasoning via Visual Abstraction
  8. 2026Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models