Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Achieving robust perception-reasoning synergy is a central goal for advanced Vision-Language Models (VLMs). Recent advancements have pursued this goal via architectural designs or agentic workflows. However, these approaches are often limited by static textual reasoning or complicated by the significant compute and engineering burden of external agentic complexity. Worse, this heavy investment does not yield proportional gains, often witnessing a "seesaw effect" on perception and reasoning. This motivates a fundamental rethinking of the true bottleneck. In this paper, we argue that the root cause of this trade-off is an ambiguity in modality credit assignment: when a VLM fails, is it due to flawed perception ("bad seeing") or flawed logic ("bad thinking")? To resolve this, we introduce a reinforcement learning framework that improves perception-reasoning synergy by reliably rewarding the perception fidelity. We explicitly decompose the generation process into interleaved perception and reasoning steps. This decoupling enables targeted supervision on perception. Crucially, we introduce Perception Verification (PV), leveraging a "blindfolded reasoning" proxy to reward perceptual fidelity independently of reasoning outcomes. Furthermore, to scale training across free-form VL tasks, we propose Structured Verbal Verification, which replaces high-variance LLM judging with structured algorithmic execution. These techniques are integrated into a Modality-Aware Credit Assignment (MoCA) mechanism, which routes rewards to the specific source of error -- either bad seeing or bad thinking -- enabling a single VLM to achieve simultaneous performance gains across a wide task spectrum.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 46 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
31 of 46 references.
Closest ancestors · 7
Same direction, newest first
- 2025MiMo-VL Technical Report4 Jun
- 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning21 May
- 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs21 May
- 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
- 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark9 Jan
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
Other references · 24
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2016Deep Variational Information Bottleneckcited by 930
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
- 2021InfographicVQAcited by 437
- 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301
- 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
- 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 198
12 more
- 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 183
- 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 179
- 2023Document Understanding Dataset and Evaluation (DUDE)cited by 135
- 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 96
- 2025Emergent Hierarchical Reasoning in LLMs through Reinforcement Learningcited by 42
- 2025VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Searchcited by 35
- 2025Reverse-Engineered Reasoning for Open-Ended Generationcited by 32
- 2025To Code or not to Code? Adaptive Tool Integration for Math Language Models via Expectation-Maximizationcited by 25
- 2026Thinking with Blueprints: Assisting Vision-Language Models in Spatial Reasoning via Structured Object Representationcited by 21
- 2026RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Timecited by 14
- 2025CogDoc: Towards Unified thinking in Documentscited by 3
- 2025Language-Driven Object-Oriented Two-Stage Method for Scene Graph Anticipationcited by 1
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generationcited by 8
- 2026SpecPath: Testing Coding Agents Across Contract-Equivalent Specification Historiescited by 1
- 2026Starve to Perceive: Taming Lazy Perception in VLMs with Constrained Visual Bandwidthcited by 0
- 2026Reach or Solve? Attributing Agentic RL Gains with Checkpoint Handoffscited by 0
- 2026When Should a VLM Look? Paying Only for Visual Calls That Were Needed and Usedcited by 0
- 2026Certified Interface Aliases: Exact Collisions in Vision-Language Preprocessing, and When They Existcited by 0
- 2026Unmentioned Checklist Findings Change How Reinforcement Learning Appears to Improve Chest Radiograph Report Checkingcited by 0
- 2026Unmentioned Checklist Findings Change How Reinforcement Learning Appears to Improve Chest Radiograph Report Checking4 Oct 2026
- 2026Certified Interface Aliases: Exact Collisions in Vision-Language Preprocessing, and When They Exist26 Sep 2026
- 2026When Should a VLM Look? Paying Only for Visual Calls That Were Needed and Used19 Sep 2026
- 2026Reach or Solve? Attributing Agentic RL Gains with Checkpoint Handoffs17 Sep 2026
- 2026SpecPath: Testing Coding Agents Across Contract-Equivalent Specification Histories10 Aug 2026
- 2026Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation6 Jul 2026
- 2026Starve to Perceive: Taming Lazy Perception in VLMs with Constrained Visual Bandwidth18 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025Perceptual-Evidence Anchored Reinforced Learning for Multimodal Reasoning
- 2025VTPerception-R1: Enhancing Multimodal Reasoning via Explicit Visual and Textual Perceptual Grounding
- 2026PDCR: Perception-Decomposed Confidence Reward for Vision-Language Reasoning
- 2026Thinking with Deltas: Incentivizing Reinforcement Learning via Differential Visual Reasoning Policy
- 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
- 2026Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs
- 2026Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention
- 2026Disentangling Perception and Reasoning in Multimodal LLMs via Reward Design