Credit Where It is Due: Cross-Modality Connectivity Drives Precise Reinforcement Learning for MLLM Reasoning
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced the reasoning capabilities of Multimodal Large Language Models (MLLMs), yet how visual evidence is integrated during reasoning remains poorly understood. We explore multimodal RLVR through the lens of cross-modal attention connectivity and find that only a small fraction of tokens (approximately 15%) exhibit strong visual-textual coupling. These high-connectivity tokens act as anchors that ground reasoning in the image, while the majority follow linguistic patterns. During RLVR training, credit assignment naturally concentrates on these anchors, sharpening their visual grounding over time. Building on this insight, we propose Anchor-Token Reinforcement Learning (AT-RL), a lightweight framework that selectively reinforces high-connectivity tokens via graph-based clustering of attention topology. Evaluated across the series (3B-32B), AT-RL introduces only 1.2% overhead yet enables the 32B model to surpass the 72B-Instruct baseline on MathVista (80.2), with consistent gains observed across STEM, video and general tasks. Conversely, training solely on low-connectivity tokens causes severe degradation, confirming that effective multimodal RL hinges on precise credit assignment to visual anchors. Our work reveals that reasoning quality is governed not by token quantity but by the fidelity of cross-modal anchoring.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 43 of its 59 references that are in the map. First posted on arXiv 12 Feb 2026.
Builds on
43 of 59 references.
Closest ancestors · 10
Same direction, newest first
- 2026Video-KTR: Reinforcing Video Reasoning via Key Token Attribution27 Jan
- 2026SketchVL: Policy Optimization via Fine-Grained Credit Assignment for Chart Understanding and More9 Jan
- 2026MMErroR: A Benchmark for Erroneous Reasoning in Vision-Language Models6 Jan
- 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoning14 Oct
- 2025BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception10 Oct
- 2025Spotlight on Token Perception for Multimodal Reinforcement Learning10 Oct
- 2025We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning14 Aug
- 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark9 Jan
- 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation14 Jun
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
Other references · 33
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2025Group Sequence Policy Optimizationcited by 788
- 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
- 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
- 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
- 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 376
- 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
- 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 225
21 more
- 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 224
- 2024Token-level Direct Preference Optimizationcited by 135
- 2025MMVU: Measuring Expert-Level Multi-Discipline Video Understandingcited by 131
- 2025Geometric-Mean Policy Optimizationcited by 107
- 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answeringcited by 105
- 2025Soft Adaptive Policy Optimizationcited by 89
- 2024Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capabilitycited by 80
- 2025TreeRPO: Tree Relative Policy Optimizationcited by 45
- 2024MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Taskscited by 41
- 2025OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpaintingcited by 38
- 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
- 2025GVPO: Group Variance Policy Optimization for Large Language Model Post-Trainingcited by 32
- 2025CE-GPPO: Coordinating Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learningcited by 23
- 2025GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Controlcited by 18
- 2025Group Critical-token Policy Optimization for Autoregressive Image Generationcited by 16
- 2025MAPO: Mixed Advantage Policy Optimizationcited by 11
- 2025GRPO is Secretly a Process Reward Modelcited by 10
- 2025GeoPQA: Bridging the Visual Perception Gap in MLLMs for Geometric Reasoningcited by 8
- 2025Zoom-Zero: Reinforced Coarse-to-Fine Video Understanding via Temporal Zoom-incited by 8
- 2025Attention Debiasing for Token Pruning in Vision Language Modelscited by 3
- 2024A Soft e-Textile Sensor for Enhanced Deep Learning-based Shape Sensing of Soft Continuum Robotscited by 1
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026DyCo-RL: Dynamic Cross-Modal Coordination for Visual Reasoningcited by 1
- 2026Bridging Visual Representation and Reinforcement Learning from Verifiable Rewards in Large Vision-Language Modelscited by 0
- 2026PointVG-R: Internalizing Geometric Reasoning in MLLMs for Precise Pointing Localization via Visual Chain of Thoughtcited by 0
- 2026Anchoring What Matters: A Dual-Level Learning Framework for Visually-Grounded Multimodal Reasoningcited by 0
- 2026Visual sensitivity is not claim retractability: persistence-aware credit assignment for multimodal reinforcement learningcited by 0
- 2026Advancing Entropy-Level Credit Assignment in RLVR via Proximal Entropy Policy Optimizationcited by 0
- 2026Advancing Entropy-Level Credit Assignment in RLVR via Proximal Entropy Policy Optimization30 Sep 2026
- 2026Visual sensitivity is not claim retractability: persistence-aware credit assignment for multimodal reinforcement learning29 Sep 2026
- 2026Anchoring What Matters: A Dual-Level Learning Framework for Visually-Grounded Multimodal Reasoning16 Sep 2026
- 2026PointVG-R: Internalizing Geometric Reasoning in MLLMs for Precise Pointing Localization via Visual Chain of Thought23 Jun 2026
- 2026DyCo-RL: Dynamic Cross-Modal Coordination for Visual Reasoning6 Jun 2026
- 2026Bridging Visual Representation and Reinforcement Learning from Verifiable Rewards in Large Vision-Language Models28 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention
- 2026Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs
- 2026Reinforcing Multimodal Reasoning via Token-Level Perception-Grounded Advantage Estimation
- 2026Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs
- 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
- 2026Visually-Guided Policy Optimization for Multimodal Reasoning
- 2026Rethinking Token-Level Policy Optimization for Multimodal Chain-of-Thought
- 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use