mlsanityData to 14 Oct 2026

Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection

arXiv 2606.03937 ↗First posted 2 Jun 2026cs.AIcited by 1

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

While token-level entropy is commonly recognized as effective for credit assignment in text-only reinforcement learning with verifiable rewards (RLVR), it remains unclear whether this mechanism still holds in visual reasoning. Our controlled study shows that this mechanism collapses in visual reasoning due to the omission of vision-sensitive tokens with naturally low entropy. Although existing multimodal RL methods increasingly acknowledge the importance of visual perception, they struggle to satisfy the inherent demand for interleaving precise perceptual grounding with semantic reasoning, either lacking systematic visual measurements or overlooking that token entropy primarily drives semantic exploration. To address this, we introduce VEPO (Vision-Entropy token-selection for Policy Optimization), an effective RL framework explicitly integrating visual sensitivity with token entropy via a principled multiplicative coupling, where VEPO redirects gradient credit toward tokens which are simultaneously visually grounded and highly informative. Extensive experiments demonstrate VEPO's leading performance, significantly outperforming the entropy-only baseline by 2.28 points at 7B-scale and 3.15 points at 3B-scale. Ablations further substantiate the soundness of our method.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 64 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

34 of 64 references.

Closest ancestors · 17

Same direction, newest first

  1. 2026Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models2 Apr
  2. 2026Rethinking Token-Level Policy Optimization for Multimodal Chain-of-Thought24 Mar
  3. 2026VisRef: Visual Refocusing while Thinking Improves Test-Time Scaling in Multi-Modal Large Reasoning Models27 Feb
  4. 2026Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs9 Feb
  5. 2026Reinforced Attention Learning4 Feb
  6. 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning30 Oct
  7. 2025Spotlight on Token Perception for Multimodal Reinforcement Learning10 Oct
  8. 2025What MLLMs Learn about When they Learn about Multimodal Reasoning2 Oct
  9. 2025Perception-Aware Policy Optimization for Multimodal Reasoning8 Jul
  10. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers30 Jun
  11. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
  12. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation17 Apr
5 more
  1. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
  2. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning10 Mar
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  4. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?1 Jul
  5. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb

Other references · 17

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2024OpenAI o1 System Cardcited by 1,244
  6. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  7. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 485
  8. 2024DPO Meets PPO: Reinforced Token Optimization for RLHFcited by 127
  9. 2026TIP: Token Importance in On-Policy Distillationcited by 104
  10. 2025The Art of Scaling Reinforcement Learning Compute for LLMscited by 101
  11. 2025Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMscited by 55
  12. 2025Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agentscited by 46
5 more
  1. 2025Seed-Prover 1.5: Mastering Undergraduate-Level Theorem Proving via Learning from Experiencecited by 44
  2. 2025Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVRcited by 39
  3. 2026Where Hindsight Credit Can Reside: A Signed-Capacity View of Token Updates in RLVRcited by 8
  4. 2025The Role of Entropy in Visual Grounding: Analysis and Optimizationcited by 1
  5. 2026Where Reasoning Breaks: Logic-Aware Path Selection by Controlling Logical Connectives in LLMs Reasoning Chainscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Reinforcing Multimodal Reasoning via Token-Level Perception-Grounded Advantage Estimation
  2. 2026PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping
  3. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
  4. 2026Visually-Guided Policy Optimization for Multimodal Reasoning
  5. 2026Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs
  6. 2026Thinking with Deltas: Incentivizing Reinforcement Learning via Differential Visual Reasoning Policy
  7. 2026Credit Where It is Due: Cross-Modality Connectivity Drives Precise Reinforcement Learning for MLLM Reasoning
  8. 2025Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward