mlsanityData to 7 Oct 2026

Anchoring What Matters: A Dual-Level Learning Framework for Visually-Grounded Multimodal Reasoning

arXiv 2609.18057 ↗First posted 16 Sep 2026cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) has significantly improved the reasoning capabilities of large vision-language models (LVLMs). However, standard on-policy RLVR algorithms face a critical optimization bottleneck in preserving and reinforcing visually grounded reasoning behaviors: valuable visually-grounded reasoning trajectories are discarded after a single update, while uniform token advantage allocation prevents the model from reinforcing critical perception or reasoning steps. To bridge this gap, we propose PIVOT, a dual-level learning framework that anchors policy optimization around informative visual reasoning signals. Specifically, PIVOT introduces a self-calibrated experience replay mechanism, which selectively collects and replays visually-grounded historical experiences as stable reference anchors for policy optimization. Building upon this, we further design a vision-guided advantage allocation mechanism to allocate additional vision-aware advantages to tokens based on their local visual support and impact on downstream reasoning. Extensive experiments across diverse benchmarks demonstrate that PIVOT achieves highly competitive performance in enhancing the multimodal reasoning capabilities of LVLMs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 12 of its 43 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

12 of 43 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Visually-Guided Policy Optimization for Multimodal Reasoning10 Apr
  2. 2026Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models2 Apr
  3. 2026Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs26 Mar
  4. 2026Rethinking Token-Level Policy Optimization for Multimodal Chain-of-Thought24 Mar
  5. 2026Credit Where It is Due: Cross-Modality Connectivity Drives Precise Reinforcement Learning for MLLM Reasoning12 Feb
  6. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning10 Mar
  7. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul

Other references · 5

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  3. 2025RLEP: Reinforcement Learning with Experience Replay for LLM Reasoningcited by 35
  4. 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Modelscited by 35
  5. 2025EFRame: Deeper Reasoning via Exploration-Filter-Replay Reinforcement Learning Frameworkcited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs
  2. 2026IVR-R1: Refining Trajectories through Iterative Visual-Grounded Reasoning in Reinforcement Learning
  3. 2026Beyond Where to Look: Trajectory-Guided Reinforcement Learning for Multimodal RLVR
  4. 2025More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models
  5. 2026Reinforcing Multimodal Reasoning via Token-Level Perception-Grounded Advantage Estimation
  6. 2026PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping
  7. 2025MMRPT: MultiModal Reinforcement Pre-Training via Masked Vision-Dependent Reasoning
  8. 2025Spotlight on Token Perception for Multimodal Reinforcement Learning