mlsanityData to 14 Oct 2026

Visual sensitivity is not claim retractability: persistence-aware credit assignment for multimodal reinforcement learning

arXiv 2609.36572 ↗First posted 29 Sep 2026cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models also fits: VLM benchmarks and visual reasoning

also in 1 more
Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) has been extended to Large Vision-Language Models (LVLMs), and perception-aware methods further encourage policies to rely on visual evidence. Yet relying on the image does not guarantee that visual claims are supported by it. Before RL training, 27.81% of the correctly answered responses of Qwen2.5-VL-7B on four multimodal reasoning benchmarks contain at least one direct visual claim that the image does not support. Since outcome-level RL rewards each response as a whole, these claims inherit the positive credit of the correct answer. We introduce a fixed-rollout counterfactual diagnostic that re-scores the same response under an intervened image to separate Evidence-Function Sensitivity (EFS), how strongly the model's predictions change, from claim persistence, whether the model keeps supporting the same claim rather than retracting it. The diagnostic reveals Sensitivity-Persistence Decoupling (SPD): under DAPO and VPPO, EFS increases and claims become more retractable overall, yet unsupported claims become significantly more persistent, whereas GRPO raises EFS without this deterioration. We therefore propose Persistence-Aware Credit Gating (PACG), which attenuates positive credit for unusually persistent visual claims and leaves all other credit unchanged. It requires no supported/unsupported labels and adds no inference cost. On Qwen2.5-VL-7B, PACG raises the nine-benchmark average over three seeds from 58.1% to 59.9% with DAPO and from 59.8% to 60.9% with VPPO, while making unsupported claims more retractable. The gains extend to a larger model, a newer backbone, and the accuracy of HallusionBench also improves consistently. These results suggest that visual sensitivity and claim retractability are complementary dimensions of multimodal credit assignment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 52 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

47 of 52 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026CFPO: Counterfactual Policy Optimization for Multimodal Reasoning22 Jun
  2. 2026PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping7 Jun
  3. 2026Structured Role-Aware Policy Optimization for Multimodal Reasoning8 May
  4. 2026Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs26 Mar
  5. 2026Credit Where It is Due: Cross-Modality Connectivity Drives Precise Reinforcement Learning for MLLM Reasoning12 Feb
  6. 2025Spotlight on Token Perception for Multimodal Reinforcement Learning10 Oct
  7. 2025RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learning2 Oct
  8. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward8 Jun
  9. 2025R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO22 May
  10. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation17 Apr
  11. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
  12. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization17 Mar
2 more
  1. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  2. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar

Other references · 33

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  3. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  4. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  5. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  6. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  7. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  8. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  9. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  10. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  11. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  12. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 478
21 more
  1. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  2. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 383
  3. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 303
  4. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluationcited by 300
  5. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 286
  6. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contextscited by 245
  7. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 198
  8. 2024DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Modelscited by 146
  9. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 145
  10. 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 131
  11. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 99
  12. 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 86
  13. 2024Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignmentcited by 42
  14. 2025Treble Counterfactual VLMs: A Causal Approach to Hallucinationcited by 35
  15. 2025GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoningcited by 14
  16. 2025Explanation-Driven Counterfactual Testing for Faithfulness in Vision-Language Model Explanationscited by 12
  17. 2026SketchVL: Policy Optimization via Fine-Grained Credit Assignment for Chart Understanding and Morecited by 7
  18. 2026What, Whether and How? Unveiling Process Reward Models for Thinking with Images Reasoningcited by 7
  19. 2026Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMscited by 7
  20. 2026Evidence-RL: Towards Evidence-intensive Visual Reasoningcited by 3
  21. 2026Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Perceptual-Evidence Anchored Reinforced Learning for Multimodal Reasoning
  2. 2026SIVA-RL: Sensitivity-Invariance Visual Alignment for Multimodal Reinforcement Learning
  3. 2026Disentangling Perception and Reasoning in Multimodal LLMs via Reward Design
  4. 2026Residual Visual Credit Optimization: Conserved Evidence Routing for Multimodal Reinforcement Learning
  5. 2026Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention
  6. 2026Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning
  7. 2026Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning
  8. 2026Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models