mlsanityData to 14 Oct 2026

Residual Visual Credit Optimization: Conserved Evidence Routing for Multimodal Reinforcement Learning

arXiv 2610.04918 ↗First posted 4 Oct 2026cs.LG cs.CLcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Reinforcement learning with verifiable rewards scales multimodal reasoning, but an outcome reward says how much a trajectory is worth, not how that value should be spread over the decisions that produced it. We introduce Residual Visual Credit Optimization (RVCO), which treats token credit as a conserved routing problem. A controlled visual intervention yields a per-token evidence response; robust within-trajectory coordinates remove incidental scale; and a budgeted entropic router distributes a fixed amount of sequence utility according to perceptual dependence. A residual support path guarantees positive credit at every valid position, and an analytic correction restores the prescribed credit mass exactly. The resulting field is selective, bounded, full-support, and invariant to response-local score shifts, and recovers hard token selection as a limiting case. Across four model families and seven reasoning benchmarks, RVCO improves accuracy over strong RLVR baselines while maintaining late-stage optimization stability, corruption robustness, and competitive training cost. Rewards, rollouts, and the group-relative advantage estimator are unchanged; only the geometry of token-level credit differs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 44 of its 51 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

44 of 51 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs17 Jun
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  3. 2023Let's Verify Step by Step31 May
  4. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2025Qwen3-VL Technical Reportcited by 2,793
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  8. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  9. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  10. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  11. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  12. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
28 more
  1. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  2. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  3. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  4. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  5. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  6. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  7. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  8. 2021InfographicVQAcited by 437
  9. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 383
  10. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 323
  11. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contextscited by 245
  12. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 208
  13. 2024DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Modelscited by 146
  14. 2019Learning with Fenchel-Young Lossescited by 119
  15. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 99
  16. 2025SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Modelscited by 66
  17. 2023UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformerscited by 54
  18. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 39
  19. 2025Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoningcited by 32
  20. 2019The Limited Multi-Label Projection Layercited by 22
  21. 2025VisNumBench: Evaluating Number Sense of Multimodal Large Language Modelscited by 14
  22. 2026Visually-Guided Policy Optimization for Multimodal Reasoningcited by 11
  23. 2026Rethinking Token-Level Policy Optimization for Multimodal Chain-of-Thoughtcited by 9
  24. 2026VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoningcited by 8
  25. 2026Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Modelscited by 7
  26. 2026PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshapingcited by 5
  27. 2026Learning Self-Correction in Vision-Language Models via Rollout Augmentationcited by 4
  28. 2026Improving Vision-language Models with Perception-centric Process Reward Modelscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Visual sensitivity is not claim retractability: persistence-aware credit assignment for multimodal reinforcement learning
  2. 2026Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection
  3. 2026Structured Role-Aware Policy Optimization for Multimodal Reasoning
  4. 2026Beyond Entropy: Self-Diagnostic Multi-Role Token Optimization for Video Reasoning
  5. 2026SIVA-RL: Sensitivity-Invariance Visual Alignment for Multimodal Reinforcement Learning
  6. 2026Credit Where It is Due: Cross-Modality Connectivity Drives Precise Reinforcement Learning for MLLM Reasoning
  7. 2026IVR-R1: Refining Trajectories through Iterative Visual-Grounded Reasoning in Reinforcement Learning
  8. 2026Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs