mlsanityData to 7 Oct 2026

MVR: Multi-view Video Reward Shaping for Reinforcement Learning

arXiv 2603.01694 ↗First posted 2 Mar 2026cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Reward design is of great importance for solving complex tasks with reinforcement learning. Recent studies have explored using image-text similarity produced by vision-language models (VLMs) to augment rewards of a task with visual feedback. A common practice linearly adds VLM scores to task or success rewards without explicit shaping, potentially altering the optimal policy. Moreover, such approaches, often relying on single static images, struggle with tasks whose desired behavior involves complex, dynamic motions spanning multiple visually different states. Furthermore, single viewpoints can occlude critical aspects of an agent's behavior. To address these issues, this paper presents Multi-View Video Reward Shaping (MVR), a framework that models the relevance of states regarding the target task using videos captured from multiple viewpoints. MVR leverages video-text similarity from a frozen pre-trained VLM to learn a state relevance function that mitigates the bias towards specific static poses inherent in image-based methods. Additionally, we introduce a state-dependent reward shaping formulation that integrates task-specific rewards and VLM-based guidance, automatically reducing the influence of VLM guidance once the desired motion pattern is achieved. We confirm the efficacy of the proposed framework with extensive experiments on challenging humanoid locomotion tasks from HumanoidBench and manipulation tasks from MetaWorld, verifying the design choices through ablation studies.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 54 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

44 of 54 references.

References · 44

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  4. 2023Mastering Diverse Domains through World Modelscited by 1,193
  5. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  7. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  8. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  9. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  10. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  11. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  12. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
32 more
  1. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 185
  2. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 155
  3. 2020Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscited by 154
  4. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  5. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 147
  6. 2023Vision-Language Models as Success Detectorscited by 138
  7. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
  8. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 128
  9. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
  10. 2020Learning to Utilize Shaping Rewards: A New Approach of Reward Shapingcited by 112
  11. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 112
  12. 2024Vision Language Models are In-Context Value Learnerscited by 84
  13. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 76
  14. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 70
  15. 2023Vision-Language Models as a Source of Rewardscited by 65
  16. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 60
  17. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusioncited by 56
  18. 2024A Survey on Robotics with Foundation Models: toward Embodied AIcited by 53
  19. 2025ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrationscited by 52
  20. 2020Smooth Exploration for Robotic Reinforcement Learningcited by 45
  21. 2024Vision-Language Models Provide Promptable Representations for Reinforcement Learningcited by 41
  22. 2024AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agentscited by 32
  23. 2024Rank2Reward: Learning Shaped Reward Functions from Passive Videocited by 30
  24. 2023Pretrained Language Models as Visual Planners for Human Assistancecited by 28
  25. 2024FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learningcited by 26
  26. 2023Can Pre-Trained Text-to-Image Models Generate Visual Goals for Reinforcement Learning?cited by 20
  27. 2024PROGRESSOR: A Perceptually Guided Reward Estimator with Self-Supervised Online Refinementcited by 11
  28. 2023Guide Your Agent with Adaptive Multimodal Rewardscited by 10
  29. 2025GenFlowRL: Shaping Rewards with Generative Object-Centric Flow in Visual Reinforcement Learningcited by 8
  30. 2024The Dark Side of Rich Rewards: Understanding and Mitigating Noise in VLM Rewardscited by 7
  31. 2025Subtask-Aware Visual Reward Learning from Segmented Demonstrationscited by 7
  32. 2024Learning Prehensile Dexterity by Imitating and Emulating State-only Observationscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Vision-Based Generic Potential Function for Policy Alignment in Multi-Agent Reinforcement Learning
  2. 2026Automating Potential-based Reward Shaping with Vision Language Model Guidance
  3. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model
  4. 2026VLM-AR3L: Vision-Language Models for Absolute and Relative Rewards in Reinforcement Learning
  5. 2025Reinforcement Learning for Large Model: A Survey
  6. 2026CoRe: Combined Rewards with Vision-Language Model Feedback for Preference-Aligned Reinforcement Learning
  7. 2025Reinforcement Learning Tuning for VideoLLMs: Reward Design and Data Efficiency
  8. 2024Code as Reward: Empowering Reinforcement Learning with VLMs