mlsanityData to 14 Oct 2026

TeViR: Text-to-Video Reward with Diffusion Models for Efficient Reinforcement Learning

arXiv 2505.19769 ↗First posted 26 May 2025cs.RO cs.AIcited by 8

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Developing scalable and generalizable reward engineering for reinforcement learning (RL) is crucial for creating general-purpose agents, especially in the challenging domain of robotic manipulation. While recent advances in reward engineering with Vision-Language Models (VLMs) have shown promise, their sparse reward nature significantly limits sample efficiency. This paper introduces TeViR, a novel method that leverages a pre-trained text-to-video diffusion model to generate dense rewards by comparing the predicted image sequence with current observations. Experimental results across 11 complex robotic tasks demonstrate that TeViR outperforms traditional methods leveraging sparse rewards and other state-of-the-art (SOTA) methods, achieving better sample efficiency and performance without ground truth environmental rewards. TeViR's ability to efficiently guide agents in complex environments highlights its potential to advance reinforcement learning applications in robotic manipulation.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 46 of its 63 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

46 of 63 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusion21 Dec
  2. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models16 Oct
  3. 2023Boosting Continuous Control with Consistency Policy10 Oct

Other references · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Implicit Modelscited by 9,781
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  4. 2022Video Diffusion Modelscited by 2,305
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  6. 2016Concrete Problems in AI Safetycited by 1,642
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  8. 2018Exploration by Random Network Distillationcited by 993
  9. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  10. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  11. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  12. 2023Learning Interactive Real-World Simulatorscited by 458
31 more
  1. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  2. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  3. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  4. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 327
  5. 2023Language to Rewards for Robotic Skill Synthesiscited by 312
  6. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 298
  7. 2023Reward Design with Language Modelscited by 261
  8. 2022Interactive Language: Talking to Robots in Real Timecited by 218
  9. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 218
  10. 2023Any-point Trajectory Modeling for Policy Learningcited by 215
  11. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 189
  12. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 188
  13. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 187
  14. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 165
  15. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 164
  16. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 156
  17. 2021The Surprising Effectiveness of Representation Learning for Visual Imitationcited by 154
  18. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 154
  19. 2023Vision-Language Models as Success Detectorscited by 142
  20. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 134
  21. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 122
  22. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 119
  23. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 107
  24. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 78
  25. 2016Unsupervised Perceptual Rewards for Imitation Learningcited by 65
  26. 2023Language Instructed Reinforcement Learning for Human-AI Coordinationcited by 63
  27. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 54
  28. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 43
  29. 2020Deep Reinforcement Learning based Automatic Exploration for Navigation in Unknown Environmentcited by 34
  30. 2023Can Pre-Trained Text-to-Image Models Generate Visual Goals for Reinforcement Learning?cited by 20
  31. 2022A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combatcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  2. 2025World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic Manipulationcited by 35
  3. 2026Scaling World Model for Hierarchical Manipulation Policiescited by 16
  4. 2026Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuningcited by 13
  5. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  6. 2025Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representationscited by 5
  7. 2026Dual-Granularity Contrastive Reward via Generated Episodic Guidance for Efficient Embodied RLcited by 2
  8. 2026Keyframe-Guided Structured Rewards for Reinforcement Learning in Long-Horizon Laboratory Roboticscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Goal-Driven Reward by Video Diffusion Models for Reinforcement Learning
  2. 2026TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward
  3. 2024Text-Aware Diffusion for Policy Learning
  4. 2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Models
  5. 2026EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models
  6. 2026From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models
  7. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learning
  8. 2024Video-Language Critic: Transferable Reward Functions for Language-Conditioned Robotics