mlsanityData to 7 Oct 2026

Reinforcement Learning with Videos: Combining Offline Observations with Interaction

arXiv 2011.06507 ↗First posted 12 Nov 2020cs.LG cs.AI cs.CVcited by 80

Not on the map. Why?

Cited by 22 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reinforcement learning is a powerful framework for robots to acquire skills from experience, but often requires a substantial amount of online data collection. As a result, it is difficult to collect sufficiently diverse experiences that are needed for robots to generalize broadly. Videos of humans, on the other hand, are a readily available source of broad and interesting experiences. In this paper, we consider the question: can we perform reinforcement learning directly on experience collected by humans? This problem is particularly difficult, as such videos are not annotated with actions and exhibit substantial visual domain shift relative to the robot's embodiment. To address these challenges, we propose a framework for reinforcement learning with videos (RLV). RLV learns a policy and value function using experience collected by humans in combination with data collected by robots. In our experiments, we find that RLV is able to leverage such videos to learn challenging vision-based skills with less than half as many samples as RL methods that learn from scratch.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 15; 2023: 18; 2024: 21; 2025: 10; 2026: 4.

80 papers.

Sort:

By area:

  1. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  2. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  3. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 214
  4. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
  5. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  6. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 159
  7. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  8. 2024Flow as the Cross-Domain Manipulation Interfacecited by 126
  9. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  10. 2023Learning to Act without Actionscited by 116
  11. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  12. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 101
38 more
  1. 2022VideoDex: Learning Dexterity from Internet Videoscited by 101
  2. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  3. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 79
  4. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 76
  5. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  6. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 64
  7. 2021Human Hands as Probes for Interactive Object Understandingcited by 51
  8. 2023Zero-Shot Robot Manipulation from Passive Human Videoscited by 49
  9. 2024Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformerscited by 47
  10. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 46
  11. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 45
  12. 2021MobILE: Model-Based Imitation Learning From Observation Alonecited by 44
  13. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  14. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 40
  15. 2022Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrationscited by 34
  16. 2024ViViDex: Learning Vision-based Dexterous Manipulation from Human Videoscited by 33
  17. 2021RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learningcited by 31
  18. 2023Robotic Offline RL from Internet Videos via Value-Function Pre-Trainingcited by 31
  19. 2023Polybot: Training One Policy Across Robots While Embracing Variabilitycited by 28
  20. 2022Semi-Supervised Offline Reinforcement Learning with Action-Free Trajectoriescited by 27
  21. 2024CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulationcited by 25
  22. 2024Grounding Video Models to Actions through Goal Conditioned Explorationcited by 25
  23. 2024Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representationscited by 23
  24. 2024Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulationcited by 22
  25. 2023Look Ma, No Hands! Agent-Environment Factorization of Egocentric Videoscited by 21
  26. 2023MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observationscited by 18
  27. 2025AMPLIFY: Actionless Motion Priors for Robot Learning from Videoscited by 18
  28. 2021Shaping embodied agent behavior with activity-context priors from egocentric videocited by 17
  29. 2022Cross-Domain Transfer via Semantic Skill Imitationcited by 17
  30. 2024Video-Language Critic: Transferable Reward Functions for Language-Conditioned Roboticscited by 17
  31. 2023Learning Video-Conditioned Policies for Unseen Manipulation Taskscited by 15
  32. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddingscited by 14
  33. 2022Learning Reward Functions for Robotic Manipulation by Observing Humanscited by 14
  34. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  35. 2024Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrationscited by 12
  36. 2024Diffusion Imitation from Observationcited by 12
  37. 2022Adversarial Imitation Learning from Video using a State Observercited by 10
  38. 2024Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulationcited by 10

50 most cited of 80 All 80 (JSON)