mlsanityData to 7 Oct 2026

Graph Inverse Reinforcement Learning from Diverse Videos

arXiv 2207.14299 ↗First posted 28 Jul 2022cs.LG cs.CV cs.ROcited by 45

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Research on Inverse Reinforcement Learning (IRL) from third-person videos has shown encouraging results on removing the need for manual reward design for robotic tasks. However, most prior works are still limited by training from a relatively restricted domain of videos. In this paper, we argue that the true potential of third-person IRL lies in increasing the diversity of videos for better scaling. To learn a reward function from diverse videos, we propose to perform graph abstraction on the videos followed by temporal matching in the graph space to measure the task progress. Our insight is that a task can be described by entity interactions that form a graph, and this graph abstraction can help remove irrelevant information such as textures, resulting in more robust reward functions. We evaluate our approach, GraphIRL, on cross-embodiment learning in X-MAGICAL and learning from human demonstrations for real-robot manipulation. We show significant improvements in robustness to diverse video demonstrations over previous approaches, and even achieve better results than manual reward design on a real robot pushing task. Videos are available at https://sateeshkumar21.github.io/GraphIRL .

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 51 of its 63 references that are in the map. First posted on arXiv 28 Jul 2022.

Builds on

51 of 63 references.

Closest ancestors · 7

Same direction, newest first

  1. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation24 Mar
  2. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videos12 Aug
  3. 2021XIRL: Cross-embodiment Inverse Reinforcement Learning7 Jun
  4. 2020Visual Imitation Made Easy11 Aug
  5. 2020State-Only Imitation Learning for Dexterous Manipulation7 Apr
  6. 2019Self-Supervised Correspondence in Visuomotor Policy Learning16 Sep
  7. 2018Zero-Shot Visual Imitation23 Apr

Other references · 44

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2018Learning Dexterous In-Hand Manipulationcited by 643
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  7. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 503
  8. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  9. 2018Behavioral Cloning from Observationcited by 482
  10. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  11. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  12. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
32 more
  1. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  2. 2018Graph networks as learnable physics engines for inference and controlcited by 242
  3. 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
  4. 2018Generative Adversarial Imitation from Observationcited by 183
  5. 2017Scene Graph Generation from Objects, Phrases and Region Captionscited by 154
  6. 2019Temporal Cycle-Consistency Learningcited by 151
  7. 2018Visual Semantic Navigation using Scene Priorscited by 127
  8. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  9. 2019Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networkscited by 112
  10. 2019Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrationscited by 104
  11. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  12. 2018Playing hard exploration games by watching YouTubecited by 98
  13. 2017Visual Interaction Networkscited by 94
  14. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  15. 2017Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learningcited by 84
  16. 2020Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphscited by 83
  17. 2020Reinforcement Learning with Videos: Combining Offline Observations with Interactioncited by 80
  18. 2020Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learningcited by 69
  19. 2019Learning Compositional Koopman Operators for Model-Based Controlcited by 67
  20. 2021Cross-Domain Imitation Learning via Optimal Transportcited by 49
  21. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  22. 2021Learning by Aligning Videos in Timecited by 47
  23. 2020Learning Long-term Visual Dynamics with Region Proposal Interaction Networkscited by 44
  24. 2019Graph-Structured Visual Imitationcited by 43
  25. 2019Compositional Video Predictioncited by 43
  26. 2021Representation Learning via Global Temporal Alignment and Cycle-Consistencycited by 43
  27. 2020S3K: Self-Supervised Semantic Keypoints for Robotic Manipulation via Multi-View Consistencycited by 28
  28. 2018Grasp2Vec: Learning Object Representations from Self-Supervised Graspingcited by 27
  29. 2019Object-centric Forward Modeling for Model Predictive Controlcited by 19
  30. 2021Learning to Align Sequential Actions in the Wildcited by 19
  31. 2021Attribute-Based Robotic Grasping with One-Grasp Adaptationcited by 7
  32. 2019Visual Geometric Skill Inference by Watching Human Demonstrationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 9; 2024: 17; 2025: 11; 2026: 6.

45 papers.

Sort:

By area:

  1. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 214
  2. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 164
  3. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  4. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  5. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 59
  6. 2024Koala: Key frame-conditioned long video-LLMcited by 57
  7. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 46
  8. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  9. 2024Hand-Object Interaction Pretraining from Videoscited by 38
  10. 2025ZeroMimic: Distilling Robotic Manipulation Skills from Web Videoscited by 31
  11. 2025Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstrationcited by 27
  12. 2024CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulationcited by 25
33 more
  1. 2024Bridging the Human to Robot Dexterity Gap through Object-Oriented Rewardscited by 25
  2. 2025X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Realcited by 25
  3. 2023For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equalcited by 22
  4. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 18
  5. 2024Composing Pre-Trained Object-Centric Representations for Robotics From "What" and "Where" Foundation Modelscited by 16
  6. 2023What Matters to You? Towards Visual Representation Alignment for Robot Learningcited by 14
  7. 2022A System for Morphology-Task Generalization via Unified Representation and Behavior Distillationcited by 12
  8. 2024One-Shot Imitation under Mismatched Executioncited by 11
  9. 2024Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulationcited by 10
  10. 2025Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learningcited by 10
  11. 2023Multi-intention Inverse Q-learning for Interpretable Behavior Representationcited by 9
  12. 2024VLMimic: Vision Language Models are Visual Imitation Learner for Fine-grained Actionscited by 8
  13. 2025Subtask-Aware Visual Reward Learning from Segmented Demonstrationscited by 7
  14. 2023Giving Robots a Hand: Learning Generalizable Manipulation with Eye-in-Hand Human Video Demonstrationscited by 6
  15. 2024Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Modelscited by 6
  16. 2025Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Modelscited by 6
  17. 2025X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrationscited by 6
  18. 2026Recurrent Reasoning with Vision-Language Models for Estimating Long-Horizon Embodied Task Progresscited by 6
  19. 2024Offline Goal-Conditioned Reinforcement Learning for Safety-Critical Tasks with Recovery Policycited by 5
  20. 2024TieBot: Learning to Knot a Tie from Visual Demonstration through a Real-to-Sim-to-Real Approachcited by 5
  21. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspectivecited by 4
  22. 2024Learning to Recover from Plan Execution Errors during Robot Manipulation: A Neuro-symbolic Approachcited by 3
  23. 2024Language-Model-Assisted Bi-Level Programming for Reward Learning from Internet Videoscited by 3
  24. 2026RARM: Confidence-Gated Progress Reward Modeling for RL in Manipulationcited by 3
  25. 2023Boosting Reinforcement Learning and Planning with Demonstrations: A Surveycited by 1
  26. 2024MeMo: Meaningful, Modular Controllers via Noise Injectioncited by 1
  27. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advancescited by 0
  28. 2025Going Beyond Expert Performance via Deep Implicit Imitation Reinforcement Learningcited by 0
  29. 2025ManiLong-Shot: Interaction-Aware One-Shot Imitation Learning for Long-Horizon Manipulationcited by 0
  30. 2026SLOPE: Optimistic Potential Landscape Shaping for Model-based Reinforcement Learningcited by 0
  31. 2026LAMP: Lift Image-Editing as General 3D Priors for Open-world Manipulationcited by 0
  32. 2026ConTraIRL: Factorized Contrastive Abstractions for Transferable IRLcited by 0
  33. 2026SIR: Structured Image Representations for Explainable Robot Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Representation Alignment from Human Feedback for Cross-Embodiment Reward Learning from Mixed-Quality Demonstrations
  2. 2024Offline Imitation Learning Through Graph Search and Retrieval
  3. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations
  4. 2026Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learning
  5. 2025NIL: No-data Imitation Learning by Leveraging Pre-trained Video Diffusion Models
  6. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables
  7. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrations
  8. 2026Human-to-Robot Interaction: Learning from Video Demonstration for Robot Imitation