mlsanityData to 7 Oct 2026

Model-Based Inverse Reinforcement Learning from Visual Demonstrations

arXiv 2010.09034 ↗First posted 18 Oct 2020cs.RO cs.LGcited by 52

On the map: Offline RL and imitation learning from data → Real-world robot offline RL

Abstract (arXiv)

Scaling model-based inverse reinforcement learning (IRL) to real robotic manipulation tasks with unknown dynamics remains an open problem. The key challenges lie in learning good dynamics models, developing algorithms that scale to high-dimensional state-spaces and being able to learn from both visual and proprioceptive demonstrations. In this work, we present a gradient-based inverse reinforcement learning framework that utilizes a pre-trained visual dynamics model to learn cost functions when given only visual human demonstrations. The learned cost functions are then used to reproduce the demonstrated behavior via visual model predictive control. We evaluate our framework on hardware on two basic object manipulation tasks.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 32 references that are in the map. First posted on arXiv 18 Oct 2020.

Builds on

16 of 32 references.

References · 16

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  3. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  4. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  5. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
  6. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  7. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  8. 2020DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulationcited by 164
  9. 2019kPAM: KeyPoint Affordances for Category-Level Robotic Manipulationcited by 152
  10. 2018Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulationcited by 140
  11. 2019Unsupervised Learning of Object Keypoints for Perception and Controlcited by 109
  12. 2019Unsupervised Learning of Object Structure and Dynamics from Videoscited by 92
4 more
  1. 2019Generalized Inner Loop Meta-Learningcited by 92
  2. 2016Unsupervised Perceptual Rewards for Imitation Learningcited by 64
  3. 2019Meta-Learning via Learned Losscited by 60
  4. 2020Approximate Inverse Reinforcement Learning from Vision-based Imitation Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 8; 2023: 11; 2024: 15; 2025: 8; 2026: 1.

52 papers.

Sort:

By area:

  1. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  2. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  3. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  4. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 214
  5. 2022Masked World Models for Visual Controlcited by 169
  6. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  7. 2022VideoDex: Learning Dexterity from Internet Videoscited by 101
  8. 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 88
  9. 2025Humanoid Policy ~ Human Policycited by 65
  10. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 56
  11. 2023Towards A Unified Agent with Foundation Modelscited by 46
  12. 2021Visual Adversarial Imitation Learning using Variational Modelscited by 42
38 more
  1. 2024Surgical Robot Transformer (SRT): Imitation Learning for Surgical Taskscited by 38
  2. 2024Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applicationscited by 34
  3. 2021VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigationcited by 27
  4. 2023One-Shot Imitation Learning: A Pose Estimation Perspectivecited by 21
  5. 2024AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrationscited by 19
  6. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 18
  7. 2022Cross-Domain Transfer via Semantic Skill Imitationcited by 17
  8. 2024Cross-Embodiment Robot Manipulation Skill Transfer using Latent Space Alignmentcited by 17
  9. 2023Learning Video-Conditioned Policies for Unseen Manipulation Taskscited by 15
  10. 2022Learning Reward Functions for Robotic Manipulation by Observing Humanscited by 14
  11. 2023A Survey of Demonstration Learningcited by 12
  12. 2022Continual Learning from Demonstration of Robotics Skillscited by 11
  13. 2023Human-oriented Representation Learning for Robotic Manipulationcited by 10
  14. 2023A Bayesian Approach to Robust Inverse Reinforcement Learningcited by 9
  15. 2024Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Promptscited by 9
  16. 2024VLMimic: Vision Language Models are Visual Imitation Learner for Fine-grained Actionscited by 8
  17. 2025H2R-Grounder: A Paired-Data-Free Paradigm for Translating Human Interaction Videos into Physically Grounded Robot Videoscited by 8
  18. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrationscited by 7
  19. 2024Reward-free World Models for Online Imitation Learningcited by 7
  20. 2023Giving Robots a Hand: Learning Generalizable Manipulation with Eye-in-Hand Human Video Demonstrationscited by 6
  21. 2023Scalable and Efficient Continual Learning from Demonstration via a Hypernetwork-generated Stable Dynamics Modelcited by 6
  22. 2025Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Modelscited by 6
  23. 2021Vision-based system identification and 3D keypoint discovery using dynamics constraintscited by 5
  24. 2022Learning Object Manipulation Skills from Video via Approximate Differentiable Physicscited by 5
  25. 2024Multi-task real-robot data with gaze attention for dual-arm fine manipulationcited by 5
  26. 2024TieBot: Learning to Knot a Tie from Visual Demonstration through a Real-to-Sim-to-Real Approachcited by 5
  27. 2021Back to Reality for Imitation Learningcited by 3
  28. 2024CANDERE-COACH: Reinforcement Learning from Noisy Feedbackcited by 3
  29. 2024Enhancing Inverse Reinforcement Learning through Encoding Dynamic Information in Reward Shapingcited by 3
  30. 2024Language-Model-Assisted Bi-Level Programming for Reward Learning from Internet Videoscited by 3
  31. 2021A Differentiable Recipe for Learning Visual Non-Prehensile Planar Manipulationcited by 2
  32. 2025Model Predictive Adversarial Imitation Learning for Planning from Observationcited by 2
  33. 2021Imitation Learning via Simultaneous Optimization of Policies and Auxiliary Trajectoriescited by 1
  34. 2022Modeling Human Behavior Part I -- Learning and Belief Approachescited by 1
  35. 2024Visual IRL for Human-Like Robotic Manipulationcited by 1
  36. 2021Learning Time-Invariant Reward Functions through Model-Based Inverse Reinforcement Learningcited by 0
  37. 2024Semi-Supervised One-Shot Imitation Learningcited by 0
  38. 2025Inverse RL Scene Dynamics Learning for Nonlinear Predictive Control in Autonomous Vehiclescited by 0

50 most cited of 52 All 52 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Robotic Arm Manipulation with Inverse Reinforcement Learning & TD-MPC
  2. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation
  3. 2026Online World Modeling Enables Real-World Inverse Reinforcement Learning from Observation
  4. 2020State-Only Imitation Learning for Dexterous Manipulation
  5. 2026Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards
  6. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
  7. 2019RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration
  8. 2021Residual Reinforcement Learning from Demonstrations