mlsanityData to 7 Oct 2026

RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration

arXiv 1906.07372 ↗First posted 18 Jun 2019cs.LG cs.RO stat.MLcited by 19

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Augmenting reinforcement learning with imitation learning is often hailed as a method by which to improve upon learning from scratch. However, most existing methods for integrating these two techniques are subject to several strong assumptions---chief among them that information about demonstrator actions is available. In this paper, we investigate the extent to which this assumption is necessary by introducing and evaluating reinforced inverse dynamics modeling (RIDM), a novel paradigm for combining imitation from observation (IfO) and reinforcement learning with no dependence on demonstrator action information. Moreover, RIDM requires only a single demonstration trajectory and is able to operate directly on raw (unaugmented) state features. We find experimentally that RIDM performs favorably compared to a baseline approach for several tasks in simulation as well as for tasks on a real UR5 robot arm. Experiment videos can be found at https://sites.google.com/view/ridm-reinforced-inverse-dynami.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 18 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 4; 2022: 2; 2023: 3; 2024: 5; 2025: 2; 2026: 1.

19 papers.

Sort:

By area:

  1. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 90
  2. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 46
  3. 2023LS-IQ: Implicit Reward Regularization for Inverse Reinforcement Learningcited by 28
  4. 2021VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigationcited by 27
  5. 2024A Survey of Imitation Learning Methods, Environments and Metricscited by 16
  6. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  7. 2024PRIME: Scaffolding Manipulation Tasks with Behavior Primitives for Data-Efficient Imitation Learningcited by 8
  8. 2023Mimicking Better by Matching the Approximate Action Distributioncited by 5
  9. 2025Adapting a World Model for Trajectory Following in a 3D Gamecited by 5
  10. 2021DEALIO: Data-Efficient Adversarial Learning for Imitation from Observationcited by 4
  11. 2024Explorative Imitation Learning: A Path Signature Approach for Continuous Environmentscited by 3
  12. 2020Concurrent Training Improves the Performance of Behavioral Cloning from Observationcited by 2
7 more
  1. 2022D-Shape: Demonstration-Shaped Reinforcement Learning via Goal Conditioningcited by 2
  2. 2024Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learningcited by 2
  3. 2021Shaped Policy Search for Evolutionary Strategies using Waypointscited by 1
  4. 2022Accelerate Reinforcement Learning with PID Controllers in the Pendulum Simulationscited by 0
  5. 2023A Q-learning approach to the continuous control problem of robot inverted pendulum balancingcited by 0
  6. 2025Learning from Observation: A Survey of Recent Advancescited by 0
  7. 2026Act First, Reason Later: Accelerating On-Policy Distillation for Multi-Turn Agents via Reference-Conditioned Inverse Dynamicscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrations
  2. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets
  3. 2026Online World Modeling Enables Real-World Inverse Reinforcement Learning from Observation
  4. 2022Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations
  5. 2026On the Sample Efficiency of Inverse Dynamics Models for Semi-Supervised Imitation Learning
  6. 2024Inverse Reinforcement Learning by Estimating Expertise of Demonstrators
  7. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement
  8. 2024Hybrid Inverse Reinforcement Learning