# RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration

arXiv: https://arxiv.org/abs/1906.07372 · HTML: https://mlsanity.com/paper/1906.07372/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 18 Jun 2019.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Built on by (most cited first, 19 of 19)

- [State-Only Imitation Learning for Dexterous Manipulation](https://mlsanity.com/paper/2004.04650/) (2020)
- [Vision-based Manipulation from Single Human Video with Open-World Object Graphs](https://mlsanity.com/paper/2405.20321/) (2024)
- [VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation](https://mlsanity.com/paper/2105.09371/) (2021)
- [LS-IQ: Implicit Reward Regularization for Inverse Reinforcement Learning](https://mlsanity.com/paper/2303.00599/) (2023)
- [A Survey of Imitation Learning Methods, Environments and Metrics](https://mlsanity.com/paper/2404.19456/) (2024)
- [Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks](https://mlsanity.com/paper/2107.05825/) (2021)
- [PRIME: Scaffolding Manipulation Tasks with Behavior Primitives for Data-Efficient Imitation Learning](https://mlsanity.com/paper/2403.00929/) (2024)
- [Mimicking Better by Matching the Approximate Action Distribution](https://mlsanity.com/paper/2306.09805/) (2023)
- [Adapting a World Model for Trajectory Following in a 3D Game](https://mlsanity.com/paper/2504.12299/) (2025)
- [DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation](https://mlsanity.com/paper/2104.00163/) (2021)
- [Explorative Imitation Learning: A Path Signature Approach for Continuous Environments](https://mlsanity.com/paper/2407.04856/) (2024)
- [Concurrent Training Improves the Performance of Behavioral Cloning from Observation](https://mlsanity.com/paper/2008.01205/) (2020)
- [D-Shape: Demonstration-Shaped Reinforcement Learning via Goal Conditioning](https://mlsanity.com/paper/2210.14428/) (2022)
- [Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learning](https://mlsanity.com/paper/2412.00086/) (2024)
- [Shaped Policy Search for Evolutionary Strategies using Waypoints](https://mlsanity.com/paper/2105.14639/) (2021)
- [Accelerate Reinforcement Learning with PID Controllers in the Pendulum Simulations](https://mlsanity.com/paper/2210.00770/) (2022)
- [A Q-learning approach to the continuous control problem of robot inverted pendulum balancing](https://mlsanity.com/paper/2312.02649/) (2023)
- [Learning from Observation: A Survey of Recent Advances](https://mlsanity.com/paper/2509.19379/) (2025)
- [Act First, Reason Later: Accelerating On-Policy Distillation for Multi-Turn Agents via Reference-Conditioned Inverse Dynamics](https://mlsanity.com/paper/2609.36608/) (2026)

## Abstract (arXiv)

Augmenting reinforcement learning with imitation learning is often hailed as a method by which to improve upon learning from scratch. However, most existing methods for integrating these two techniques are subject to several strong assumptions---chief among them that information about demonstrator actions is available. In this paper, we investigate the extent to which this assumption is necessary by introducing and evaluating reinforced inverse dynamics modeling (RIDM), a novel paradigm for combining imitation from observation (IfO) and reinforcement learning with no dependence on demonstrator action information. Moreover, RIDM requires only a single demonstration trajectory and is able to operate directly on raw (unaugmented) state features. We find experimentally that RIDM performs favorably compared to a baseline approach for several tasks in simulation as well as for tasks on a real UR5 robot arm. Experiment videos can be found at https://sites.google.com/view/ridm-reinforced-inverse-dynami.