mlsanityData to 14 Oct 2026

Imitation by Predicting Observations

arXiv 2107.03851 ↗First posted 8 Jul 2021cs.LG cs.AIcited by 13

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning enables agents to reuse and adapt the hard-won expertise of others, offering a solution to several key challenges in learning behavior. Although it is easy to observe behavior in the real-world, the underlying actions may not be accessible. We present a new method for imitation solely from observations that achieves comparable performance to experts on challenging continuous control tasks while also exhibiting robustness in the presence of observations unrelated to the task. Our method, which we call FORM (for "Future Observation Reward Model") is derived from an inverse RL objective and imitates using a model of expert behavior learned by generative modelling of the expert's observations, without needing ground truth actions. We show that FORM performs comparably to a strong baseline IRL method (GAIL) on the DeepMind Control Suite benchmark, while outperforming GAIL in the presence of task-irrelevant features.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 47 of its 68 references that are in the map. First posted on arXiv 8 Jul 2021.

Builds on

47 of 68 references.

Closest ancestors · 17

Same direction, newest first

  1. 2021Off-Policy Imitation Learning from Observations25 Feb
  2. 2020Non-Adversarial Imitation Learning and its Connections to Adversarial Methods8 Aug
  3. 2020Intrinsic Reward Driven Imitation Learning via Generative Model26 Jun
  4. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matching25 Jun
  5. 2020Energy-Based Imitation Learning20 Apr
  6. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  7. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  8. 2019Task-Relevant Adversarial Imitation Learning2 Oct
  9. 2019Recent Advances in Imitation Learning from Observation30 May
  10. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  11. 2018An Algorithmic Perspective on Imitation Learning16 Nov
  12. 2018Deep Imitative Models for Flexible Inference, Planning, and Control15 Oct
5 more
  1. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct
  2. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  3. 2018Generative Adversarial Imitation from Observation17 Jul
  4. 2018Imitating Latent Policies from Observation21 May
  5. 2018Behavioral Cloning from Observation4 May

Other references · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2016Layer Normalizationcited by 5,669
  4. 2017Improved Training of Wasserstein GANscited by 2,235
  5. 2016OpenAI Gymcited by 1,736
  6. 2016Generative Adversarial Imitation Learningcited by 1,597
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  8. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  9. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  10. 2018DeepMind Control Suitecited by 956
  11. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 803
  12. 2017Wasserstein GANcited by 639
18 more
  1. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  2. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  3. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  4. 2018Maximum a Posteriori Policy Optimisationcited by 328
  5. 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 264
  6. 2018Zero-Shot Visual Imitationcited by 117
  7. 2017The Numerics of GANscited by 105
  8. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 100
  9. 2018Neural probabilistic motor primitives for humanoid controlcited by 97
  10. 2017Robust Imitation of Diverse Behaviorscited by 86
  11. 2017Learning human behaviors from motion capture by adversarial imitationcited by 77
  12. 2020Imitating Interactive Intelligencecited by 55
  13. 2019Making Efficient Use of Demonstrations to Solve Hard Exploration Problemscited by 45
  14. 2019Learning Predictive Models From Observation and Interactioncited by 41
  15. 2016Learning to Act by Predicting the Futurecited by 36
  16. 2016Model-based Adversarial Imitation Learningcited by 29
  17. 2018Learning what you can do before doing anythingcited by 26
  18. 2020Estimating Q(s,s') with Deep Deterministic Dynamics Gradientscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 1; 2023: 6; 2024: 0; 2025: 3; 2026: 1.

13 papers.

Sort:

By area:

  1. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 148
  2. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 78
  3. 2023Imitation Learning from Observation with Automatic Discount Schedulingcited by 10
  4. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrationscited by 7
  5. 2023Accelerating exploration and representation learning with offline pre-trainingcited by 7
  6. 2023Imitation Learning from Observation through Optimal Transportcited by 7
  7. 2022Imitation Learning by State-Only Distribution Matchingcited by 6
  8. 2025TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal Distancecited by 6
  9. 2021Off-policy Imitation Learning from Visual Inputscited by 2
  10. 2021How Can Creativity Occur in Multi-Agent Systems?cited by 0
  11. 2025On Generalization and Distributional Update for Mimicking Observations with Adequate Explorationcited by 0
  12. 2025Learning from Observation: A Survey of Recent Advancescited by 0
1 more
  1. 2026Reinforcement Learning from Cross-domain Videos with Video Prediction Modelcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  2. 2024Online Adaptation for Enhancing Imitation Learning Policies
  3. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets
  4. 2022Diverse Imitation Learning via Self-Organizing Generative Models
  5. 2025A Model-Based Approach to Imitation Learning through Multi-Step Predictions
  6. 2022Imitation Learning by Estimating Expertise of Demonstrators
  7. 2021Imitation Learning by Reinforcement Learning
  8. 2021Hyperparameter Selection for Imitation Learning