mlsanityData to 14 Oct 2026

Learning from Observation: A Survey of Recent Advances

arXiv 2509.19379 ↗First posted 20 Sep 2025cs.LG cs.AI cs.ROcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation Learning (IL) algorithms offer an efficient way to train an agent by mimicking an expert's behavior without requiring a reward function. IL algorithms often necessitate access to state and action information from expert demonstrations. Although expert actions can provide detailed guidance, requiring such action information may prove impractical for real-world applications where expert actions are difficult to obtain. To address this limitation, the concept of learning from observation (LfO) or state-only imitation learning (SOIL) has recently gained attention, wherein the imitator only has access to expert state visitation information. In this paper, we present a framework for LfO and use it to survey and classify existing LfO methods in terms of their trajectory construction, assumptions and algorithm's design choices. This survey also draws connections between several related fields like offline RL, model-based RL and hierarchical RL. Finally, we use our framework to identify open problems and suggest future research directions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 65 of its 98 references that are in the map. First posted on arXiv 20 Sep 2025.

Builds on

65 of 98 references.

Closest ancestors · 35

Same direction, newest first

  1. 2024Diffusion Imitation from Observation7 Oct
  2. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets13 Jun
  3. 2023Domain Adaptive Imitation Learning with Visual Observation1 Dec
  4. 2023Imitation Learning from Observation with Automatic Discount Scheduling11 Oct
  5. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges5 Sep
  6. 2023MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observations30 Mar
  7. 2023Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching5 Mar
  8. 2023Visual Imitation Learning with Patch Rewards2 Feb
  9. 2022Out-of-Dynamics Imitation Learning from Multimodal Demonstrations13 Nov
  10. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddings24 Sep
  11. 2022Imitation Learning from Observations under Transition Model Disparity25 Apr
  12. 2022Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimization4 Mar
23 more
  1. 2022Robust Learning from Observation with Model Misspecification12 Feb
  2. 2022A Ranking Game for Imitation Learning7 Feb
  3. 2022Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matching4 Feb
  4. 2021Off-policy Imitation Learning from Visual Inputs8 Nov
  5. 2021Learning Feasibility to Imitate Demonstrators with Different Dynamics28 Oct
  6. 2021Imitation by Predicting Observations8 Jul
  7. 2021Imitation Learning: Progress, Taxonomies and Challenges23 Jun
  8. 2021Cross-domain Imitation from Observations20 May
  9. 2021Learning from Imperfect Demonstrations from Agents with Varying Dynamics10 Mar
  10. 2021Domain-Robust Visual Imitation Learning with Mutual Information Constraints8 Mar
  11. 2021Off-Policy Imitation Learning from Observations25 Feb
  12. 2021MobILE: Model-Based Imitation Learning From Observation Alone22 Feb
  13. 2020State-only Imitation with Transition Dynamics Mismatch27 Feb
  14. 2019To Follow or not to Follow: Selective Imitation Learning from Observations16 Dec
  15. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  16. 2019State Alignment-based Imitation Learning21 Nov
  17. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement10 Oct
  18. 2019Recent Advances in Imitation Learning from Observation30 May
  19. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation16 May
  20. 2018An Algorithmic Perspective on Imitation Learning16 Nov
  21. 2018Generative Adversarial Imitation from Observation17 Jul
  22. 2018Imitating Latent Policies from Observation21 May
  23. 2018Behavioral Cloning from Observation4 May

Other references · 30

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  3. 2016Generative Adversarial Imitation Learningcited by 1,597
  4. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2018DeepMind Control Suitecited by 956
  7. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  8. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  9. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  10. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 260
  11. 2021Mastering Atari Games with Limited Datacited by 237
  12. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 223
18 more
  1. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  2. 2021OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimationcited by 121
  3. 2019When to use parametric models in reinforcement learning?cited by 118
  4. 2018Zero-Shot Visual Imitationcited by 117
  5. 2017Third-Person Imitation Learningcited by 103
  6. 2018Playing hard exploration games by watching YouTubecited by 99
  7. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 91
  8. 2020ViNG: Learning Open-World Navigation with Visual Goalscited by 80
  9. 2018Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitationcited by 76
  10. 2019Third-Person Visual Imitation Learning via Decoupled Hierarchical Controllercited by 73
  11. 2018ViZDoom Competitions: Playing Doom from Pixelscited by 51
  12. 2020Model-based Reinforcement Learning: A Surveycited by 39
  13. 2022LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimationcited by 23
  14. 2019RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstrationcited by 19
  15. 2018One-Shot Learning of Multi-Step Tasks from Observation via Activity Localization in Auxiliary Videocited by 16
  16. 2020A Conceptual Framework for Externally-influenced Agents: An Assisted Reinforcement Learning Reviewcited by 15
  17. 2023Offline Imitation from Observation via Primal Wasserstein State Occupancy Matchingcited by 5
  18. 2023Imitation from Observation With Bootstrapped Contrastive Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Imitation Learning from Observation through Optimal Transport
  2. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  3. 2024Explorative Imitation Learning: A Path Signature Approach for Continuous Environments
  4. 2025Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement
  5. 2020On the Guaranteed Almost Equivalence between Imitation Learning from Observation and Demonstration
  6. 2026Towards Generalisable Imitation Learning Through Conditioned Transition Estimation and Online Behaviour Alignment
  7. 2019Imitation Learning from Video by Leveraging Proprioception
  8. 2023A Simple Solution for Offline Imitation from Observations and Examples with Possibly Incomplete Trajectories