mlsanityData to 14 Oct 2026

Adversarial Imitation Learning from Visual Observations using Latent Information

arXiv 2309.17371 ↗First posted 29 Sep 2023cs.LG cs.SY eess.SYcited by 11

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We focus on the problem of imitation learning from visual observations, where the learning agent has access to videos of experts as its sole learning source. The challenges of this framework include the absence of expert actions and the partial observability of the environment, as the ground-truth states can only be inferred from pixels. To tackle this problem, we first conduct a theoretical analysis of imitation learning in partially observable environments. We establish upper bounds on the suboptimality of the learning agent with respect to the divergence between the expert and the agent latent state-transition distributions. Motivated by this analysis, we introduce an algorithm called Latent Adversarial Imitation from Observations, which combines off-policy adversarial imitation techniques with a learned latent representation of the agent's state from sequences of observations. In experiments on high-dimensional continuous robotic tasks, we show that our model-free approach in latent space matches state-of-the-art performance. Additionally, we show how our method can be used to improve the efficiency of reinforcement learning from pixels by leveraging expert videos. To ensure reproducibility, we provide free access to our code.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 51 of its 66 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

51 of 66 references.

Closest ancestors · 20

Same direction, newest first

  1. 2023Visual Imitation Learning with Patch Rewards2 Feb
  2. 2022Imitation Learning from Observations under Transition Model Disparity25 Apr
  3. 2022Adversarial Imitation Learning from Video using a State Observer1 Feb
  4. 2021Visual Adversarial Imitation Learning using Variational Models16 Jul
  5. 2021Imitation Learning from MPC for Quadrupedal Multi-Gait Control26 Mar
  6. 2021Domain-Robust Visual Imitation Learning with Mutual Information Constraints8 Mar
  7. 2021Off-Policy Imitation Learning from Observations25 Feb
  8. 2021MobILE: Model-Based Imitation Learning From Observation Alone22 Feb
  9. 2020On the Guaranteed Almost Equivalence between Imitation Learning from Observation and Demonstration16 Oct
  10. 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning28 Jun
  11. 2020Primal Wasserstein Imitation Learning8 Jun
  12. 2020Domain-Adversarial and Conditional State Space Model for Imitation Learning31 Jan
8 more
  1. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  2. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  3. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement10 Oct
  4. 2019Learning Belief Representations for Imitation Learning in POMDPs22 Jun
  5. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  6. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets6 Sep
  7. 2018Generative Adversarial Imitation from Observation17 Jul
  8. 2018Behavioral Cloning from Observation4 May

Other references · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2022Generative Adversarial Networkscited by 5,627
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  5. 2015Continuous control with deep reinforcement learningcited by 3,407
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  7. 2017Improved Training of Wasserstein GANscited by 2,235
  8. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  9. 2016Generative Adversarial Imitation Learningcited by 1,597
  10. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  11. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  12. 2020Mastering Atari with Discrete World Modelscited by 1,007
19 more
  1. 2018DeepMind Control Suitecited by 956
  2. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  3. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  4. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  5. 2020Reinforcement Learning with Augmented Datacited by 468
  6. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  7. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  8. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  9. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  10. 2022Model-Based Imitation Learning for Urban Drivingcited by 150
  11. 2022VideoDex: Learning Dexterity from Internet Videoscited by 106
  12. 2017Third-Person Imitation Learningcited by 103
  13. 2021Generalized Proximal Policy Optimization with Sample Reusecited by 40
  14. 2018Learning from Demonstration in the Wildcited by 25
  15. 2021Learning by Watchingcited by 22
  16. 2022SelfD: Self-Learning Large-Scale Driving Policies From the Webcited by 18
  17. 2022Combining imitation and deep reinforcement learning to accomplish human-level performance on a virtual foraging taskcited by 4
  18. 2022Opportunities and Challenges from Using Animal Videos in Reinforcement Learning for Navigationcited by 4
  19. 2022Generalized Policy Improvement Algorithms with Theoretically Supported Sample Reusecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 4.

11 papers.

  1. 2024Visually Robust Adversarial Imitation Learning from Videos with Contrastive Learningcited by 9
  2. 2025Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representationscited by 5
  3. 2024Sample-efficient Unsupervised Policy Cloning from Ensemble Self-supervised Labeled Videoscited by 4
  4. 2024Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guaranteescited by 2
  5. 2026Dual-Granularity Contrastive Reward via Generated Episodic Guidance for Efficient Embodied RLcited by 2
  6. 2025Beyond Domain Randomization: Event-Inspired Perception for Visually Robust Adversarial Imitation from Videoscited by 1
  7. 2025Robust Point Cloud Reinforcement Learning via PCA-Based Canonicalizationcited by 1
  8. 2024A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observationscited by 0
  9. 2026Latent Wasserstein Adversarial Imitation Learningcited by 0
  10. 2026Optimizing Neurorobot Policy under Limited Demonstration Data through Preference Regretcited by 0
  11. 2026Harnessing VLM in Embodied RL: From Reward Regressor to Success Detector via Exploration-Augmented Key-Observation Gatingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Latent Policies for Adversarial Imitation Learning
  2. 2022Imitation Learning by State-Only Distribution Matching
  3. 2019Sample-efficient Adversarial Imitation Learning from Observation
  4. 2022Robust Learning from Observation with Model Misspecification
  5. 2022Resolving Copycat Problems in Visual Imitation Learning via Residual Action Prediction
  6. 2020ADAIL: Adaptive Adversarial Imitation Learning
  7. 2019Provably Efficient Imitation Learning from Observation Alone
  8. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets