mlsanityData to 7 Oct 2026

Seeing Differently, Acting Similarly: Heterogeneously Observable Imitation Learning

arXiv 2106.09256 ↗First posted 17 Jun 2021cs.LG cs.AIcited by 4

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

In many real-world imitation learning tasks, the demonstrator and the learner have to act under different observation spaces. This situation brings significant obstacles to existing imitation learning approaches, since most of them learn policies under homogeneous observation spaces. On the other hand, previous studies under different observation spaces have strong assumptions that these two observation spaces coexist during the entire learning process. However, in reality, the observation coexistence will be limited due to the high cost of acquiring expert observations. In this work, we study this challenging problem with limited observation coexistence under heterogeneous observations: Heterogeneously Observable Imitation Learning (HOIL). We identify two underlying issues in HOIL: the dynamics mismatch and the support mismatch, and further propose the Importance Weighting with REjection (IWRE) algorithm based on importance weighting and learning with rejection to solve HOIL problems. Experimental results show that IWRE can solve various HOIL tasks, including the challenging tasks of transforming the vision-based demonstrations to random access memory (RAM)-based policies in the Atari domain, even with limited visual observations.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 30 of its 63 references that are in the map. First posted on arXiv 17 Jun 2021.

Builds on

30 of 63 references.

Closest ancestors · 7

Same direction, newest first

  1. 2021Cross-domain Imitation from Observations20 May
  2. 2020Error Bounds of Imitating Policies and Environments22 Oct
  3. 2020Active Imitation Learning with Noisy Guidance26 May
  4. 2019Domain Adaptive Imitation Learning30 Sep
  5. 2019Imitation Learning from Pixel-Level Demonstrations by HashReward9 Sep
  6. 2019Learning Belief Representations for Imitation Learning in POMDPs22 Jun
  7. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation16 May

Other references · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  8. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  9. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  10. 2019Learning by Cheatingcited by 361
  11. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
  12. 2019Reinforcement Learning in Healthcare: A Surveycited by 308
11 more
  1. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  2. 2013A Survey on Multi-view Learningcited by 152
  3. 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 143
  4. 2017Third-Person Imitation Learningcited by 103
  5. 2020Rethinking Importance Weighting for Deep Learning under Distribution Shiftcited by 92
  6. 2020Robust Asymmetric Learning in POMDPscited by 23
  7. 2018Importance Weighted Transfer of Samples in Reinforcement Learningcited by 16
  8. 2017Learning with Feature Evolvable Streamscited by 15
  9. 2014Scalable Planning and Learning for Multiagent POMDPs: Extended Versioncited by 14
  10. 2016One-Pass Learning with Incremental and Decremental Featurescited by 13
  11. 2016Learning from the memory of Atari 2600cited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2.

4 papers.

  1. 2024RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanationcited by 7
  2. 2023Unlabeled Imperfect Demonstrations in Adversarial Imitation Learningcited by 3
  3. 2023Imitation Learning from Purified Demonstrationscited by 3
  4. 2024A Bayesian Solution To The Imitation Gapcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2023Adversarial Imitation Learning from Visual Observations using Latent Information
  2. 2019Imitation Learning from Video by Leveraging Proprioception
  3. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement
  4. 2025Learning from Observation: A Survey of Recent Advances
  5. 2025Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement
  6. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets
  7. 2023A Simple Solution for Offline Imitation from Observations and Examples with Possibly Incomplete Trajectories
  8. 2024Overcoming Knowledge Barriers: Online Imitation Learning from Visual Observation with Pretrained World Models