mlsanityData to 7 Oct 2026

Cross-domain Imitation from Observations

arXiv 2105.10037 ↗First posted 20 May 2021cs.LG cs.AIcited by 42

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning seeks to circumvent the difficulty in designing proper reward functions for training agents by utilizing expert behavior. With environments modeled as Markov Decision Processes (MDP), most of the existing imitation algorithms are contingent on the availability of expert demonstrations in the same MDP as the one in which a new imitation policy is to be learned. In this paper, we study the problem of how to imitate tasks when there exist discrepancies between the expert and agent MDP. These discrepancies across domains could include differing dynamics, viewpoint, or morphology; we present a novel framework to learn correspondences across such domains. Importantly, in contrast to prior works, we use unpaired and unaligned trajectories containing only states in the expert domain, to learn this correspondence. We utilize a cycle-consistency constraint on both the state space and a domain agnostic latent space to do this. In addition, we enforce consistency on the temporal position of states via a normalized position estimator function, to align the trajectories across the two domains. Once this correspondence is found, we can directly transfer the demonstrations on one domain to the other and use it for imitation. Experiments across a wide variety of challenging domains demonstrate the efficacy of our approach.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 25 of its 46 references that are in the map. First posted on arXiv 20 May 2021.

Builds on

25 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2020State-only Imitation with Transition Dynamics Mismatch27 Feb
  2. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement10 Oct
  3. 2018Behavioral Cloning from Observation4 May

Other references · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  3. 2016OpenAI Gymcited by 1,684
  4. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
  7. 2016End to End Learning for Self-Driving Carscited by 1,000
  8. 2016Least Squares Generative Adversarial Networkscited by 1,000
  9. 2016Deep Variational Information Bottleneckcited by 915
  10. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  11. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 597
  12. 2019On Variational Bounds of Mutual Informationcited by 550
10 more
  1. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  2. 2019AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videoscited by 105
  3. 2017Third-Person Imitation Learningcited by 103
  4. 2020RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Realcited by 89
  5. 2017Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learningcited by 84
  6. 2019Third-Person Visual Imitation Learning via Decoupled Hierarchical Controllercited by 73
  7. 2020RetinaGAN: An Object-aware Approach to Sim-to-Real Transfercited by 68
  8. 2018Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translationcited by 50
  9. 2019Learning from Trajectories via Subgoal Discoverycited by 27
  10. 2020Mutual Information Based Knowledge Transfer Under State-Action Dimension Mismatchcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 5; 2023: 9; 2024: 13; 2025: 7; 2026: 5.

42 papers.

Sort:

By area:

  1. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  2. 2021Cross-Domain Imitation Learning via Optimal Transportcited by 49
  3. 2022Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matchingcited by 43
  4. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  5. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
  6. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  7. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  8. 2024A Survey of Imitation Learning Methods, Environments and Metricscited by 16
  9. 2023Effective Restoration of Source Knowledge in Continual Test Time Adaptationcited by 15
  10. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddingscited by 14
  11. 2023Prior-guided Source-free Domain Adaptation for Human Pose Estimationcited by 13
  12. 2023CEIL: Generalized Contextual Imitation Learningcited by 12
30 more
  1. 2023Domain Adaptive Imitation Learning with Visual Observationcited by 11
  2. 2024Visually Robust Adversarial Imitation Learning from Videos with Contrastive Learningcited by 9
  3. 2024xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editingcited by 9
  4. 2025Generalizable Domain Adaptation for Sim-and-Real Policy Co-Trainingcited by 9
  5. 2023Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrationscited by 8
  6. 2025Preference VLM: Leveraging VLMs for Scalable Preference-Based Reinforcement Learningcited by 6
  7. 2022Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemblecited by 5
  8. 2023SUMMIT: Source-Free Adaptation of Uni-Modal Models to Multi-Modal Targetscited by 5
  9. 2021Seeing Differently, Acting Similarly: Heterogeneously Observable Imitation Learningcited by 4
  10. 2021Off-Dynamics Inverse Reinforcement Learning from Hetero-Domaincited by 3
  11. 2023Similarity-based Knowledge Transfer for Cross-Domain Reinforcement Learningcited by 3
  12. 2024Spatially Visual Perception for End-to-End Robotic Learningcited by 2
  13. 2024SENSOR: Imitate Third-Person Expert's Behaviors via Active Sensoringcited by 1
  14. 2024Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Reviewcited by 1
  15. 2024Cross-Domain Policy Transfer by Representation Alignment via Multi-Domain Behavioral Cloningcited by 1
  16. 2025Beyond Domain Randomization: Event-Inspired Perception for Visually Robust Adversarial Imitation from Videoscited by 1
  17. 2025In-Context Policy Adaptation via Cross-Domain Skill Diffusioncited by 1
  18. 2025Balance Equation-based Distributionally Robust Offline Imitation Learningcited by 1
  19. 2026Semi-Supervised Cross-Domain Imitation Learningcited by 1
  20. 2022Transfer RL via the Undo Maps Formalismcited by 0
  21. 2023Learning Representative Trajectories of Dynamical Systems via Domain-Adaptive Imitationcited by 0
  22. 2024CONTRAST: Continual Multi-source Adaptation to Dynamic Distributionscited by 0
  23. 2024Plug-and-Play Transformer Modules for Test-Time Adaptationcited by 0
  24. 2024CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Communitycited by 0
  25. 2025Learning from Observation: A Survey of Recent Advancescited by 0
  26. 2025When a Robot is More Capable than a Human: Learning from Constrained Demonstratorscited by 0
  27. 2026Cross-Domain Demo-to-Code via Neurosymbolic Counterfactual Reasoningcited by 0
  28. 2026Reducing Oracle Feedback with Vision-Language Embeddings for Preference-Based RLcited by 0
  29. 2026Reinforcement Learning from Cross-domain Videos with Video Prediction Modelcited by 0
  30. 2026DADiff: Diffusion-Driven Cross-Domain Policy Adaptation for Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Domain Adaptive Imitation Learning
  2. 2020Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency
  3. 2019State Alignment-based Imitation Learning
  4. 2023A Simple Solution for Offline Imitation from Observations and Examples with Possibly Incomplete Trajectories
  5. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets
  6. 2020Causal Transfer for Imitation Learning and Decision Making under Sensor-shift
  7. 2022Imitation Learning by Estimating Expertise of Demonstrators
  8. 2026Difference-Aware Retrieval Policies for Imitation Learning