# Cross-domain Imitation from Observations

arXiv: https://arxiv.org/abs/2105.10037 · HTML: https://mlsanity.com/paper/2105.10037/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 20 May 2021.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 25 of 25)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation](https://mlsanity.com/paper/1707.03374/) (2017)
- [AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos](https://mlsanity.com/paper/1912.04443/) (2019)
- [Third-Person Imitation Learning](https://mlsanity.com/paper/1703.01703/) (2017)
- [RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Real](https://mlsanity.com/paper/2006.09001/) (2020)
- [Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning](https://mlsanity.com/paper/1703.02949/) (2017)
- [Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller](https://mlsanity.com/paper/1911.09676/) (2019)
- [RetinaGAN: An Object-aware Approach to Sim-to-Real Transfer](https://mlsanity.com/paper/2011.03148/) (2020)
- [Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation](https://mlsanity.com/paper/1806.07377/) (2018)
- [Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement](https://mlsanity.com/paper/1910.04417/) (2019)
- [State-only Imitation with Transition Dynamics Mismatch](https://mlsanity.com/paper/2002.11879/) (2020)
- [Learning from Trajectories via Subgoal Discovery](https://mlsanity.com/paper/1911.07224/) (2019)
- [Mutual Information Based Knowledge Transfer Under State-Action Dimension Mismatch](https://mlsanity.com/paper/2006.07041/) (2020)

## Built on by (most cited first, 30 of 42)

- [A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges](https://mlsanity.com/paper/2309.02473/) (2023)
- [Cross-Domain Imitation Learning via Optimal Transport](https://mlsanity.com/paper/2110.03684/) (2021)
- [Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matching](https://mlsanity.com/paper/2202.02433/) (2022)
- [Robust Imitation Learning against Variations in Environment Dynamics](https://mlsanity.com/paper/2206.09314/) (2022)
- [Cross-Domain Policy Adaptation by Capturing Representation Mismatch](https://mlsanity.com/paper/2405.15369/) (2024)
- [ODRL: A Benchmark for Off-Dynamics Reinforcement Learning](https://mlsanity.com/paper/2410.20750/) (2024)
- [A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents](https://mlsanity.com/paper/2402.04580/) (2024)
- [A Survey of Imitation Learning Methods, Environments and Metrics](https://mlsanity.com/paper/2404.19456/) (2024)
- [Effective Restoration of Source Knowledge in Continual Test Time Adaptation](https://mlsanity.com/paper/2311.04991/) (2023)
- [Learn what matters: cross-domain imitation learning with task-relevant embeddings](https://mlsanity.com/paper/2209.12093/) (2022)
- [CEIL: Generalized Contextual Imitation Learning](https://mlsanity.com/paper/2306.14534/) (2023)
- [Prior-guided Source-free Domain Adaptation for Human Pose Estimation](https://mlsanity.com/paper/2308.13954/) (2023)
- [Domain Adaptive Imitation Learning with Visual Observation](https://mlsanity.com/paper/2312.00548/) (2023)
- [Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training](https://mlsanity.com/paper/2509.18631/) (2025)
- [Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrations](https://mlsanity.com/paper/2305.13030/) (2023)
- [Visually Robust Adversarial Imitation Learning from Videos with Contrastive Learning](https://mlsanity.com/paper/2407.12792/) (2024)
- [xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing](https://mlsanity.com/paper/2409.08687/) (2024)
- [Preference VLM: Leveraging VLMs for Scalable Preference-Based Reinforcement Learning](https://mlsanity.com/paper/2502.01616/) (2025)
- [Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemble](https://mlsanity.com/paper/2206.00238/) (2022)
- [SUMMIT: Source-Free Adaptation of Uni-Modal Models to Multi-Modal Targets](https://mlsanity.com/paper/2308.11880/) (2023)
- [Seeing Differently, Acting Similarly: Heterogeneously Observable Imitation Learning](https://mlsanity.com/paper/2106.09256/) (2021)
- [Similarity-based Knowledge Transfer for Cross-Domain Reinforcement Learning](https://mlsanity.com/paper/2312.03764/) (2023)
- [Off-Dynamics Inverse Reinforcement Learning from Hetero-Domain](https://mlsanity.com/paper/2110.11443/) (2021)
- [Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review](https://mlsanity.com/paper/2404.17687/) (2024)
- [Spatially Visual Perception for End-to-End Robotic Learning](https://mlsanity.com/paper/2411.17458/) (2024)
- [SENSOR: Imitate Third-Person Expert's Behaviors via Active Sensoring](https://mlsanity.com/paper/2404.03386/) (2024)
- [Cross-Domain Policy Transfer by Representation Alignment via Multi-Domain Behavioral Cloning](https://mlsanity.com/paper/2407.16912/) (2024)
- [Beyond Domain Randomization: Event-Inspired Perception for Visually Robust Adversarial Imitation from Videos](https://mlsanity.com/paper/2505.18899/) (2025)
- [In-Context Policy Adaptation via Cross-Domain Skill Diffusion](https://mlsanity.com/paper/2509.04535/) (2025)
- [When a Robot is More Capable than a Human: Learning from Constrained Demonstrators](https://mlsanity.com/paper/2510.09096/) (2025)

## Abstract (arXiv)

Imitation learning seeks to circumvent the difficulty in designing proper reward functions for training agents by utilizing expert behavior. With environments modeled as Markov Decision Processes (MDP), most of the existing imitation algorithms are contingent on the availability of expert demonstrations in the same MDP as the one in which a new imitation policy is to be learned. In this paper, we study the problem of how to imitate tasks when there exist discrepancies between the expert and agent MDP. These discrepancies across domains could include differing dynamics, viewpoint, or morphology; we present a novel framework to learn correspondences across such domains. Importantly, in contrast to prior works, we use unpaired and unaligned trajectories containing only states in the expert domain, to learn this correspondence. We utilize a cycle-consistency constraint on both the state space and a domain agnostic latent space to do this. In addition, we enforce consistency on the temporal position of states via a normalized position estimator function, to align the trajectories across the two domains. Once this correspondence is found, we can directly transfer the demonstrations on one domain to the other and use it for imitation. Experiments across a wide variety of challenging domains demonstrate the efficacy of our approach.