# Learn what matters: cross-domain imitation learning with task-relevant embeddings

arXiv: https://arxiv.org/abs/2209.12093 · HTML: https://mlsanity.com/paper/2209.12093/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 30 of its 59 references that are in the map.
- First posted on arXiv 24 Sep 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 30 of 30)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [Time-Contrastive Networks: Self-Supervised Learning from Video](https://mlsanity.com/paper/1704.06888/) (2017)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [Decoupling Representation Learning from Reinforcement Learning](https://mlsanity.com/paper/2009.08319/) (2020)
- [Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model](https://mlsanity.com/paper/1907.00953/) (2019)
- [Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning](https://mlsanity.com/paper/1809.02925/) (2018)
- [Generative Adversarial Imitation from Observation](https://mlsanity.com/paper/1807.06158/) (2018)
- [Unsupervised State Representation Learning in Atari](https://mlsanity.com/paper/1906.08226/) (2019)
- [XIRL: Cross-embodiment Inverse Reinforcement Learning](https://mlsanity.com/paper/2106.03911/) (2021)
- [Third-Person Imitation Learning](https://mlsanity.com/paper/1703.01703/) (2017)
- [State-Only Imitation Learning for Dexterous Manipulation](https://mlsanity.com/paper/2004.04650/) (2020)
- [Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning](https://mlsanity.com/paper/1703.02949/) (2017)
- [Reinforcement Learning with Videos: Combining Offline Observations with Interaction](https://mlsanity.com/paper/2011.06507/) (2020)
- [Deep Reinforcement and InfoMax Learning](https://mlsanity.com/paper/2006.07217/) (2020)
- [State Alignment-based Imitation Learning](https://mlsanity.com/paper/1911.10947/) (2019)
- [What Matters for Adversarial Imitation Learning?](https://mlsanity.com/paper/2106.00672/) (2021)
- [Cross-Domain Imitation Learning via Optimal Transport](https://mlsanity.com/paper/2110.03684/) (2021)
- [Cross-domain Imitation from Observations](https://mlsanity.com/paper/2105.10037/) (2021)
- [State-only Imitation with Transition Dynamics Mismatch](https://mlsanity.com/paper/2002.11879/) (2020)
- [The MAGICAL Benchmark for Robust Imitation](https://mlsanity.com/paper/2011.00401/) (2020)
- [Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors](https://mlsanity.com/paper/2203.17138/) (2022)
- [Which Mutual-Information Representation Learning Objectives are Sufficient for Control?](https://mlsanity.com/paper/2106.07278/) (2021)
- [Domain-Robust Visual Imitation Learning with Mutual Information Constraints](https://mlsanity.com/paper/2103.05079/) (2021)
- [Robust Learning from Observation with Model Misspecification](https://mlsanity.com/paper/2202.06003/) (2022)
- [Cross Domain Robot Imitation with Invariant Representation](https://mlsanity.com/paper/2109.05940/) (2021)
- [Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatch](https://mlsanity.com/paper/2104.07810/) (2021)

## Built on by (most cited first, 14 of 14)

- [Humanoid Policy ~ Human Policy](https://mlsanity.com/paper/2503.13441/) (2025)
- [Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning](https://mlsanity.com/paper/2306.12755/) (2023)
- [A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents](https://mlsanity.com/paper/2402.04580/) (2024)
- [CEIL: Generalized Contextual Imitation Learning](https://mlsanity.com/paper/2306.14534/) (2023)
- [CLUE: Calibrated Latent Guidance for Offline Reinforcement Learning](https://mlsanity.com/paper/2306.13412/) (2023)
- [Domain Adaptive Imitation Learning with Visual Observation](https://mlsanity.com/paper/2312.00548/) (2023)
- [Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrations](https://mlsanity.com/paper/2305.13030/) (2023)
- [xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing](https://mlsanity.com/paper/2409.08687/) (2024)
- [Fair Representation Learning for Continuous Sensitive Attributes using Expectation of Integral Probability Metrics](https://mlsanity.com/paper/2505.06435/) (2025)
- [Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review](https://mlsanity.com/paper/2404.17687/) (2024)
- [SENSOR: Imitate Third-Person Expert's Behaviors via Active Sensoring](https://mlsanity.com/paper/2404.03386/) (2024)
- [Cross-Domain Policy Transfer by Representation Alignment via Multi-Domain Behavioral Cloning](https://mlsanity.com/paper/2407.16912/) (2024)
- [Semi-Supervised Cross-Domain Imitation Learning](https://mlsanity.com/paper/2602.10793/) (2026)
- [Learning from Observation: A Survey of Recent Advances](https://mlsanity.com/paper/2509.19379/) (2025)

## Abstract (arXiv)

We study how an autonomous agent learns to perform a task from demonstrations in a different domain, such as a different environment or different agent. Such cross-domain imitation learning is required to, for example, train an artificial agent from demonstrations of a human expert. We propose a scalable framework that enables cross-domain imitation learning without access to additional demonstrations or further domain knowledge. We jointly train the learner agent's policy and learn a mapping between the learner and expert domains with adversarial training. We effect this by using a mutual information criterion to find an embedding of the expert's state space that contains task-relevant information and is invariant to domain specifics. This step significantly simplifies estimating the mapping between the learner and expert domains and hence facilitates end-to-end learning. We demonstrate successful transfer of policies between considerably different domains, without extra supervision such as additional demonstrations, and in situations where other methods fail.