# Domain-Adversarial and Conditional State Space Model for Imitation Learning

arXiv: https://arxiv.org/abs/2001.11628 · HTML: https://mlsanity.com/paper/2001.11628/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 33 of its 48 references that are in the map.
- First posted on arXiv 31 Jan 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 30 of 33)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model](https://mlsanity.com/paper/1907.00953/) (2019)
- [Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning](https://mlsanity.com/paper/1809.02925/) (2018)
- [Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation](https://mlsanity.com/paper/1707.03374/) (2017)
- [Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping](https://mlsanity.com/paper/1709.07857/) (2017)
- [State Representation Learning for Control: An Overview](https://mlsanity.com/paper/1802.04181/) (2018)
- [AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos](https://mlsanity.com/paper/1912.04443/) (2019)
- [Third-Person Imitation Learning](https://mlsanity.com/paper/1703.01703/) (2017)
- [Recent Advances in Imitation Learning from Observation](https://mlsanity.com/paper/1905.13566/) (2019)
- [Learning human behaviors from motion capture by adversarial imitation](https://mlsanity.com/paper/1707.02201/) (2017)
- [Image-to-image translation for cross-domain disentanglement](https://mlsanity.com/paper/1805.09730/) (2018)
- [Graph-Structured Visual Imitation](https://mlsanity.com/paper/1907.05518/) (2019)
- [Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information](https://mlsanity.com/paper/1810.01266/) (2018)
- [Learning Robotic Manipulation through Visual Planning and Acting](https://mlsanity.com/paper/1905.04411/) (2019)
- [Variational Inference MPC for Bayesian Model-based Reinforcement Learning](https://mlsanity.com/paper/1907.04202/) (2019)
- [Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration](https://mlsanity.com/paper/1603.06348/) (2016)
- [Multi-Task Domain Adaptation for Deep Learning of Instance Grasping from Simulation](https://mlsanity.com/paper/1710.06422/) (2017)
- [Learning Belief Representations for Imitation Learning in POMDPs](https://mlsanity.com/paper/1906.09510/) (2019)
- [Path Integral Networks: End-to-End Differentiable Optimal Control](https://mlsanity.com/paper/1706.09597/) (2017)

## Built on by (most cited first, 7 of 7)

- [Domain-Robust Visual Imitation Learning with Mutual Information Constraints](https://mlsanity.com/paper/2103.05079/) (2021)
- [Adversarial Imitation Learning from Visual Observations using Latent Information](https://mlsanity.com/paper/2309.17371/) (2023)
- [Visually Robust Adversarial Imitation Learning from Videos with Contrastive Learning](https://mlsanity.com/paper/2407.12792/) (2024)
- [Class Is Invariant to Context and Vice Versa: On Learning Invariance for Out-Of-Distribution Generalization](https://mlsanity.com/paper/2208.03462/) (2022)
- [Tactile-Sensitive NewtonianVAE for High-Accuracy Industrial Connector Insertion](https://mlsanity.com/paper/2203.05955/) (2022)
- [Beyond Domain Randomization: Event-Inspired Perception for Visually Robust Adversarial Imitation from Videos](https://mlsanity.com/paper/2505.18899/) (2025)
- [World-Model-Based Control for Industrial box-packing of Multiple Objects using NewtonianVAE](https://mlsanity.com/paper/2308.02136/) (2023)

## Abstract (arXiv)

State representation learning (SRL) in partially observable Markov decision processes has been studied to learn abstract features of data useful for robot control tasks. For SRL, acquiring domain-agnostic states is essential for achieving efficient imitation learning. Without these states, imitation learning is hampered by domain-dependent information useless for control. However, existing methods fail to remove such disturbances from the states when the data from experts and agents show large domain shifts. To overcome this issue, we propose a domain-adversarial and conditional state space model (DAC-SSM) that enables control systems to obtain domain-agnostic and task- and dynamics-aware states. DAC-SSM jointly optimizes the state inference, observation reconstruction, forward dynamics, and reward models. To remove domain-dependent information from the states, the model is trained with domain discriminators in an adversarial manner, and the reconstruction is conditioned on domain labels. We experimentally evaluated the model predictive control performance via imitation learning for continuous control of sparse reward tasks in simulators and compared it with the performance of the existing SRL method. The agents from DAC-SSM achieved performance comparable to experts and more than twice the baselines. We conclude domain-agnostic states are essential for imitation learning that has large domain shifts and can be obtained using DAC-SSM.