# Invariant Causal Imitation Learning for Generalizable Policies

arXiv: https://arxiv.org/abs/2311.01489 · HTML: https://mlsanity.com/paper/2311.01489/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 31 of its 59 references that are in the map.
- First posted on arXiv 2 Nov 2023.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 30 of 31)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Environment Inference for Invariant Learning](https://mlsanity.com/paper/2010.07249/) (2020)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [One-Shot Imitation Learning](https://mlsanity.com/paper/1703.07326/) (2017)
- [One-Shot Visual Imitation Learning via Meta-Learning](https://mlsanity.com/paper/1709.04905/) (2017)
- [A Divergence Minimization Perspective on Imitation Learning Methods](https://mlsanity.com/paper/1911.02256/) (2019)
- [Invariant Risk Minimization Games](https://mlsanity.com/paper/2002.04692/) (2020)
- [Imitation Learning via Off-Policy Distribution Matching](https://mlsanity.com/paper/1912.05032/) (2019)
- [A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models](https://mlsanity.com/paper/1611.03852/) (2016)
- [Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation](https://mlsanity.com/paper/1707.03374/) (2017)
- [One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning](https://mlsanity.com/paper/1802.01557/) (2018)
- [Invariant Causal Prediction for Block MDPs](https://mlsanity.com/paper/2003.06016/) (2020)
- [Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller](https://mlsanity.com/paper/1911.09676/) (2019)
- [Task-Embedded Control Networks for Few-Shot Imitation Learning](https://mlsanity.com/paper/1810.03237/) (2018)
- [Strictly Batch Imitation Learning by Energy-based Distribution Matching](https://mlsanity.com/paper/2006.14154/) (2020)
- [Causal Imitation Learning with Unobserved Confounders](https://mlsanity.com/paper/2208.06267/) (2022)
- [Energy-Based Imitation Learning](https://mlsanity.com/paper/2004.09395/) (2020)
- [Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learning](https://mlsanity.com/paper/2006.01096/) (2020)
- [Natural Environment Benchmarks for Reinforcement Learning](https://mlsanity.com/paper/1811.06032/) (2018)
- [Model-based Adversarial Imitation Learning](https://mlsanity.com/paper/1612.02179/) (2016)
- [Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies](https://mlsanity.com/paper/2103.16772/) (2021)
- [Resolving Spurious Correlations in Causal Models of Environments via Interventions](https://mlsanity.com/paper/2002.05217/) (2020)
- [Inverse Active Sensing: Modeling and Understanding Timely Decision-Making](https://mlsanity.com/paper/2006.14141/) (2020)
- [Causal Transfer for Imitation Learning and Decision Making under Sensor-shift](https://mlsanity.com/paper/2003.00806/) (2020)

## Built on by (most cited first, 15 of 15)

- [VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Caching](https://mlsanity.com/paper/2502.02175/) (2025)
- [Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation](https://mlsanity.com/paper/2508.06426/) (2025)
- [Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning](https://mlsanity.com/paper/2406.03234/) (2024)
- [Reinforced Sequential Decision-Making for Sepsis Treatment: The POSNEGDM Framework with Mortality Classifier and Transformer](https://mlsanity.com/paper/2403.07309/) (2024)
- [Causality from Bottom to Top: A Survey](https://mlsanity.com/paper/2403.11219/) (2024)
- [Sparse Model Inversion: Efficient Inversion of Vision Transformers for Data-Free Applications](https://mlsanity.com/paper/2510.27186/) (2025)
- [BECAUSE: Bilinear Causal Representation for Generalizable Offline Model-based Reinforcement Learning](https://mlsanity.com/paper/2407.10967/) (2024)
- [CIVIL: Causal and Intuitive Visual Imitation Learning](https://mlsanity.com/paper/2504.17959/) (2025)
- [Learning Causally Invariant Reward Functions from Diverse Demonstrations](https://mlsanity.com/paper/2409.08012/) (2024)
- [Causal Imitation Learning under Expert-Observable and Expert-Unobservable Confounding](https://mlsanity.com/paper/2502.07656/) (2025)
- [DIGIC: Domain Generalizable Imitation Learning by Causal Discovery](https://mlsanity.com/paper/2402.18910/) (2024)
- [Improved Monte Carlo Planning via Causal Disentanglement for Structurally-Decomposed Markov Decision Processes](https://mlsanity.com/paper/2406.16151/) (2024)
- [Disentangled Representations for Causal Cognition](https://mlsanity.com/paper/2407.00744/) (2024)
- [Simulated Human Learning in a Dynamic, Partially-Observed, Time-Series Environment](https://mlsanity.com/paper/2511.15032/) (2025)
- [Learning Causal States Under Partial Observability and Perturbation](https://mlsanity.com/paper/2512.00357/) (2025)

## Abstract (arXiv)

Consider learning an imitation policy on the basis of demonstrated behavior from multiple environments, with an eye towards deployment in an unseen environment. Since the observable features from each setting may be different, directly learning individual policies as mappings from features to actions is prone to spurious correlations -- and may not generalize well. However, the expert's policy is often a function of a shared latent structure underlying those observable features that is invariant across settings. By leveraging data from multiple environments, we propose Invariant Causal Imitation Learning (ICIL), a novel technique in which we learn a feature representation that is invariant across domains, on the basis of which we learn an imitation policy that matches expert behavior. To cope with transition dynamics mismatch, ICIL learns a shared representation of causal features (for all training environments), that is disentangled from the specific representations of noise variables (for each of those environments). Moreover, to ensure that the learned policy matches the observation distribution of the expert's policy, ICIL estimates the energy of the expert's observations and uses a regularization term that minimizes the imitator policy's next state energy. Experimentally, we compare our methods against several benchmarks in control and healthcare tasks and show its effectiveness in learning imitation policies capable of generalizing to unseen environments.