# Causal Imitation Learning with Unobserved Confounders

arXiv: https://arxiv.org/abs/2208.06267 · HTML: https://mlsanity.com/paper/2208.06267/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 9 of its 47 references that are in the map.
- First posted on arXiv 12 Aug 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 9 of 9)

- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training](https://mlsanity.com/paper/1709.02023/) (2017)
- [Boundary-Seeking Generative Adversarial Networks](https://mlsanity.com/paper/1702.08431/) (2017)
- [Causal Generative Neural Networks](https://mlsanity.com/paper/1711.08936/) (2017)
- [Causal Transfer for Imitation Learning and Decision Making under Sensor-shift](https://mlsanity.com/paper/2003.00806/) (2020)

## Built on by (most cited first, 30 of 41)

- [A Survey on Causal Reinforcement Learning](https://mlsanity.com/paper/2302.05209/) (2023)
- [Causal Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.01452/) (2023)
- [Passive learning of active causal strategies in agents and language models](https://mlsanity.com/paper/2305.16183/) (2023)
- [Neural Causal Abstractions](https://mlsanity.com/paper/2401.02602/) (2024)
- [Interpretable Reward Redistribution in Reinforcement Learning: A Causal Approach](https://mlsanity.com/paper/2305.18427/) (2023)
- [Invariant Causal Imitation Learning for Generalizable Policies](https://mlsanity.com/paper/2311.01489/) (2023)
- [Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search](https://mlsanity.com/paper/2407.11070/) (2024)
- [Comparing Causal Frameworks: Potential Outcomes, Structural Models, Graphs, and Abstractions](https://mlsanity.com/paper/2306.14351/) (2023)
- [Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning](https://mlsanity.com/paper/2406.03234/) (2024)
- [Confounding Robust Deep Reinforcement Learning: A Causal Approach](https://mlsanity.com/paper/2510.21110/) (2025)
- [Causal Imitability Under Context-Specific Independence Relations](https://mlsanity.com/paper/2306.00585/) (2023)
- [MACCA: Offline Multi-agent Reinforcement Learning with Causal Credit Assignment](https://mlsanity.com/paper/2312.03644/) (2023)
- [Automatic Reward Shaping from Confounded Offline Data](https://mlsanity.com/paper/2505.11478/) (2025)
- [Causality from Bottom to Top: A Survey](https://mlsanity.com/paper/2403.11219/) (2024)
- [Initial State Interventions for Deconfounded Imitation Learning](https://mlsanity.com/paper/2307.15980/) (2023)
- [A Data-Driven State Aggregation Approach for Dynamic Discrete Choice Models](https://mlsanity.com/paper/2304.04916/) (2023)
- [BECAUSE: Bilinear Causal Representation for Generalizable Offline Model-based Reinforcement Learning](https://mlsanity.com/paper/2407.10967/) (2024)
- [Reinforcement Learning with Continuous Actions Under Unmeasured Confounding](https://mlsanity.com/paper/2505.00304/) (2025)
- [Benign Shortcut for Debiasing: Fair Visual Recognition via Intervention with Shortcut Features](https://mlsanity.com/paper/2308.08482/) (2023)
- [Sequential Decision Making with Expert Demonstrations under Unobserved Heterogeneity](https://mlsanity.com/paper/2404.07266/) (2024)
- [Epistemic Regret Minimization: Label-Free Causal Critique Beyond Outcome Reward](https://mlsanity.com/paper/2602.11675/) (2026)
- [NeuroCERIL: Robotic Imitation Learning via Hierarchical Cause-Effect Reasoning in Programmable Attractor Neural Networks](https://mlsanity.com/paper/2211.06462/) (2022)
- [Active and Passive Causal Inference Learning](https://mlsanity.com/paper/2308.09248/) (2023)
- [Limitations of Agents Simulated by Predictive Models](https://mlsanity.com/paper/2402.05829/) (2024)
- [Why Online Reinforcement Learning is Causal](https://mlsanity.com/paper/2403.04221/) (2024)
- [Exploring and Addressing Reward Confusion in Offline Preference Learning](https://mlsanity.com/paper/2407.16025/) (2024)
- [Causal Imitation Learning under Expert-Observable and Expert-Unobservable Confounding](https://mlsanity.com/paper/2502.07656/) (2025)
- [Text-Driven Causal Representation Learning for Source-Free Domain Generalization](https://mlsanity.com/paper/2507.09961/) (2025)
- [Learning Causal Structure Distributions for Robust Planning](https://mlsanity.com/paper/2508.06742/) (2025)
- [On Transportability for Structural Causal Bandits](https://mlsanity.com/paper/2511.17953/) (2025)

## Abstract (arXiv)

One of the common ways children learn is by mimicking adults. Imitation learning focuses on learning policies with suitable performance from demonstrations generated by an expert, with an unspecified performance measure, and unobserved reward signal. Popular methods for imitation learning start by either directly mimicking the behavior policy of an expert (behavior cloning) or by learning a reward function that prioritizes observed expert trajectories (inverse reinforcement learning). However, these methods rely on the assumption that covariates used by the expert to determine her/his actions are fully observed. In this paper, we relax this assumption and study imitation learning when sensory inputs of the learner and the expert differ. First, we provide a non-parametric, graphical criterion that is complete (both necessary and sufficient) for determining the feasibility of imitation from the combinations of demonstration data and qualitative assumptions about the underlying environment, represented in the form of a causal model. We then show that when such a criterion does not hold, imitation could still be feasible by exploiting quantitative knowledge of the expert trajectories. Finally, we develop an efficient procedure for learning the imitating policy from experts' trajectories.