# Sequential Causal Imitation Learning with Unobserved Confounders

arXiv: https://arxiv.org/abs/2208.06276 · HTML: https://mlsanity.com/paper/2208.06276/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 6 of its 28 references that are in the map.
- First posted on arXiv 12 Aug 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 6 of 6)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [Probabilistic Evaluation of Sequential Plans from Causal Models with Hidden Variables](https://mlsanity.com/paper/1302.4977/) (2013)
- [Causal Transfer for Imitation Learning and Decision Making under Sensor-shift](https://mlsanity.com/paper/2003.00806/) (2020)

## Built on by (most cited first, 25 of 25)

- [A Survey on Causal Reinforcement Learning](https://mlsanity.com/paper/2302.05209/) (2023)
- [Causal Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.01452/) (2023)
- [Neural Causal Abstractions](https://mlsanity.com/paper/2401.02602/) (2024)
- [TGRL: An Algorithm for Teacher Guided Reinforcement Learning](https://mlsanity.com/paper/2307.03186/) (2023)
- [Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning](https://mlsanity.com/paper/2406.03234/) (2024)
- [Instrumental Variables in Causal Inference and Machine Learning: A Survey](https://mlsanity.com/paper/2212.05778/) (2022)
- [Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding](https://mlsanity.com/paper/2306.01157/) (2023)
- [Confounding Robust Deep Reinforcement Learning: A Causal Approach](https://mlsanity.com/paper/2510.21110/) (2025)
- [Deep Autoregressive Models as Causal Inference Engines](https://mlsanity.com/paper/2409.18581/) (2024)
- [Automatic Reward Shaping from Confounded Offline Data](https://mlsanity.com/paper/2505.11478/) (2025)
- [Deconfounding Imitation Learning with Variational Inference](https://mlsanity.com/paper/2211.02667/) (2022)
- [Causality from Bottom to Top: A Survey](https://mlsanity.com/paper/2403.11219/) (2024)
- [Initial State Interventions for Deconfounded Imitation Learning](https://mlsanity.com/paper/2307.15980/) (2023)
- [Confounding Robust Continuous Control via Automatic Reward Shaping](https://mlsanity.com/paper/2602.10305/) (2026)
- [Active and Passive Causal Inference Learning](https://mlsanity.com/paper/2308.09248/) (2023)
- [Exploring and Addressing Reward Confusion in Offline Preference Learning](https://mlsanity.com/paper/2407.16025/) (2024)
- [Causal Imitation Learning under Expert-Observable and Expert-Unobservable Confounding](https://mlsanity.com/paper/2502.07656/) (2025)
- [On Transportability for Structural Causal Bandits](https://mlsanity.com/paper/2511.17953/) (2025)
- [DIGIC: Domain Generalizable Imitation Learning by Causal Discovery](https://mlsanity.com/paper/2402.18910/) (2024)
- [Disentangled Representations for Causal Cognition](https://mlsanity.com/paper/2407.00744/) (2024)
- [Causal Policy Learning in Reinforcement Learning: Backdoor-Adjusted Soft Actor-Critic](https://mlsanity.com/paper/2506.05445/) (2025)
- [Causal Imitation Learning Under Measurement Error and Distribution Shift](https://mlsanity.com/paper/2601.22206/) (2026)
- [Quantifying Potential Observation Missingness in Inverse Reinforcement Learning](https://mlsanity.com/paper/2605.12831/) (2026)
- [Scalable Causal Imitation Learning](https://mlsanity.com/paper/2607.17003/) (2026)
- [Learning Implicit Causal World Models from Multi-Agent Demonstrations](https://mlsanity.com/paper/2607.26336/) (2026)

## Abstract (arXiv)

"Monkey see monkey do" is an age-old adage, referring to na"ive imitation without a deep understanding of a system's underlying mechanics. Indeed, if a demonstrator has access to information unavailable to the imitator (monkey), such as a different set of sensors, then no matter how perfectly the imitator models its perceived environment (See), attempting to reproduce the demonstrator's behavior (Do) can lead to poor outcomes. Imitation learning in the presence of a mismatch between demonstrator and imitator has been studied in the literature under the rubric of causal imitation learning (Zhang et al., 2020), but existing solutions are limited to single-stage decision-making. This paper investigates the problem of causal imitation learning in sequential settings, where the imitator must make multiple decisions per episode. We develop a graphical criterion that is necessary and sufficient for determining the feasibility of causal imitation, providing conditions when an imitator can match a demonstrator's performance despite differing capabilities. Finally, we provide an efficient algorithm for determining imitability and corroborate our theory with simulations.