mlsanityData to 7 Oct 2026

Initial State Interventions for Deconfounded Imitation Learning

arXiv 2307.15980 ↗First posted 29 Jul 2023cs.LG cs.SY eess.SYcited by 4

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning suffers from causal confusion. This phenomenon occurs when learned policies attend to features that do not causally influence the expert actions but are instead spuriously correlated. Causally confused agents produce low open-loop supervised loss but poor closed-loop performance upon deployment. We consider the problem of masking observed confounders in a disentangled representation of the observation space. Our novel masking algorithm leverages the usual ability to intervene in the initial system state, avoiding any requirement involving expert querying, expert reward functions, or causal graph specification. Under certain assumptions, we theoretically prove that this algorithm is conservative in the sense that it does not incorrectly mask observations that causally influence the expert; furthermore, intervening on the initial state serves to strictly reduce excess conservatism. The masking algorithm is applied to behavior cloning for two illustrative control systems: CartPole and Reacher.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 18 of its 36 references that are in the map. First posted on arXiv 29 Jul 2023.

Builds on

18 of 36 references.

Closest ancestors · 7

Same direction, newest first

  1. 2022Sequential Causal Imitation Learning with Unobserved Confounders12 Aug
  2. 2022Causal Imitation Learning with Unobserved Confounders12 Aug
  3. 2022Causal Imitation Learning under Temporally Correlated Noise2 Feb
  4. 2021Object-Aware Regularization for Addressing Causal Confusion in Imitation Learning27 Oct
  5. 2020Imitation Learning with Stability and Safety Guarantees16 Dec
  6. 2020Causal Transfer for Imitation Learning and Decision Making under Sensor-shift2 Mar
  7. 2019Causal Confusion in Imitation Learning28 May

Other references · 11

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016OpenAI Gymcited by 1,684
  4. 2016End to End Learning for Self-Driving Carscited by 1,000
  5. 2018Understanding disentangling in β-VAEcited by 332
  6. 2022Causal Machine Learning: A Survey and Open Problemscited by 107
  7. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 92
  8. 2021Shaking the foundations: delusions in sequence models for interaction and controlcited by 61
  9. 2021Safe Reinforcement Learning with Chance-constrained Model Predictive Controlcited by 18
  10. 2019Optimal Passenger-Seeking Policies on E-hailing Platforms Using Markov Decision Process and Imitation Learningcited by 10
  11. 2021Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025CIVIL: Causal and Intuitive Visual Imitation Learningcited by 3
  2. 2025Causal Imitation Learning under Expert-Observable and Expert-Unobservable Confoundingcited by 1
  3. 2026Causal Imitation Learning Under Measurement Error and Distribution Shiftcited by 0
  4. 2026Scalable Causal Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Deconfounding Imitation Learning with Variational Inference
  2. 2025Confounded Causal Imitation Learning with Instrumental Variables
  3. 2023Invariant Causal Imitation Learning for Generalizable Policies
  4. 2025The Pitfalls of Imitation Learning when Actions are Continuous
  5. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning
  6. 2026TransMASK: Masked State Representation through Learned Transformation
  7. 2020Fighting Copycat Agents in Behavioral Cloning from Observation Histories
  8. 2020Imitating Unknown Policies via Exploration