mlsanityData to 7 Oct 2026

Causal Confusion in Imitation Learning

arXiv 1905.11979 ↗First posted 28 May 2019cs.LG stat.MLcited by 286

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Behavioral cloning reduces policy learning to supervised learning by training a discriminative model to predict expert actions given observations. Such discriminative models are non-causal: the training procedure is unaware of the causal structure of the interaction between the expert and the environment. We point out that ignoring causality is particularly damaging because of the distributional shift in imitation learning. In particular, it leads to a counter-intuitive "causal misidentification" phenomenon: access to more information can yield worse performance. We investigate how this problem arises, and propose a solution to combat it through targeted interventions---either environment interaction or expert queries---to determine the correct causal model. We show that causal misidentification occurs in several benchmark control domains as well as realistic driving settings, and validate our solution against DAgger and other baselines and ablations.

Key facts. Cited by 286 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 28 of its 58 references that are in the map. First posted on arXiv 28 May 2019.

Builds on

28 of 58 references.

References · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  8. 2016End to End Learning for Self-Driving Carscited by 1,000
  9. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  10. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  11. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  12. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
16 more
  1. 2017Causal Effect Inference with Deep Latent-Variable Modelscited by 408
  2. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  3. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  4. 2018Understanding disentangling in β-VAEcited by 332
  5. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 302
  6. 2013Causal Discovery with Continuous Additive Noise Modelscited by 252
  7. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  8. 2018Isolating Sources of Disentanglement in Variational Autoencoderscited by 167
  9. 2018The Blessings of Multiple Causescited by 103
  10. 2016Discovering Causal Signals in Imagescited by 79
  11. 2015Learning Causal Graphs with Small Interventionscited by 63
  12. 2015A fast PC algorithm for high dimensional causal discovery with multi-core PCscited by 45
  13. 2017Identifying Best Interventions through Online Importance Samplingcited by 39
  14. 2019Monocular Plan View Networks for Autonomous Drivingcited by 37
  15. 2017Estimating Mutual Information for Discrete-Continuous Mixturescited by 34
  16. 2018Causal Inference via Kernel Deviance Measurescited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 2; 2020: 6; 2021: 36; 2022: 43; 2023: 52; 2024: 32; 2025: 52; 2026: 63.

286 papers.

Sort:

By area:

  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  2. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  3. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  4. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  5. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  6. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
  7. 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 209
  8. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  9. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
  10. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  11. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  12. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 137
38 more
  1. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 135
  2. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  3. 2021Salient ImageNet: How to discover spurious features in Deep Learning?cited by 114
  4. 2023Policy Expansion for Bridging Offline-to-Online Reinforcement Learningcited by 103
  5. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 102
  6. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  7. 2021Interventional Video Grounding with Dual Contrastive Learningcited by 94
  8. 2019Urban Driving with Conditional Imitation Learningcited by 84
  9. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
  10. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  11. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  12. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  13. 2020Fighting Copycat Agents in Behavioral Cloning from Observation Historiescited by 70
  14. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shiftcited by 68
  15. 2021SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policiescited by 68
  16. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulatorscited by 65
  17. 2021Less is More: Generating Grounded Navigation Instructions from Landmarkscited by 62
  18. 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representationscited by 60
  19. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 56
  20. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Modelcited by 56
  21. 2021Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learningcited by 53
  22. 2022Causal Confusion and Reward Misidentification in Preference-Based Reward Learningcited by 52
  23. 2022Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systemscited by 50
  24. 2021Counter-Strike Deathmatch with Large-Scale Behavioural Cloningcited by 49
  25. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 49
  26. 2024Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Samplingcited by 46
  27. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  28. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 43
  29. 2023A Survey on Causal Reinforcement Learningcited by 43
  30. 2025HAMLET: Switch your Vision-Language-Action Model into a History-Aware Policycited by 42
  31. 2025iPad: Iterative Proposal-centric End-to-End Autonomous Drivingcited by 41
  32. 2022Causal Imitation Learning with Unobserved Confounderscited by 40
  33. 2021Causal Reinforcement Learning using Observational and Interventional Datacited by 38
  34. 2022Sequence Model Imitation Learning with Unobserved Contextscited by 34
  35. 2025ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Contextcited by 34
  36. 2023Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behaviorcited by 33
  37. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  38. 2025CarPlanner: Consistent Auto-regressive Trajectory Planning for Large-scale Reinforcement Learning in Autonomous Drivingcited by 31

50 most cited of 286 All 286 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Confounded Causal Imitation Learning with Instrumental Variables
  2. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning
  3. 2026When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning
  4. 2026Learning When to Stop: Selective Imitation Learning Under Arbitrary Dynamics Shift
  5. 2021Fully General Online Imitation Learning
  6. 2018Imitating Latent Policies from Observation
  7. 2025A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search
  8. 2021Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coverage