mlsanityData to 7 Oct 2026

Causal Imitation Learning under Temporally Correlated Noise

arXiv 2202.01312 ↗First posted 2 Feb 2022cs.LG cs.ROcited by 28

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We develop algorithms for imitation learning from policy data that was corrupted by temporally correlated noise in expert actions. When noise affects multiple timesteps of recorded data, it can manifest as spurious correlations between states and actions that a learner might latch on to, leading to poor policy performance. To break up these spurious correlations, we apply modern variants of the instrumental variable regression (IVR) technique of econometrics, enabling us to recover the underlying policy without requiring access to an interactive expert. In particular, we present two techniques, one of a generative-modeling flavor (DoubIL) that can utilize access to a simulator, and one of a game-theoretic flavor (ResiduIL) that can be run entirely offline. We find both of our algorithms compare favorably to behavioral cloning on simulated control tasks.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 39 references that are in the map. First posted on arXiv 2 Feb 2022.

Builds on

16 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap4 Mar
  2. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shift4 Feb
  3. 2019Causal Confusion in Imitation Learning28 May
  4. 2019Provably Efficient Imitation Learning from Observation Alone27 May

Other references · 12

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  5. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  6. 2015Fast Convergence of Regularized Learning in Gamescited by 149
  7. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 116
  8. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 92
  9. 2015Algorithmic Stability for Adaptive Data Analysiscited by 88
  10. 2020Minimax Estimation of Conditional Moment Modelscited by 83
  11. 2013Counterfactual Probabilities: Computational Methods, Bounds and Applicationscited by 67
  12. 2021On Instrumental Variable Regression for Deep Offline Policy Evaluationcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 6; 2024: 8; 2025: 7; 2026: 3.

28 papers.

Sort:

By area:

  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  2. 2024A Minimaximalist Approach to Reinforcement Learning from Human Feedbackcited by 141
  3. 2022Causal Confusion and Reward Misidentification in Preference-Based Reward Learningcited by 52
  4. 2024Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Samplingcited by 46
  5. 2023A Survey on Causal Reinforcement Learningcited by 43
  6. 2022Sequence Model Imitation Learning with Unobserved Contextscited by 34
  7. 2024Bi-ACT: Bilateral Control-Based Imitation Learning via Action Chunking with Transformercited by 27
  8. 2023Causal Reinforcement Learning: A Surveycited by 22
  9. 2023A Survey of Demonstration Learningcited by 12
  10. 2024Multi-Agent Imitation Learning: Value is Easy, Regret is Hardcited by 11
  11. 2024SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulationcited by 8
  12. 2023Causal Imitability Under Context-Specific Independence Relationscited by 6
16 more
  1. 2022Deconfounding Imitation Learning with Variational Inferencecited by 5
  2. 2023Initial State Interventions for Deconfounded Imitation Learningcited by 4
  3. 2025Efficient Imitation under Misspecificationcited by 4
  4. 2026Closed-Loop Action Chunks with Dynamic Corrections for Training-Free Diffusion Policycited by 3
  5. 2024Learning Causally Invariant Reward Functions from Diverse Demonstrationscited by 2
  6. 2025Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coachingcited by 2
  7. 2025Self-Guided Action Diffusioncited by 2
  8. 2022NeuroCERIL: Robotic Imitation Learning via Hierarchical Cause-Effect Reasoning in Programmable Attractor Neural Networkscited by 1
  9. 2025Causal Imitation Learning under Expert-Observable and Expert-Unobservable Confoundingcited by 1
  10. 2025Exploiting Policy Idling for Dexterous Manipulationcited by 1
  11. 2024The Virtues of Pessimism in Inverse Reinforcement Learningcited by 0
  12. 2024AI-Guided Molecular Simulations in VR: Exploring Strategies for Imitation Learning in Hyperdimensional Molecular Systemscited by 0
  13. 2025Causal Policy Learning in Reinforcement Learning: Backdoor-Adjusted Soft Actor-Criticcited by 0
  14. 2025Goal Discovery with Causal Capacity for Efficient Reinforcement Learningcited by 0
  15. 2026Causal Imitation Learning Under Measurement Error and Distribution Shiftcited by 0
  16. 2026Scalable Causal Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Confounded Causal Imitation Learning with Instrumental Variables
  2. 2018Imitating Latent Policies from Observation
  3. 2022Causal Imitation Learning with Unobserved Confounders
  4. 2023Invariant Causal Imitation Learning for Generalizable Policies
  5. 2024Don't Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion
  6. 2026Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback
  7. 2025Reinforcement Learning via Implicit Imitation Guidance
  8. 2025The Pitfalls of Imitation Learning when Actions are Continuous