mlsanityData to 7 Oct 2026

Fighting Copycat Agents in Behavioral Cloning from Observation Histories

arXiv 2010.14876 ↗First posted 28 Oct 2020cs.LG cs.ROcited by 70

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning trains policies to map from input observations to the actions that an expert would choose. In this setting, distribution shift frequently exacerbates the effect of misattributing expert actions to nuisance correlates among the observed variables. We observe that a common instance of this causal confusion occurs in partially observed settings when expert actions are strongly correlated over time: the imitator learns to cheat by predicting the expert's previous action, rather than the next action. To combat this "copycat problem", we propose an adversarial approach to learn a feature representation that removes excess information about the previous expert action nuisance correlate, while retaining the information necessary to predict the next action. In our experiments, our approach improves performance significantly across a variety of partially observed imitation learning tasks.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 53 references that are in the map. First posted on arXiv 28 Oct 2020.

Builds on

29 of 53 references.

Closest ancestors · 2

Same direction, newest first

  1. 2019Causal Confusion in Imitation Learning28 May
  2. 2018An Algorithmic Perspective on Imitation Learning16 Nov

Other references · 27

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  5. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  6. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  7. 2016Generative Adversarial Imitation Learningcited by 1,576
  8. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  9. 2016End to End Learning for Self-Driving Carscited by 1,000
  10. 2016Deep Variational Information Bottleneckcited by 915
  11. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 597
  12. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
15 more
  1. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  2. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  3. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  4. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  5. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  6. 2019Recurrent Independent Mechanismscited by 219
  7. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  8. 2019A Meta-Transfer Objective for Learning to Disentangle Causal Mechanismscited by 158
  9. 2018Invariance, Causality and Robustnesscited by 123
  10. 2017Learning Independent Causal Mechanismscited by 104
  11. 2017Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Predictioncited by 93
  12. 2016Amortised MAP Inference for Image Super-resolutioncited by 68
  13. 2019Non-Monotonic Sequential Text Generationcited by 55
  14. 2019Monocular Plan View Networks for Autonomous Drivingcited by 37
  15. 2020Learning Task-Driven Control Policies via Information Bottleneckscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 7; 2023: 15; 2024: 8; 2025: 18; 2026: 16.

70 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  3. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  4. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  5. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  6. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  7. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  8. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 142
  9. 2025Pseudo-Simulation for Autonomous Drivingcited by 141
  10. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 135
  11. 2023Policy Expansion for Bridging Offline-to-Online Reinforcement Learningcited by 103
  12. 2023Hidden Biases of End-to-End Driving Modelscited by 95
38 more
  1. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
  2. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shiftcited by 68
  3. 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representationscited by 60
  4. 2024Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Samplingcited by 46
  5. 2025Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)cited by 45
  6. 2023A Survey on Causal Reinforcement Learningcited by 43
  7. 2025HAMLET: Switch your Vision-Language-Action Model into a History-Aware Policycited by 42
  8. 2025LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilitiescited by 36
  9. 2022Sequence Model Imitation Learning with Unobserved Contextscited by 34
  10. 2025ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Contextcited by 34
  11. 2025Learning Long-Context Diffusion Policies via Past-Token Predictioncited by 28
  12. 2021Keyframe-Focused Visual Imitation Learningcited by 25
  13. 2023Rethinking Imitation-based Planner for Autonomous Drivingcited by 25
  14. 2021Causal Navigation by Continuous-time Neural Networkscited by 23
  15. 2023For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equalcited by 22
  16. 2024Better than Your Teacher: LLM Agents that learn from Privileged AI Feedbackcited by 21
  17. 2023Coaching a Teachable Studentcited by 17
  18. 2026BPP: Long-Context Robot Imitation Learning by Focusing on Key History Framescited by 17
  19. 2021Object-Aware Regularization for Addressing Causal Confusion in Imitation Learningcited by 16
  20. 2025Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorercited by 16
  21. 2022Fighting Fire with Fire: Avoiding DNN Shortcuts through Primingcited by 15
  22. 2022Resolving Copycat Problems in Visual Imitation Learning via Residual Action Predictioncited by 14
  23. 2024A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Taskscited by 14
  24. 2026RoboTTT: Context Scaling for Robot Policiescited by 13
  25. 2024Causal Action Influence Aware Counterfactual Data Augmentationcited by 12
  26. 2023Scaling Laws for Imitation Learning in Single-Agent Gamescited by 11
  27. 2025Structured Agent Distillation for Large Language Modelcited by 11
  28. 2025Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulationcited by 10
  29. 2024M2DA: Multi-Modal Fusion Transformer Incorporating Driver Attention for Autonomous Drivingcited by 7
  30. 2026VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memorycited by 6
  31. 2022Deconfounding Imitation Learning with Variational Inferencecited by 5
  32. 2025GABRIL: Gaze-Based Regularization for Mitigating Causal Confusion in Imitation Learningcited by 3
  33. 2026Rethinking Camera Choice: An Empirical Study on Fisheye Camera Properties in Robotic Manipulationcited by 3
  34. 2023Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning?cited by 2
  35. 2025Ratatouille: Imitation Learning Ingredients for Real-world Social Robot Navigationcited by 2
  36. 2026Training and Evaluating Diffusion Policies with Long Context Lengthscited by 2
  37. 2023Target-point Attention Transformer: A novel trajectory predict network for end-to-end autonomous drivingcited by 1
  38. 2025Causal Imitation Learning under Expert-Observable and Expert-Unobservable Confoundingcited by 1

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2018Behavioral Cloning from Observation
  2. 2022Diverse Imitation Learning via Self-Organizing Generative Models
  3. 2020Imitating Unknown Policies via Exploration
  4. 2021Imitation by Predicting Observations
  5. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  6. 2023Adversarial Imitation Learning from Visual Observations using Latent Information
  7. 2023Initial State Interventions for Deconfounded Imitation Learning
  8. 2023Sample-efficient Adversarial Imitation Learning