mlsanityData to 7 Oct 2026

Resolving Copycat Problems in Visual Imitation Learning via Residual Action Prediction

arXiv 2207.09705 ↗First posted 20 Jul 2022cs.CVcited by 14

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning is a widely used policy learning method that enables intelligent agents to acquire complex skills from expert demonstrations. The input to the imitation learning algorithm is usually composed of both the current observation and historical observations since the most recent observation might not contain enough information. This is especially the case with image observations, where a single image only includes one view of the scene, and it suffers from a lack of motion information and object occlusions. In theory, providing multiple observations to the imitation learning agent will lead to better performance. However, surprisingly people find that sometimes imitation from observation histories performs worse than imitation from the most recent observation. In this paper, we explain this phenomenon from the information flow within the neural network perspective. We also propose a novel imitation learning neural network architecture that does not suffer from this issue by design. Furthermore, our method scales to high-dimensional image observations. Finally, we benchmark our approach on two widely used simulators, CARLA and MuJoCo, and it successfully alleviates the copycat problem and surpasses the existing solutions.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 48 references that are in the map. First posted on arXiv 20 Jul 2022.

Builds on

31 of 48 references.

Closest ancestors · 5

Same direction, newest first

  1. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shift4 Feb
  2. 2020Fighting Copycat Agents in Behavioral Cloning from Observation Histories28 Oct
  3. 2019Causal Confusion in Imitation Learning28 May
  4. 2018An Algorithmic Perspective on Imitation Learning16 Nov
  5. 2018Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning29 May

Other references · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  6. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  7. 2016Generative Adversarial Imitation Learningcited by 1,576
  8. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  9. 2016End to End Learning for Self-Driving Carscited by 1,000
  10. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  11. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  12. 2018Recognition in Terra Incognitacited by 655
14 more
  1. 2020Tracking Objects as Pointscited by 483
  2. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  3. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  4. 2019Learning by Cheatingcited by 361
  5. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  6. 2019Probing Neural Network Comprehension of Natural Language Argumentscited by 212
  7. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  8. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  9. 2021Learning to drive from a world on railscited by 114
  10. 2017Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Predictioncited by 93
  11. 2019Deep Drone Racing: From Simulation to Reality with Domain Randomizationcited by 63
  12. 2019Monocular Plan View Networks for Autonomous Drivingcited by 37
  13. 2021Keyframe-Focused Visual Imitation Learningcited by 25
  14. 2022Fighting Fire with Fire: Avoiding DNN Shortcuts through Primingcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 3; 2025: 4; 2026: 1.

14 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  3. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 142
  4. 2023A Survey on Causal Reinforcement Learningcited by 43
  5. 2022Sequence Model Imitation Learning with Unobserved Contextscited by 34
  6. 2024Can Transformers Capture Spatial Relations between Objects?cited by 11
  7. 2024Exploring the Causality of End-to-End Autonomous Drivingcited by 7
  8. 2023Addressing Limitations of State-Aware Imitation Learning for Autonomous Drivingcited by 3
  9. 2025GABRIL: Gaze-Based Regularization for Mitigating Causal Confusion in Imitation Learningcited by 3
  10. 2025Improving Robustness to Out-of-Distribution States in Imitation Learning via Deep Koopman-Boosted Diffusion Policycited by 1
  11. 2025A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approachcited by 1
  12. 2024AI-Guided Molecular Simulations in VR: Exploring Strategies for Imitation Learning in Hyperdimensional Molecular Systemscited by 0
2 more
  1. 2025Exposing the Copycat Problem of Imitation-based Planner: A Novel Closed-Loop Simulator, Causal Benchmark and Joint IL-RL Baselinecited by 0
  2. 2026Conditional Residual Prediction: Improving Autoregressive Video Diffusion without a Bidirectional Teachercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Adversarial Imitation Learning from Visual Observations using Latent Information
  2. 2021The Surprising Effectiveness of Representation Learning for Visual Imitation
  3. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  4. 2023Robust Visual Imitation Learning with Inverse Dynamics Representations
  5. 2019Imitation Learning from Video by Leveraging Proprioception
  6. 2023Memory-Consistent Neural Networks for Imitation Learning
  7. 2019Model-based Behavioral Cloning with Future Image Similarity Learning
  8. 2019Recent Advances in Imitation Learning from Observation