mlsanityData to 7 Oct 2026

An Empirical Investigation of Representation Learning for Imitation

arXiv 2205.07886 ↗First posted 16 May 2022cs.LG cs.AIcited by 15

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning often needs a large demonstration set in order to handle the full range of situations that an agent might find itself in during deployment. However, collecting expert demonstrations can be expensive. Recent work in vision, reinforcement learning, and NLP has shown that auxiliary representation learning objectives can reduce the need for large amounts of expensive, task-specific data. Our Empirical Investigation of Representation Learning for Imitation (EIRLI) investigates whether similar benefits apply to imitation learning. We propose a modular framework for constructing representation learning algorithms, then use our framework to evaluate the utility of representation learning for imitation across several environment suites. In the settings we evaluate, we find that existing algorithms for image-based representation learning provide limited value relative to a well-tuned baseline with image augmentations. To explain this result, we investigate differences between imitation learning and other settings where representation learning has provided significant benefit, such as image classification. Finally, we release a well-documented codebase which both replicates our findings and provides a modular framework for creating new representation learning algorithms out of reusable components.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 45 references that are in the map. First posted on arXiv 16 May 2022.

Builds on

36 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2020The MAGICAL Benchmark for Robust Imitation1 Nov
  2. 2019Task-Relevant Adversarial Imitation Learning2 Oct

Other references · 34

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  8. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  9. 2016Generative Adversarial Imitation Learningcited by 1,576
  10. 2018World Modelscited by 1,444
  11. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  12. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
22 more
  1. 2018DeepMind Control Suitecited by 935
  2. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  3. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  4. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  5. 2019On Variational Bounds of Mutual Informationcited by 550
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  7. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  8. 2020Reinforcement Learning with Augmented Datacited by 452
  9. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  10. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
  11. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  12. 2018Deep Variational Reinforcement Learning for POMDPscited by 122
  13. 2020Reward-rational (implicit) choice: A unifying formalism for reward learningcited by 119
  14. 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 101
  15. 2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 90
  16. 2020Deep Reinforcement and InfoMax Learningcited by 79
  17. 2020The Conditional Entropy Bottleneckcited by 75
  18. 2020Predictive Information Accelerates Learning in RLcited by 56
  19. 2021PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learningcited by 41
  20. 2020Masked Contrastive Representation Learning for Reinforcement Learningcited by 32
  21. 2020CEB Improves Model Robustnesscited by 14
  22. 2019Learning Representations in Reinforcement Learning:An Information Bottleneck Approachcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 6; 2024: 1; 2025: 0; 2026: 2.

15 papers.

Sort:

By area:

  1. 2023Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitationcited by 30
  2. 2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
  3. 2022Planning for Sample Efficient Imitation Learningcited by 20
  4. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  5. 2022Agent-Controller Representations: Principled Offline RL with Rich Exogenous Informationcited by 15
  6. 2022Behavior Prior Representation learning for Offline Reinforcement Learningcited by 12
  7. 2023Aligning Robot and Human Representationscited by 11
  8. 2022PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representationscited by 10
  9. 2023Understanding and Addressing the Pitfalls of Bisimulation-based Representations in Offline Reinforcement Learningcited by 10
  10. 2023Improving Long-Horizon Imitation Through Instruction Predictioncited by 8
  11. 2023Robust Visual Imitation Learning with Inverse Dynamics Representationscited by 8
  12. 2022PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scalecited by 5
3 more
  1. 2023Policy Contrastive Imitation Learningcited by 3
  2. 2026Capturing Visual Environment Structure Correlates with Control Performancecited by 2
  3. 2026Improving Imitation Learning Efficiency for Manipulation through Geometric Prior Pretrainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021The Surprising Effectiveness of Representation Learning for Visual Imitation
  2. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  3. 2021Imitation Learning: Progress, Taxonomies and Challenges
  4. 2020Provable Representation Learning for Imitation Learning via Bi-level Optimization
  5. 2021Provable Representation Learning for Imitation with Contrastive Fourier Features
  6. 2021Playful Interactions for Representation Learning
  7. 2026Difference-Aware Retrieval Policies for Imitation Learning
  8. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges