mlsanityData to 7 Oct 2026

Visual Imitation Learning with Calibrated Contrastive Representation

arXiv 2401.11396 ↗First posted 21 Jan 2024cs.LG cs.CVcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Adversarial Imitation Learning (AIL) allows the agent to reproduce expert behavior with low-dimensional states and actions. However, challenges arise in handling visual states due to their less distinguishable representation compared to low-dimensional proprioceptive features. While existing methods resort to adopt complex network architectures or separate the process of learning representation and decision-making, they overlook valuable intra-agent information within demonstrations. To address this problem, this paper proposes a simple and effective solution by incorporating calibrated contrastive representative learning into visual AIL framework. Specifically, we present an image encoder in visual AIL, utilizing a combination of unsupervised and supervised contrastive learning to extract valuable features from visual states. Based on the fact that the improved agent often produces demonstrations of varying quality, we propose to calibrate the contrastive loss by treating each agent demonstrations as a mixed sample. The incorporation of contrastive learning can be jointly optimized with the AIL framework, without modifying the architecture or incurring significant computational costs. Experimental results on DMControl Suite demonstrate our proposed method is sample efficient and can outperform other compared methods from different aspects.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 34 of its 49 references that are in the map. First posted on arXiv 21 Jan 2024.

Builds on

34 of 49 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023Policy Contrastive Imitation Learning6 Jul
  2. 2023Unlabeled Imperfect Demonstrations in Adversarial Imitation Learning13 Feb
  3. 2023Visual Imitation Learning with Patch Rewards2 Feb
  4. 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transport30 Jun
  5. 2021Visual Adversarial Imitation Learning using Variational Models16 Jul
  6. 2021Imitation Learning: Progress, Taxonomies and Challenges23 Jun
  7. 2021On the Sample Complexity of Stability Constrained Imitation Learning18 Feb
  8. 2020Error Bounds of Imitating Policies and Environments22 Oct
  9. 2019Imitation Learning from Pixel-Level Demonstrations by HashReward9 Sep
  10. 2019Imitation Learning as f-Divergence Minimization30 May
  11. 2018Behavioral Cloning from Observation4 May

Other references · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  6. 2020Supervised Contrastive Learningcited by 2,803
  7. 2015Trust Region Policy Optimizationcited by 2,503
  8. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  9. 2016Generative Adversarial Imitation Learningcited by 1,576
  10. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  11. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  12. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
11 more
  1. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  2. 2020Reinforcement Learning with Augmented Datacited by 452
  3. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  4. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  5. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  6. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  7. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  8. 2021Reinforcement Learning with Prototypical Representationscited by 179
  9. 2021Training GANs with Stronger Augmentations via Contrastive Discriminatorcited by 45
  10. 2018Inverse reinforcement learning for video gamescited by 26
  11. 2021Keyframe-Focused Visual Imitation Learningcited by 25

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Provable Representation Learning for Imitation with Contrastive Fourier Features
  2. 2023Adversarial Imitation Learning from Visual Observations using Latent Information
  3. 2018Generative Adversarial Imitation from Observation
  4. 2018Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning
  5. 2021Domain-Robust Visual Imitation Learning with Mutual Information Constraints
  6. 2021The Surprising Effectiveness of Representation Learning for Visual Imitation
  7. 2022Adversarial Imitation Learning from Video using a State Observer
  8. 2023Robust Visual Imitation Learning with Inverse Dynamics Representations