mlsanityData to 7 Oct 2026

CRC-RL: A Novel Visual Feature Representation Architecture for Unsupervised Reinforcement Learning

arXiv 2301.13473 ↗First posted 31 Jan 2023cs.CV cs.AI cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper addresses the problem of visual feature representation learning with an aim to improve the performance of end-to-end reinforcement learning (RL) models. Specifically, a novel architecture is proposed that uses a heterogeneous loss function, called CRC loss, to learn improved visual features which can then be used for policy learning in RL. The CRC-loss function is a combination of three individual loss functions, namely, contrastive, reconstruction and consistency loss. The feature representation is learned in parallel to the policy learning while sharing the weight updates through a Siamese Twin encoder model. This encoder model is augmented with a decoder network and a feature projection network to facilitate computation of the above loss components. Through empirical analysis involving latent feature visualization, an attempt is made to provide an insight into the role played by this loss function in learning new action-dependent features and how they are linked to the complexity of the problems being solved. The proposed architecture, called CRC-RL, is shown to outperform the existing state-of-the-art methods on the challenging Deep mind control suite environments by a significant margin thereby creating a new benchmark in this field.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 31 Jan 2023.

Builds on

43 of 58 references.

References · 43

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  8. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  9. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  10. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  11. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  12. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
31 more
  1. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  2. 2018DeepMind Control Suitecited by 935
  3. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  4. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  5. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  7. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  8. 2020Reinforcement Learning with Augmented Datacited by 452
  9. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  10. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  11. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  12. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  13. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  14. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  15. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 259
  16. 2019Revisiting Self-Supervised Visual Representation Learningcited by 250
  17. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 147
  18. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 132
  19. 2019From Variational to Deterministic Autoencoderscited by 131
  20. 2018Visual Semantic Navigation using Scene Priorscited by 127
  21. 2019Observational Overfitting in Reinforcement Learningcited by 90
  22. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  23. 2018Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methodscited by 32
  24. 2017CM-GANs: Cross-modal Generative Adversarial Networks for Common Representation Learningcited by 27
  25. 2020Robotic Grasping using Deep Reinforcement Learningcited by 27
  26. 2021Reinforcement Learning with Latent Flowcited by 18
  27. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  28. 2016MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classificationcited by 16
  29. 2018Intrinsically motivated reinforcement learning for human-robot interaction in the real-worldcited by 12
  30. 2018S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learningcited by 6
  31. 2018A Review of Meta-Reinforcement Learning for Deep Neural Networks Architecture Searchcited by 4

Built on by

Not cited yet.