Accelerating Representation Learning with View-Consistent Dynamics in Data-Efficient Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Learning informative representations from image-based observations is of fundamental concern in deep Reinforcement Learning (RL). However, data-inefficiency remains a significant barrier to this objective. To overcome this obstacle, we propose to accelerate state representation learning by enforcing view-consistency on the dynamics. Firstly, we introduce a formalism of Multi-view Markov Decision Process (MMDP) that incorporates multiple views of the state. Following the structure of MMDP, our method, View-Consistent Dynamics (VCD), learns state representations by training a view-consistent dynamics model in the latent space, where views are generated by applying data augmentation to states. Empirical evaluation on DeepMind Control Suite and Atari-100k demonstrates VCD to be the SoTA data-efficient algorithm on visual control tasks.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 25 of its 31 references that are in the map. First posted on arXiv 18 Jan 2022.
Builds on
25 of 31 references.
References · 25
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
- 2018DeepMind Control Suitecited by 956
- 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
- 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
- 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
- 2020Reinforcement Learning with Augmented Datacited by 468
13 more
- 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
- 2020Debiased Contrastive Learningcited by 356
- 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
- 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 311
- 2019Challenges of Real-World Reinforcement Learningcited by 291
- 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
- 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
- 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
- 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
- 2019Unsupervised State Representation Learning in Ataricited by 144
- 2017Building machines that adapt and compute like brainscited by 99
- 2020Predictive Information Accelerates Learning in RLcited by 56
- 2021PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learningcited by 41
Built on by
Citing papers per year: 2022: 1; 2023: 1; 2024: 1.
3 papers.