Learning of feature points without additional supervision improves reinforcement learning from images
Not on the map. Why?
Abstract (arXiv)
In many control problems that include vision, optimal controls can be inferred from the location of the objects in the scene. This information can be represented using feature points, which is a list of spatial locations in learned feature maps of an input image. Previous works show that feature points learned using unsupervised pre-training or human supervision can provide good features for control tasks. In this paper, we show that it is possible to learn efficient feature point representations end-to-end, without the need for unsupervised pre-training, decoders, or additional losses. Our proposed architecture consists of a differentiable feature point extractor that feeds the coordinates of the estimated feature points directly to a soft actor-critic agent. The proposed algorithm yields performance competitive to the state-of-the art on DeepMind Control Suite tasks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 33 references that are in the map. First posted on arXiv 15 Jun 2021.
Builds on
27 of 33 references.
References · 27
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
- 2015End-to-End Training of Deep Visuomotor Policiescited by 847
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
- 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
- 2020Reinforcement Learning with Augmented Datacited by 452
- 2020dm_control: Software and Tasks for Continuous Controlcited by 368
- 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
- 2018Learning to Drive in a Daycited by 266
15 more
- 2018Visual Reinforcement Learning with Imagined Goalscited by 248
- 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
- 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 149
- 2017DARLA: Improving Zero-Shot Transfer in Reinforcement Learningcited by 147
- 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 115
- 2018Unsupervised Learning of Object Landmarks through Conditional Image Generationcited by 113
- 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
- 2019Unsupervised Learning of Object Keypoints for Perception and Controlcited by 109
- 2019Unsupervised Learning of Object Structure and Dynamics from Videoscited by 92
- 2018Unsupervised Discovery of Object Landmarks as Structural Representationscited by 73
- 2016Loss is its own Reward: Self-Supervision for Reinforcement Learningcited by 72
- 2020Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learningcited by 69
- 2017Unsupervised learning of object landmarks by factorized spatial embeddingscited by 64
- 2019Unsupervised Learning of Landmarks by Descriptor Vector Exchangecited by 39
- 2020Unsupervised Object Keypoint Learning using Local Spatial Predictabilitycited by 13
Built on by
Citing papers per year: 2023: 1.
1 paper.