Discovering Object-Centric Generalized Value Functions From Pixels
Not on the map. Why?
Abstract (arXiv)
Deep Reinforcement Learning has shown significant progress in extracting useful representations from high-dimensional inputs albeit using hand-crafted auxiliary tasks and pseudo rewards. Automatically learning such representations in an object-centric manner geared towards control and fast adaptation remains an open research problem. In this paper, we introduce a method that tries to discover meaningful features from objects, translating them to temporally coherent "question" functions and leveraging the subsequent learned general value functions for control. We compare our approach with state-of-the-art techniques alongside other ablations and show competitive performance in both stationary and non-stationary settings. Finally, we also investigate the discovered general value functions and through qualitative analysis show that the learned representations are not only interpretable but also, centered around objects that are invariant to changes across tasks facilitating fast adaptation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 27 Apr 2023.
Builds on
26 of 36 references.
References · 26
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2016Layer Normalizationcited by 5,669
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
- 2020Object-Centric Learning with Slot Attentioncited by 876
- 2015End-to-End Training of Deep Visuomotor Policiescited by 875
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
- 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
- 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
- 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
- 2017Population Based Training of Neural Networkscited by 367
- 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
14 more
- 2018Visual Reinforcement Learning with Imagined Goalscited by 251
- 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
- 2019Entity Abstraction in Visual Model-Based Reinforcement Learningcited by 110
- 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
- 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 102
- 2016Loss is its own Reward: Self-Supervision for Reinforcement Learningcited by 72
- 2019COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Explorationcited by 57
- 2019Discovery of Useful Questions as Auxiliary Taskscited by 45
- 2020Self-supervised Visual Reinforcement Learning with Object-centric Representationscited by 44
- 2020Planning from Pixels using Inverse Dynamics Modelscited by 36
- 2019Learning to Reach Goals via Iterated Supervised Learningcited by 26
- 2019A Perspective on Objects and Systematic Generalization in Model-Based RLcited by 14
- 2020Contrastive Explanations for Reinforcement Learning via Embedded Self Predictionscited by 13
- 2022What Should I Know? Using Meta-gradient Descent for Predictive Feature Discovery in a Single Stream of Experiencecited by 1
Built on by
Citing papers per year: 2023: 1.
1 paper.