Pretrained Encoders are All You Need
Not on the map. Why?
Abstract (arXiv)
Data-efficiency and generalization are key challenges in deep learning and deep reinforcement learning as many models are trained on large-scale, domain-specific, and expensive-to-label datasets. Self-supervised models trained on large-scale uncurated datasets have shown successful transfer to diverse settings. We investigate using pretrained image representations and spatio-temporal attention for state representation learning in Atari. We also explore fine-tuning pretrained representations with self-supervised techniques, i.e., contrastive predictive coding, spatio-temporal contrastive learning, and augmentations. Our results show that pretrained representations are at par with state-of-the-art self-supervised methods trained on domain-specific data. Pretrained representations, thus, yield data and compute-efficient state representations. https://github.com/PAL-ML/PEARL_v1
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 60 of its 71 references that are in the map. First posted on arXiv 9 Jun 2021.
Builds on
60 of 71 references.
References · 60
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2015Domain-Adversarial Training of Neural Networkscited by 3,023
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2018World Modelscited by 1,444
- 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
38 more
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
- 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
- 2016Understanding intermediate layers using linear classifier probescited by 1,279
- 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
- 2018DeepMind Control Suitecited by 935
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
- 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
- 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
- 2018Learning Dexterous In-Hand Manipulationcited by 643
- 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
- 2019Model-Based Reinforcement Learning for Ataricited by 495
- 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
- 2020Reinforcement Learning with Augmented Datacited by 452
- 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
- 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
- 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
- 2020Planning to Explore via Self-Supervised World Modelscited by 365
- 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
- 2016Domain Separation Networkscited by 317
- 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
- 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
- 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
- 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
- 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
- 2017DARLA: Improving Zero-Shot Transfer in Reinforcement Learningcited by 147
- 2019Unsupervised State Representation Learning in Ataricited by 144
- 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
- 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
- 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 101
- 2017Building machines that adapt and compute like brainscited by 99
- 2018Playing hard exploration games by watching YouTubecited by 98
- 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 89
- 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
- 2016Loss is its own Reward: Self-Supervision for Reinforcement Learningcited by 72
- 2021Return-Based Contrastive Representation Learning for Reinforcement Learningcited by 36
- 2018AGIL: Learning Attention from Human for Visuomotor Taskscited by 32
50 most cited of 60 All 60 (JSON)
Built on by
Citing papers per year: 2022: 1; 2023: 1.
2 papers.