mlsanityData to 7 Oct 2026

Unsupervised Salient Patch Selection for Data-Efficient Reinforcement Learning

arXiv 2402.03329 ↗First posted 10 Jan 2024cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

To improve the sample efficiency of vision-based deep reinforcement learning (RL), we propose a novel method, called SPIRL, to automatically extract important patches from input images. Following Masked Auto-Encoders, SPIRL is based on Vision Transformer models pre-trained in a self-supervised fashion to reconstruct images from randomly-sampled patches. These pre-trained models can then be exploited to detect and select salient patches, defined as hard to reconstruct from neighboring patches. In RL, the SPIRL agent processes selected salient patches via an attention module. We empirically validate SPIRL on Atari games to test its data-efficiency against relevant state-of-the-art methods, including some traditional model-based methods and keypoint-based models. In addition, we analyze our model's interpretability capabilities.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 44 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

36 of 44 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2016Layer Normalizationcited by 5,579
  6. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  7. 2015Continuous control with deep reinforcement learningcited by 3,279
  8. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  9. 2015Prioritized Experience Replaycited by 1,070
  10. 2020On Layer Normalization in the Transformer Architecturecited by 906
  11. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  12. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
24 more
  1. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  2. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  3. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  4. 2019Model-Based Reinforcement Learning for Ataricited by 495
  5. 2020Reinforcement Learning with Augmented Datacited by 452
  6. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  7. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  8. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  9. 2017Noisy Networks for Explorationcited by 262
  10. 2022Masked World Models for Visual Controlcited by 169
  11. 2017Parameter Space Noise for Explorationcited by 158
  12. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  13. 2019When to use parametric models in reinforcement learning?cited by 118
  14. 2018Unsupervised Learning of Object Landmarks through Conditional Image Generationcited by 113
  15. 2019Unsupervised Learning of Object Keypoints for Perception and Controlcited by 109
  16. 2019Unsupervised Learning of Object Structure and Dynamics from Videoscited by 92
  17. 2020Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learningcited by 69
  18. 2020Self-supervised Visual Reinforcement Learning with Object-centric Representationscited by 43
  19. 2018An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agentscited by 15
  20. 2020Unsupervised Object Keypoint Learning using Local Spatial Predictabilitycited by 13
  21. 2022Evaluating Vision Transformer Methods for Deep Reinforcement Learning from Pixelscited by 12
  22. 2018Unsupervised Video Object Segmentation for Deep Reinforcement Learningcited by 11
  23. 2022Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learningcited by 5
  24. 2022Improving Sample Efficiency of Value Based Models Using Attention and Vision Transformerscited by 4

Built on by

Not cited yet.