mlsanityData to 7 Oct 2026

Evaluating Vision Transformer Methods for Deep Reinforcement Learning from Pixels

arXiv 2204.04905 ↗First posted 11 Apr 2022cs.LG cs.CV cs.ROcited by 12

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViT) have recently demonstrated the significant potential of transformer architectures for computer vision. To what extent can image-based deep reinforcement learning also benefit from ViT architectures, as compared to standard convolutional neural network (CNN) architectures? To answer this question, we evaluate ViT training methods for image-based reinforcement learning (RL) control tasks and compare these results to a leading convolutional-network architecture method, RAD. For training the ViT encoder, we consider several recently-proposed self-supervised losses that are treated as auxiliary tasks, as well as a baseline with no additional loss terms. We find that the CNN architectures trained using RAD still generally provide superior performance. For the ViT methods, all three types of auxiliary tasks that we consider provide a benefit over plain ViT training. Furthermore, ViT reconstruction-based tasks are found to significantly outperform ViT contrastive-learning.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 40 of its 48 references that are in the map. First posted on arXiv 11 Apr 2022.

Builds on

40 of 48 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2015Continuous control with deep reinforcement learningcited by 3,279
  9. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  12. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
28 more
  1. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  2. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  4. 2017Random Erasing Data Augmentationcited by 1,235
  5. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  7. 2020Mastering Atari with Discrete World Modelscited by 990
  8. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  9. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  10. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  11. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  12. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  13. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  14. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  15. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  16. 2020Reinforcement Learning with Augmented Datacited by 452
  17. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  18. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  19. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  20. 2018Learning to Drive in a Daycited by 266
  21. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  22. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  23. 2020SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learningcited by 150
  24. 2017DARLA: Improving Zero-Shot Transfer in Reinforcement Learningcited by 147
  25. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  26. 2019Urban Driving with Conditional Imitation Learningcited by 84
  27. 2020Predictive Information Accelerates Learning in RLcited by 56
  28. 2021Sample-efficient Reinforcement Learning Representation Learning with Curiosity Contrastive Forward Dynamics Modelcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 4; 2024: 3; 2025: 0; 2026: 1.

12 papers.

  1. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  2. 2022A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyondcited by 58
  3. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  4. 2023Transformers in Reinforcement Learning: A Surveycited by 11
  5. 2024Vision Transformers for End-to-End Vision-Based Quadrotor Obstacle Avoidancecited by 10
  6. 2023MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learningcited by 9
  7. 2022Transformer in Transformer as Backbone for Deep Reinforcement Learningcited by 6
  8. 2022Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learningcited by 5
  9. 2022Deep Reinforcement Learning with Swin Transformerscited by 2
  10. 2024Empowering Clinicians with Medical Decision Transformers: A Framework for Sepsis Treatmentcited by 2
  11. 2024Unsupervised Salient Patch Selection for Data-Efficient Reinforcement Learningcited by 0
  12. 2026MILER: Semantic Mid-Level Representation for Sim-to-Real Reinforcement Learning in Unstructured Autonomous Drivingcited by 0