mlsanityData to 7 Oct 2026

Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learning

arXiv 2406.06037 ↗First posted 10 Jun 2024cs.LG cs.AI cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Recently, various pre-training methods have been introduced in vision-based Reinforcement Learning (RL). However, their generalization ability remains unclear due to evaluations being limited to in-distribution environments and non-unified experimental setups. To address this, we introduce the Atari Pre-training Benchmark (Atari-PB), which pre-trains a ResNet-50 model on 10 million transitions from 50 Atari games and evaluates it across diverse environment distributions. Our experiments show that pre-training objectives focused on learning task-agnostic features (e.g., identifying objects and understanding temporal dynamics) enhance generalization across different environments. In contrast, objectives focused on learning task-specific knowledge (e.g., identifying agents and fitting reward functions) improve performance in environments similar to the pre-training dataset but not in varied ones. We publicize our codes, datasets, and model checkpoints at https://github.com/dojeon-ai/Atari-PB.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 82 of its 95 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

82 of 95 references.

References · 82

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  9. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  10. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  11. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  12. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
38 more
  1. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  4. 2018Group Normalizationcited by 1,514
  5. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  8. 2023Mastering Diverse Domains through World Modelscited by 1,193
  9. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  10. 2018DeepMind Control Suitecited by 935
  11. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  12. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  13. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  14. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  15. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  16. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  17. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  18. 2019Model-Based Reinforcement Learning for Ataricited by 495
  19. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  20. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  21. 2020Reinforcement Learning with Augmented Datacited by 452
  22. 2021Early Convolutions Help Transformers See Bettercited by 395
  23. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  24. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  25. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  26. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  27. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  28. 2022Masked Visual Pre-training for Motor Controlcited by 246
  29. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 230
  30. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 215
  31. 2022Multi-Game Decision Transformerscited by 212
  32. 2020Eigen-CAM: Class Activation Map using Principal Componentscited by 188
  33. 2019Video Representation Learning by Dense Predictive Codingcited by 175
  34. 2022Masked World Models for Visual Controlcited by 169
  35. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  36. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  37. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  38. 2023Structured World Models from Human Videoscited by 144

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  2. 2024PreND: Enhancing Intrinsic Motivation in Reinforcement Learning through Pre-trained Network Distillationcited by 1
  3. 2025Recurrent Off-Policy Deep Reinforcement Learning Doesn't Have to be Slowcited by 0
  4. 2026Mitigating State Aliasing in Vision-Language-Action Models via Inverse Dynamics Learningcited by 0