mlsanityData to 14 Oct 2026

RePreM: Representation Pre-training with Masked Model for Reinforcement Learning

arXiv 2303.01668 ↗First posted 3 Mar 2023cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Inspired by the recent success of sequence modeling in RL and the use of masked language model for pre-training, we propose a masked model for pre-training in RL, RePreM (Representation Pre-training with Masked Model), which trains the encoder combined with transformer blocks to predict the masked states or actions in a trajectory. RePreM is simple but effective compared to existing representation pre-training methods in RL. It avoids algorithmic sophistication (such as data augmentation or estimating multiple models) with sequence modeling and generates a representation that captures long-term dynamics well. Empirically, we demonstrate the effectiveness of RePreM in various tasks, including dynamic prediction, transfer learning, and sample-efficient RL with both value-based and actor-critic methods. Moreover, we show that RePreM scales well with dataset size, dataset quality, and the scale of the encoder, which indicates its potential towards big RL models.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 56 of its 64 references that are in the map. First posted on arXiv 3 Mar 2023.

Builds on

56 of 64 references.

References · 56

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  12. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
38 more
  1. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  3. 2020Mastering Atari with Discrete World Modelscited by 1,007
  4. 2018DeepMind Control Suitecited by 956
  5. 2022A Generalist Agentcited by 808
  6. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  8. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  9. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  10. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  11. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  12. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  13. 2019Model-Based Reinforcement Learning for Ataricited by 502
  14. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  15. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  16. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 347
  17. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  18. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
  19. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  20. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  21. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  22. 2022Multi-Game Decision Transformerscited by 214
  23. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  24. 2020Representation Learning via Invariant Causal Mechanismscited by 199
  25. 2022Online Decision Transformercited by 196
  26. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 185
  27. 2018A Study on Overfitting in Deep Reinforcement Learningcited by 174
  28. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  29. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 146
  30. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  31. 2019When to use parametric models in reinforcement learning?cited by 118
  32. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
  33. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 106
  34. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  35. 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 102
  36. 2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 91
  37. 2018The Laplacian in RL: Learning Representations with Efficient Approximationscited by 86
  38. 2016Loss is its own Reward: Self-Supervision for Reinforcement Learningcited by 72

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 1; 2026: 1.

5 papers.

  1. 2024Position: Foundation Agents as the Paradigm Shift for Decision Makingcited by 9
  2. 2023Reward Certification for Policy Smoothed Reinforcement Learningcited by 7
  3. 2023Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challengescited by 2
  4. 2025Reward-Driven Interaction: Enhancing Proactive Dialogue Agents through User Satisfaction Predictioncited by 0
  5. 2026Improving and Accelerating Offline RL in Large Discrete Action Spaces with Structured Policy Initializationcited by 0