mlsanityData to 7 Oct 2026

Data-Efficient Reinforcement Learning with Self-Predictive Representations

arXiv 2007.05929 ↗First posted 12 Jul 2020cs.LG stat.MLcited by 309

Not on the map. Why?

Cited by 45 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

While deep reinforcement learning excels at solving tasks where large amounts of data can be collected through virtually unlimited interaction with the environment, learning from limited interaction remains a key challenge. We posit that an agent can learn more efficiently if we augment reward maximization with self-supervised objectives based on structure in its visual input and sequential interaction with the environment. Our method, Self-Predictive Representations(SPR), trains an agent to predict its own latent state representations multiple steps into the future. We compute target representations for future states using an encoder which is an exponential moving average of the agent's parameters and we make predictions using a learned transition model. On its own, this future prediction objective outperforms prior methods for sample-efficient deep RL from pixels. We further improve performance by adding data augmentation to the future prediction loss, which forces the agent's representations to be consistent across multiple views of an observation. Our full self-supervised objective, which combines future prediction and data augmentation, achieves a median human-normalized score of 0.415 on Atari in a setting limited to 100k steps of environment interaction, which represents a 55% relative improvement over the previous state-of-the-art. Notably, even in this limited data regime, SPR exceeds expert human scores on 7 out of 26 games. The code associated with this work is available at https://github.com/mila-iqia/spr

Key facts. Cited by 309 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 31; 2022: 64; 2023: 55; 2024: 60; 2025: 45; 2026: 54.

309 papers.

Sort:

By area:

  1. 2023Mastering Diverse Domains through World Modelscited by 1,193
  2. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  3. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  4. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  5. 2022Transformers are Sample-Efficient World Modelscited by 317
  6. 2023A Cookbook of Self-Supervised Learningcited by 248
  7. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  8. 2021Mastering Atari Games with Limited Datacited by 235
  9. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  10. 2022Masked World Models for Visual Controlcited by 169
  11. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  12. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
38 more
  1. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  2. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 137
  3. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  4. 2021APS: Active Pretraining with Successor Featurescited by 109
  5. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  6. 2021B-Pref: Benchmarking Preference-Based Reinforcement Learningcited by 106
  7. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  8. 2025FLARE: Robot Learning with Implicit World Modelingcited by 105
  9. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 104
  10. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  11. 2025Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environmentscited by 93
  12. 2023STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learningcited by 88
  13. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  14. 2022Offline Visual Representation Learning for Embodied Navigationcited by 74
  15. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representationscited by 73
  16. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
  17. 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 71
  18. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 71
  19. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 70
  20. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
  21. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
  22. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 56
  23. 2022On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baselinecited by 55
  24. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Makingcited by 53
  25. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
  26. 2022Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approachcited by 51
  27. 2021Towards More Generalizable One-shot Visual Imitation Learningcited by 48
  28. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 46
  29. 2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
  30. 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45
  31. 2023DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimizationcited by 45
  32. 2022Masked Autoencoding for Scalable and Generalizable Decision Makingcited by 44
  33. 2021Curious Representation Learning for Embodied Intelligencecited by 43
  34. 2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Controlcited by 43
  35. 2022Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Modelscited by 42
  36. 2023PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learningcited by 42
  37. 2021PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learningcited by 41
  38. 2023Proto-Value Networks: Scaling Representation Learning with Auxiliary Taskscited by 41

50 most cited of 309 All 309 (JSON)