mlsanityData to 7 Oct 2026

BYOL-Explore: Exploration by Bootstrapped Prediction

arXiv 2206.08332 ↗First posted 16 Jun 2022cs.LG cs.AI stat.MLcited by 71

Not on the map. Why?

Abstract (arXiv)

We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually-complex environments. BYOL-Explore learns a world representation, the world dynamics, and an exploration policy all-together by optimizing a single prediction loss in the latent space with no additional auxiliary objective. We show that BYOL-Explore is effective in DM-HARD-8, a challenging partially-observable continuous-action hard-exploration benchmark with visually-rich 3-D environments. On this benchmark, we solve the majority of the tasks purely through augmenting the extrinsic reward with BYOL-Explore s intrinsic reward, whereas prior work could only get off the ground with human demonstrations. As further evidence of the generality of BYOL-Explore, we show that it achieves superhuman performance on the ten hardest exploration games in Atari while having a much simpler design than other competitive agents.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 63 of its 83 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

63 of 83 references.

References · 63

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  3. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  6. 2018Group Normalizationcited by 1,514
  7. 2018World Modelscited by 1,444
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  9. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  10. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  11. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  12. 2017Hindsight Experience Replaycited by 1,096
38 more
  1. 2018Exploration by Random Network Distillationcited by 966
  2. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  3. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  4. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  5. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  6. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  7. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  8. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  9. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  10. 2019Self-Supervised Exploration via Disagreementcited by 263
  11. 2017Noisy Networks for Explorationcited by 262
  12. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
  13. 2020First return, then explorecited by 256
  14. 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 254
  15. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  16. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  17. 2018Provably Efficient Maximum Entropy Explorationcited by 240
  18. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
  19. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  20. 2020The NetHack Learning Environmentcited by 196
  21. 2021Large-Scale Representation Learning on Graphs via Bootstrappingcited by 184
  22. 2020Reward-Free Exploration for Reinforcement Learningcited by 177
  23. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 171
  24. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  25. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 138
  26. 2021Broaden Your Views for Self-Supervised Video Learningcited by 108
  27. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  28. 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 101
  29. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 100
  30. 2020BYOL works even without batch statisticscited by 95
  31. 2020Fast active learning for pure exploration in reinforcement learningcited by 92
  32. 2020On Reward-Free Reinforcement Learning with Linear Function Approximationcited by 89
  33. 2020Automatic Curriculum Learning through Value Disagreementcited by 86
  34. 2019V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Controlcited by 85
  35. 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 78
  36. 2020Adaptive Reward-Free Explorationcited by 77
  37. 2019Shaping Belief States with Generative Environment Models for RLcited by 65
  38. 2019Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discoverycited by 65

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 20; 2024: 20; 2025: 14; 2026: 9.

71 papers.

Sort:

By area:

  1. 2023A Cookbook of Self-Supervised Learningcited by 248
  2. 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 158
  3. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 76
  4. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 56
  5. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
  6. 2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Controlcited by 43
  7. 2022Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Modelscited by 42
  8. 2022Understanding Self-Predictive Learning for Reinforcement Learningcited by 40
  9. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 40
  10. 2025What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?cited by 37
  11. 2023Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learningcited by 28
  12. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
38 more
  1. 2023Gradient-based Planning with World Modelscited by 18
  2. 2025TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learningcited by 17
  3. 2023Curious Replay for Model-based Adaptationcited by 16
  4. 2022Agent-Controller Representations: Principled Offline RL with Rich Exogenous Informationcited by 15
  5. 2024Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inferencecited by 15
  6. 2024Leveraging Skills from Unlabeled Prior Data for Efficient Online Explorationcited by 13
  7. 2023Diffused Task-Agnostic Milestone Plannercited by 12
  8. 2024Towards Principled Representation Learning from Videos for Reinforcement Learningcited by 10
  9. 2024RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learningcited by 10
  10. 2023Unlocking the Power of Representations in Long-term Novelty-based Explorationcited by 8
  11. 2023Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learningcited by 8
  12. 2023Provably Efficient Representation Learning with Tractable Planning in Low-Rank POMDPcited by 8
  13. 2023Accelerating exploration and representation learning with offline pre-trainingcited by 7
  14. 2024A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learningcited by 7
  15. 2024Can foundation models actively gather information in interactive environments to test hypotheses?cited by 7
  16. 2025seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Modelscited by 7
  17. 2023MIMEx: Intrinsic Rewards from Masked Input Modelingcited by 6
  18. 2023Discovering Hierarchical Achievements in Reinforcement Learning via Contrastive Learningcited by 6
  19. 2024Learning Cognitive Maps from Transformer Representations for Efficient Planning in Partially Observed Environmentscited by 6
  20. 2023Investigating the role of model-based learning in exploration and transfercited by 5
  21. 2023Bridging Imitation and Online Reinforcement Learning: An Optimistic Talecited by 5
  22. 2023Active Sensing with Predictive Coding and Uncertainty Minimizationcited by 5
  23. 2024DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractorscited by 5
  24. 2025Contrastive Representations for Temporal Reasoningcited by 5
  25. 2026Biased Dreams: Limitations to Epistemic Uncertainty Quantification in Latent Dynamics Modelscited by 5
  26. 2022Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environmentscited by 4
  27. 2024Deep Exploration with PAC-Bayescited by 4
  28. 2023Intrinsically motivated graph exploration using network theories of human curiositycited by 3
  29. 2024Just Cluster It: An Approach for Exploration in High-Dimensions using Clustering and Pre-Trained Representationscited by 3
  30. 2024An Empirical Study on the Power of Future Prediction in Partially Observable Environmentscited by 3
  31. 2025β-DQN: Improving Deep Q-Learning By Evolving the Behaviorcited by 3
  32. 2025Contextual Similarity Distillation: Ensemble Uncertainties with a Single Modelcited by 3
  33. 2025LayerLock: Non-collapsing Representation Learning with Progressive Freezingcited by 3
  34. 2023Self-supervised network distillation: an effective approach to exploration in sparse reward environmentscited by 2
  35. 2023Representations and Exploration for Deep Reinforcement Learning using Singular Value Decompositioncited by 2
  36. 2024Generalizing Multi-Step Inverse Models for Representation Learning to Finite-Memory POMDPscited by 2
  37. 2024Adaptive Exploration for Data-Efficient General Value Function Evaluationscited by 2
  38. 2024Exploration by Learning Diverse Skills through Successor State Measurescited by 2

50 most cited of 71 All 71 (JSON)