mlsanityData to 14 Oct 2026

Bigger, Better, Faster: Human-level Atari with human-level efficiency

arXiv 2305.19452 ↗First posted 30 May 2023cs.LG cs.AIcited by 147

Not on the map. Why?

Cited by 102 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce a value-based RL agent, which we call BBF, that achieves super-human performance in the Atari 100K benchmark. BBF relies on scaling the neural networks used for value estimation, as well as a number of other design choices that enable this scaling in a sample-efficient manner. We conduct extensive analyses of these design choices and provide insights for future work. We end with a discussion about updating the goalposts for sample-efficient RL research on the ALE. We make our code and data publicly available at https://github.com/google-research/google-research/tree/master/bigger_better_faster.

Key facts. Cited by 147 papers in the map (data through 14 Oct 2026). Builds on 50 of its 72 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

50 of 72 references.

References · 50

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  8. 2023Mastering Diverse Domains through World Modelscited by 1,253
  9. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  10. 2015Prioritized Experience Replaycited by 1,099
  11. 2020Mastering Atari with Discrete World Modelscited by 1,007
  12. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
38 more
  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  2. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  3. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  4. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  5. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  6. 2019Model-Based Reinforcement Learning for Ataricited by 502
  7. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  8. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  9. 2020Reinforcement Learning with Augmented Datacited by 468
  10. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
  11. 2022Transformers are Sample-Efficient World Modelscited by 328
  12. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  13. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
  14. 2017Noisy Networks for Explorationcited by 266
  15. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  16. 2021Mastering Atari Games with Limited Datacited by 237
  17. 2020Revisiting Fundamentals of Experience Replaycited by 175
  18. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  19. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 139
  20. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  21. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  22. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  23. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 122
  24. 2019When to use parametric models in reinforcement learning?cited by 118
  25. 2021Online and Offline Reinforcement Learning by Planning with a Learned Modelcited by 90
  26. 2020Deep Reinforcement and InfoMax Learningcited by 79
  27. 2020Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Researchcited by 78
  28. 2022Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizescited by 73
  29. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 65
  30. 2021The Difficulty of Passive Learning in Deep Reinforcement Learningcited by 50
  31. 2020D2RL: Deep Dense Architectures in Reinforcement Learningcited by 42
  32. 2023Proto-Value Networks: Scaling Representation Learning with Auxiliary Taskscited by 42
  33. 2021Training Larger Networks for Deep Reinforcement Learningcited by 38
  34. 2022The Phenomenon of Policy Churncited by 28
  35. 2022Human-level Atari 200x fastercited by 28
  36. 2015How to Discount Deep Reinforcement Learning: Towards New Dynamic Strategiescited by 27
  37. 2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
  38. 2021Fast and Data-Efficient Training of Rainbow: an Experimental Study on Ataricited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 15; 2024: 46; 2025: 59; 2026: 27.

147 papers.

Sort:

By area:

  1. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  2. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 148
  3. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 108
  4. 2023STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learningcited by 89
  5. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 78
  6. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 76
  7. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 56
  8. 2024Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learningcited by 50
  9. 2023DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimizationcited by 47
  10. 20251000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilitiescited by 45
  11. 2025FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Controlcited by 44
  12. 2023Maintaining Plasticity in Deep Continual Learningcited by 43
38 more
  1. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 41
  2. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 41
  3. 2024EfficientZero V2: Mastering Discrete and Continuous Control with Limited Datacited by 40
  4. 2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 39
  5. 2025Hyperspherical Normalization for Scalable Deep Reinforcement Learningcited by 38
  6. 2023Small batch deep reinforcement learningcited by 36
  7. 2025EXPO: Stable Reinforcement Learning with Expressive Policiescited by 34
  8. 2024Efficient World Models with Context-Aware Tokenizationcited by 33
  9. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 28
  10. 2024A Study of Plasticity Loss in On-Policy Deep Reinforcement Learningcited by 26
  11. 2024Streaming Deep Reinforcement Learning Finally Workscited by 25
  12. 2024Plasticity Loss in Deep Reinforcement Learning: A Surveycited by 24
  13. 2025Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learnerscited by 23
  14. 2025Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learningcited by 23
  15. 2025Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churncited by 22
  16. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learningcited by 21
  17. 2024Mixture of Experts in a Mixture of RL settingscited by 21
  18. 2024Retrieval-Augmented Decision Transformer: External Memory for In-context RLcited by 21
  19. 2025Learning Transformer-based World Models with Contrastive Predictive Codingcited by 20
  20. 2025Scaling Off-Policy Reinforcement Learning with Batch and Weight Normalizationcited by 19
  21. 2024Neuroplastic Expansion in Deep Reinforcement Learningcited by 18
  22. 2024Deep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Bufferscited by 18
  23. 2024Dissecting Deep RL with High Update Ratios: Combatting Value Divergencecited by 17
  24. 2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 17
  25. 2024MAD-TD: Model-Augmented Data stabilizes High Update Ratio RLcited by 16
  26. 2025Learning Sim-to-Real Humanoid Locomotion in 15 Minutescited by 16
  27. 2025Value-Based Deep RL Scales Predictablycited by 15
  28. 2024Can Learned Optimization Make Reinforcement Learning Less Difficult?cited by 14
  29. 2024Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RLcited by 14
  30. 2024A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Taskscited by 14
  31. 2025Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learningcited by 14
  32. 2025Improving Transformer World Models for Data-Efficient RLcited by 13
  33. 2025The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networkscited by 13
  34. 2024UniZero: Generalized and Efficient Planning with Scalable Latent World Modelscited by 12
  35. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
  36. 2024Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timestepscited by 11
  37. 2025Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learningcited by 11
  38. 2023Replay across Experiments: A Natural Extension of Off-Policy RLcited by 10

50 most cited of 147 All 147 (JSON)