mlsanityData to 7 Oct 2026

Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark

arXiv 2103.15332 ↗First posted 29 Mar 2021cs.LG cs.AIcited by 23

Not on the map. Why?

Abstract (arXiv)

The NeurIPS 2020 Procgen Competition was designed as a centralized benchmark with clearly defined tasks for measuring Sample Efficiency and Generalization in Reinforcement Learning. Generalization remains one of the most fundamental challenges in deep reinforcement learning, and yet we do not have enough benchmarks to measure the progress of the community on Generalization in Reinforcement Learning. We present the design of a centralized benchmark for Reinforcement Learning which can help measure Sample Efficiency and Generalization in Reinforcement Learning by doing end to end evaluation of the training and rollout phases of thousands of user submitted code bases in a scalable way. We designed the benchmark on top of the already existing Procgen Benchmark by defining clear tasks and standardizing the end to end evaluation setups. The design aims to maximize the flexibility available for researchers who wish to design future iterations of such benchmarks, and yet imposes necessary practical constraints to allow for a system like this to scale. This paper presents the competition setup and the details and analysis of the top solutions identified through this setup in context of 2020 iteration of the competition at NeurIPS.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 23 of its 30 references that are in the map. First posted on arXiv 29 Mar 2021.

Builds on

23 of 30 references.

References · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Squeeze-and-Excitation Networkscited by 4,763
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  6. 2018Exploration by Random Network Distillationcited by 966
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  8. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  9. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  10. 2020Reinforcement Learning with Augmented Datacited by 452
  11. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  12. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
11 more
  1. 2017Noisy Networks for Explorationcited by 262
  2. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
  3. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  4. 2017Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Ratescited by 196
  5. 2020Phasic Policy Gradientcited by 121
  6. 2018Gotta Learn Fast: A New Benchmark for Generalization in RLcited by 59
  7. 2021The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priorscited by 40
  8. 2019P3O: Policy-on Policy-off Policy Optimizationcited by 37
  9. 2019Working Memory Graphscited by 23
  10. 2020Jelly Bean World: A Testbed for Never-Ending Learningcited by 19
  11. 2018Learning to Run challenge: Synthesizing physiologically accurate motion using deep reinforcement learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 2; 2023: 6; 2024: 2; 2025: 4; 2026: 4.

23 papers.

Sort:

By area:

  1. 2023Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstractioncited by 45
  2. 2021Heuristic-Guided Reinforcement Learningcited by 41
  3. 2021AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learningcited by 39
  4. 2021Procedural Generalization by Planning with Self-Supervised World Modelscited by 30
  5. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
  6. 2022MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learnedcited by 26
  7. 2022Hyperbolic Deep Reinforcement Learningcited by 25
  8. 2021Cross-Trajectory Representation Learning for Zero-Shot Generalization in RLcited by 20
  9. 2021RLOps: Development Life-cycle of Reinforcement Learning Aided Open RANcited by 15
  10. 2023Digital Twin-Enhanced Wireless Indoor Navigation: Achieving Efficient Environment Sensing with Zero-Shot Reinforcement Learningcited by 11
  11. 2023Transformers in Reinforcement Learning: A Surveycited by 11
  12. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11
11 more
  1. 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advancescited by 11
  2. 2025Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigationcited by 4
  3. 2024R3L: Relative Representations for Reinforcement Learningcited by 2
  4. 2023Using Offline Data to Speed Up Reinforcement Learning in Procedurally Generated Environmentscited by 1
  5. 2026CRAX: Fast Safe Reinforcement Learning Benchmarkingcited by 1
  6. 2023Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMOcited by 0
  7. 2025rQdia: Regularizing Q-Value Distributions With Image Augmentationcited by 0
  8. 2025Results of the NeurIPS 2023 Neural MMO Competition on Multi-task Reinforcement Learningcited by 0
  9. 2026PTCG-Bench: Can LLM Agents Master Pok'emon Trading Card Game?cited by 0
  10. 2026Transformer-Enhanced Reinforcement Learning: Fundamentals and Applications in Communication Networkscited by 0
  11. 2026What Does a ProcGen Generalization Gap Measure? Action Rules, Residual Entropy, and the Missing Random Floorcited by 0