mlsanityData to 7 Oct 2026

Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning

arXiv 2105.14780 ↗First posted 31 May 2021cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

The idea of transfer in reinforcement learning (TRL) is intriguing: being able to transfer knowledge from one problem to another problem without learning everything from scratch. This promises quicker learning and learning more complex methods. To gain an insight into the field and to detect emerging trends, we performed a database search. We note a surprisingly late adoption of deep learning that starts in 2018. The introduction of deep learning has not yet solved the greatest challenge of TRL: generalization. Transfer between different domains works well when domains have strong similarities (e.g. MountainCar to Cartpole), and most TRL publications focus on different tasks within the same domain that have few differences. Most TRL applications we encountered compare their improvements against self-defined baselines, and the field is still missing unified benchmarks. We consider this to be a disappointing situation. For the future, we note that: (1) A clear measure of task similarity is needed. (2) Generalization needs to improve. Promising approaches merge deep learning with planning via MCTS or introduce memory through LSTMs. (3) The lack of benchmarking tools will be remedied to enable meaningful comparison and measure progress. Already Alchemy and Meta-World are emerging as interesting benchmark suites. We note that another development, the increase in procedural content generation (PCG), can improve both benchmarking and generalization in TRL.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 32 of its 79 references that are in the map. First posted on arXiv 31 May 2021.

Builds on

32 of 79 references.

References · 32

  1. 2016OpenAI Gymcited by 1,684
  2. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  3. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  5. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  6. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  7. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  8. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  9. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  10. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  11. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 308
  12. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 259
20 more
  1. 2020First return, then explorecited by 256
  2. 2017DARLA: Improving Zero-Shot Transfer in Reinforcement Learningcited by 147
  3. 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 80
  4. 2018Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translationcited by 50
  5. 2018General Video Game AI: a Multi-Track Framework for Evaluating Agents, Games and Content Generation Algorithmscited by 45
  6. 2020Sim2Real Transfer for Reinforcement Learning without Dynamics Randomizationcited by 43
  7. 2021Adversarial Reinforcement Learning for Procedural Content Generationcited by 36
  8. 2018Learning to Search with MCTSnetscited by 32
  9. 2019Assessing Transferability from Simulation to Reality for Reinforcement Learningcited by 29
  10. 2018StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer Learningcited by 25
  11. 2019Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfercited by 24
  12. 2019Autonomous Navigation via Deep Reinforcement Learning for Resource Constraint Edge Nodes using Transfer Learningcited by 19
  13. 2019MULTIPOLAR: Multi-Source Policy Aggregation for Transfer Reinforcement Learning between Diverse Environmental Dynamicscited by 18
  14. 2019Single Episode Policy Transfer in Reinforcement Learningcited by 17
  15. 2019Playing Games in the Dark: An approach for cross-modality transfer in reinforcement learningcited by 9
  16. 2018VPE: Variational Policy Embedding for Transfer Reinforcement Learningcited by 8
  17. 2018Cross-Domain Transfer in Reinforcement Learning using Target Apprenticecited by 6
  18. 2018A Distributed Reinforcement Learning Solution With Knowledge Transfer Capability for A Bike Rebalancing Problemcited by 3
  19. 2020Towards Game-Playing AI Benchmarks via Performance Reporting Standardscited by 3
  20. 2019Evolutionarily-Curated Curriculum Learning for Deep Reinforcement Learning Agentscited by 2

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 2; 2024: 3.

7 papers.

  1. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  2. 2023Bad Habits: Policy Confounding and Out-of-Trajectory Generalization in RLcited by 10
  3. 2023Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Artcited by 1
  4. 2024Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Reviewcited by 1
  5. 2024Enhancing Two-Player Performance Through Single-Player Knowledge Transfer: An Empirical Study on Atari 2600 Gamescited by 1
  6. 2022Reliable validation of Reinforcement Learning Benchmarkscited by 0
  7. 2024A Research Agenda for Usability and Generalisation in Reinforcement Learningcited by 0