mlsanityData to 14 Oct 2026

A Research Agenda for Usability and Generalisation in Reinforcement Learning

arXiv 2412.16970 ↗First posted 22 Dec 2024cs.AI stat.MLcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

It is common practice in reinforcement learning (RL) research to train and deploy agents in bespoke simulators, typically implemented by engineers directly in general-purpose programming languages or hardware acceleration frameworks such as CUDA or JAX. This means that programming and engineering expertise is not only required to develop RL algorithms, but is also required to use already developed algorithms for novel problems. The latter poses a problem in terms of the usability of RL, in particular for private individuals and small organisations without substantial engineering expertise. We also perceive this as a challenge for effective generalisation in RL, in the sense that is no standard, shared formalism in which different problems are represented. As we typically have no consistent representation through which to provide information about any novel problem to an agent, our agents also cannot instantly or rapidly generalise to novel problems. In this position paper, we advocate for a research agenda centred around the use of user-friendly description languages for describing problems, such that (i) users with little to no engineering expertise can formally describe the problems they would like to be tackled by RL algorithms, and (ii) algorithms can leverage problem descriptions to effectively generalise among all problems describable in the language of choice.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 111 of its 167 references that are in the map. First posted on arXiv 22 Dec 2024.

Builds on

111 of 167 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Can we hop in general? A discussion of benchmark selection and design using the Hopper environment11 Oct
  2. 2024Can Learned Optimization Make Reinforcement Learning Less Difficult?9 Jul
  3. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learning25 Jun
  4. 2024Position: Benchmarking is Limited in Reinforcement Learning Research23 Jun

Other references · 107

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  7. 2023A Survey of Large Language Modelscited by 2,576
  8. 2016OpenAI Gymcited by 1,736
  9. 2016Progressive Neural Networkscited by 1,361
  10. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  11. 2018DeepMind Control Suitecited by 956
  12. 2022A Generalist Agentcited by 808
38 more
  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  2. 2017Deep Reinforcement Learning that Matterscited by 754
  3. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  4. 2023Learning Interactive Real-World Simulatorscited by 458
  5. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  6. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  7. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  8. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  9. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 276
  10. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  11. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
  12. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 265
  13. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 264
  14. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  15. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 181
  16. 2019Gradio: Hassle-Free Sharing and Testing of ML Models in the Wildcited by 178
  17. 2018A Study on Overfitting in Deep Reinforcement Learningcited by 174
  18. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  19. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  20. 2020The Hardware Lotterycited by 160
  21. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 160
  22. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  23. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
  24. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 136
  25. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
  26. 2018Generalization and Regularization in DQNcited by 118
  27. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 109
  28. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 100
  29. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 98
  30. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 97
  31. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 97
  32. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  33. 2018A Dissection of Overfitting and Generalization in Continuous Reinforcement Learningcited by 87
  34. 2017Mapping Instructions and Visual Observations to Actions with Reinforcement Learningcited by 81
  35. 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 81
  36. 2020Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Researchcited by 78
  37. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  38. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70

50 most cited of 107 All 107 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Generalization in Online Reinforcement Learning for Mobile Agents
  2. 2024A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedback
  3. 2026Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions
  4. 2024Towards a Domain-Specific Modelling Environment for Reinforcement Learning
  5. 2023Foundations for Transfer in Reinforcement Learning: A Taxonomy of Knowledge Modalities
  6. 2026Reinforcement Learning in Operational Research: A Technical Review and Practical Roadmap
  7. 2024Programmatic Reinforcement Learning: Navigating Gridworlds
  8. 2025TalkToAgent: A Human-centric Explanation of Reinforcement Learning Agents with Large Language Models