mlsanityData to 7 Oct 2026

Open-Ended Reinforcement Learning with Neural Reward Functions

arXiv 2202.08266 ↗First posted 16 Feb 2022cs.LG cs.AI cs.ROcited by 5

Not on the map. Why?

Abstract (arXiv)

Inspired by the great success of unsupervised learning in Computer Vision and Natural Language Processing, the Reinforcement Learning community has recently started to focus more on unsupervised discovery of skills. Most current approaches, like DIAYN or DADS, optimize some form of mutual information objective. We propose a different approach that uses reward functions encoded by neural networks. These are trained iteratively to reward more complex behavior. In high-dimensional robotic environments our approach learns a wide range of interesting skills including front-flips for Half-Cheetah and one-legged running for Humanoid. In the pixel-based Montezuma's Revenge environment our method also works with minimal changes and it learns complex skills that involve interacting with items and visiting diverse locations. The implementation of our approach can be found in this link: https://github.com/amujika/Open-Ended-Reinforcement-Learning-with-Neural-Reward-Functions.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 53 of its 70 references that are in the map. First posted on arXiv 16 Feb 2022.

Builds on

53 of 70 references.

References · 53

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  8. 2017Hindsight Experience Replaycited by 1,096
  9. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  10. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  11. 2018Exploration by Random Network Distillationcited by 966
  12. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
38 more
  1. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  2. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  3. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  4. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  5. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  6. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  7. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 308
  8. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  9. 2019Self-Supervised Exploration via Disagreementcited by 263
  10. 2016Variational Intrinsic Controlcited by 262
  11. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  12. 2020First return, then explorecited by 256
  13. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  14. 2017Automated Curriculum Learning for Neural Networkscited by 251
  15. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  16. 2017Teacher-Student Curriculum Learningcited by 197
  17. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  18. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 182
  19. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  20. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  21. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  22. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
  23. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 141
  24. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
  25. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  26. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  27. 2018Variational Option Discovery Algorithmscited by 121
  28. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 115
  29. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 113
  30. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 103
  31. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  32. 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 100
  33. 2020Automatic Curriculum Learning through Value Disagreementcited by 86
  34. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 86
  35. 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 79
  36. 2019Discovery of Useful Questions as Auxiliary Taskscited by 45
  37. 2017Hindsight policy gradientscited by 36
  38. 2021MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learningcited by 24

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 2.

5 papers.

  1. 2023Cooperative Open-ended Learning Framework for Zero-shot Coordinationcited by 30
  2. 2023Tackling Cooperative Incompatibility for Zero-Shot Human-AI Coordinationcited by 12
  3. 2022Augmentative Topology Agents For Open-Ended Learningcited by 3
  4. 2024HOLA-Drone: Hypergraphic Open-ended Learning for Zero-Shot Multi-Drone Cooperative Pursuitcited by 1
  5. 2024An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problemscited by 0